Skip to content

BUG: revise string_from_pyobj/try_pyarr_from_string with respect to malloc and copy (the second round) #19251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 17, 2021

Conversation

pearu
Copy link
Contributor

@pearu pearu commented Jun 15, 2021

Improved #18759 that was reverted by #19235 .

Originally, it was assumed that Fortran treats null values in byte-strings similar to C but that turned out to be wrong and caused failures in scipy tests that use f2py wrapped functions with string intent(inout) arguments for Fortran fixed-width character string arguments.

This PR (in addition to original changes) replaces the trailing nulls in strings with trailing blanks that Fortran treats as "null values" and when Fortran program returns a string with trailing blanks, these will be replaced with trailing nulls because the content of numpy ndarray of strings is interpreted as C strings.

Fixes #18431 (again) and #19201 .

This PR is tested against the current scipy master: all scipy tests pass ok.

@seberg
Copy link
Member

seberg commented Jun 15, 2021

On that issue, I was a bit curious if f2py could actually shorten the string length passed to fortran (so it represents the actual length without any padding). That probably achieves the identical behavior as padding with spaces though.
(In theory, that might allow avoiding the additional copy/buffer, but that would probably be an API change.)

@pearu
Copy link
Contributor Author

pearu commented Jun 15, 2021

On that issue, I was a bit curious if f2py could actually shorten the string length passed to fortran (so it represents the actual length without any padding). That probably achieves the identical behavior as padding with spaces though.

I think this would not work with intent(inout) or intent(out) arguments because if a Fortran program expects a fixed-length character string then f2py must provide memory with the required size (even when initially filled with blanks or nulls) because the Fortran program is allowed to write (read: index) to the string till the end of it.

UPDATE: it would not even work for intent(in) arguments because the Fortran program is allowed to read (read: index) the fixed-width string till the end of it.

@pearu pearu linked an issue Jun 15, 2021 that may be closed by this pull request
@charris charris merged commit 53d74a1 into numpy:main Jun 17, 2021
@charris
Copy link
Member

charris commented Jun 17, 2021

Let's give this a shot. Thanks Pearu.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SciPy crash with nightly numpy wheel--possible f2py issue BUG: F2Py Invalid read in string_from_pyobj
3 participants