-
-
Notifications
You must be signed in to change notification settings - Fork 10.8k
BUG: revise string_from_pyobj/try_pyarr_from_string with respect to malloc and copy. #18759
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Note that you can simplify by using raw strings to dispense with most of the escaped characters.
Just avoid having more than three |
We also prefer the style
Although it is hard to enforce with outside code coming in. |
Applied the style to the changeset of this PR. |
I have marked this PR as a draft because the handling of nulls is still inconsistent. For instance, the second item in the output of #15311 (comment) is unexpected: all the nulls are replaced except the last one resulting I am starting to think that the current approach of replacing nulls with spaces is wrong. IIRC, the approach tries to fix a Fortran I/O issue where nulls are not included in the output. But this issue appears to be a non-issue: the fixed-size character arrays have the correct width, just when outputting it, null values take zero width. So my plan is to use the same interpretation of nulls that numpy uses. For instance:
The trailing nulls are not included in the repr output of the ndarray object while these exist as seen in the bytes buffer output. This may be a BC sensitive change and requires updating f2py docs but the docs are broken anyway: #15311 |
I've very little experience with fortran, and have no idea why replacing the nulls with spaces is a good idea. I'll defer to your judgement here. |
In general, stripping all trailing nulls seems like the safest bet to me as well (including not replacing NULLs, although I have no clue about possible backward incompatibility). Using NULL bytes inside the string is fairly brittle to begin with, so I am willing to make the bet that the combination of NULL bytes and string wrapping just isn't a thing (and if it is will fail loudly). |
e2a3f6e
to
7f5523f
Compare
If nobody will object, I'll merge this PR in two days in case there is no further reviewing activity. |
Co-authored-by: Eric Wieser <wieser.eric@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good now, thanks - just a few minor style comments, and a possible follow-up
@eric-wieser thanks for the review! Here's more follow-up:
|
The scipy tests with the "nightly" numpy wheel are experiencing Python interpreter crashes in the code that uses the Fortran library L-BFGS-B (see scipy/scipy#14203). I created a separate issue for this: #19201 |
@pearu @melissawm I'd like to revert this PR until gh-19201. The bug is most likely in this PR; even if it's not and the bug is in SciPy then we still have a problem because leaving this in breaks the most recent SciPy release; and this has broken SciPy CI which prevents working on a release blocker. It would also be good to use the SciPy test suite as the |
Fixes #18431.