Skip to content

Conversation

cmarmo
Copy link
Contributor

@cmarmo cmarmo commented Jul 26, 2022

This pull requests makes mask_invalid consistent with mask_where when copy is set to False.
Fixes #19332.

I'd rather make the two functions consistent than change the documentation.
Also from the documentation

Only applies to arrays with a dtype where NaNs or infs make sense

I added a test checking that an error is thrown when isinfinite is not applicable.

Thanks for considering it.

@cmarmo cmarmo changed the title Make mask_invalid consistent with mask_where if copy is set to False BUG: Make mask_invalid consistent with mask_where if copy is set to False Aug 1, 2022
numpy/ma/core.py Outdated
Comment on lines 2360 to 2363
try:
return masked_where(~(np.isfinite(getdata(a))), a, copy=copy)
except TypeError:
raise
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was there a particular reason for this try/except, or is it a debug remnant?

Suggested change
try:
return masked_where(~(np.isfinite(getdata(a))), a, copy=copy)
except TypeError:
raise
return masked_where(~(np.isfinite(getdata(a))), a, copy=copy)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, I can't recall why I put the try/except. Suggestion applied. Thanks!

with pytest.raises(TypeError,
match="not supported for the input types"):
np.ma.masked_invalid(a)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We were looking at this in the triage meeting as well. I guess the test is great (I honestly still need to parse it fully).
But we are missing an additional test for the successful path that was fixed I think: Checking that the array was indeed modified in-place with copy=False?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added a new test in test_masked_array_no_copy() ... just to explain myself: I didn't add it in the first place because mask_invalid becomes a straight call of masked_where which is already tested. But I guess the more tests we have the better?
Thanks!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, thanks. It is true that masked_where is tested, but it is also just nice to see the fix in action in the PR and we mainly have integration tests anyway.

A test too much cannot hurt :).

I find it odd that we consider inf invalid by default, but that is not a change.

Thanks @cmarmo, lets get this in!

@seberg
Copy link
Member

seberg commented Sep 22, 2022

I suppose we should do this just to align things anyway...

Although, now I actually suspect that this may have been intentional (Chesterton's fence greets): The function always replaces the mask, but does not always copy the data, which is a pattern to the madness that does make sense, the docs are probably just fuzzy on that intention (if it was the intention).
Sorry, long shot, but @ahaldane do you happen to have a gut feeling on this?

@InessaPawson
Copy link
Member

@mhvk Do you have any thoughts on this?

@mhvk
Copy link
Contributor

mhvk commented Oct 5, 2022

Not really. Overall, to me this PR makes sense: do as the doc states and just call masked_where. I've never understood why if data is kept, masks are not, though clearly it was designed that way.

@seberg
Copy link
Member

seberg commented Oct 5, 2022

Thanks @cmarmo and Marten, lets give this a shot then!

@seberg seberg merged commit 02b68f1 into numpy:main Oct 5, 2022
@charris
Copy link
Member

charris commented Oct 5, 2022

Should probably add a release note for this.

@seberg
Copy link
Member

seberg commented Oct 5, 2022

Hmmm, maybe better. @cmarmo are you interested in having a look at that, the instructions are in numpy/doc/release/upcoming_changes/README.txt (you basically add a file in that folder with a 22046.change.rst as name).

Otherwise, hopefully I will remember to follow up and do it :).

@bsipocz bsipocz added the 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes label Oct 5, 2022
@bsipocz
Copy link
Member

bsipocz commented Oct 5, 2022

Hmm, if the label is used consistently, a script could help to look for missing release notes before release time?

@cmarmo
Copy link
Contributor Author

cmarmo commented Oct 5, 2022

@cmarmo are you interested in having a look at that, the instructions are in numpy/doc/release/upcoming_changes/README.txt (you basically add a file in that folder with a 22046.change.rst as name).

I can take care of that if it can wait until the week-end. :)

@cmarmo cmarmo deleted the masked-invalid branch October 8, 2022 07:01
seberg pushed a commit that referenced this pull request Oct 14, 2022
This pull request add the changelog for #22046.
seberg added a commit to seberg/numpy that referenced this pull request Dec 19, 2022
This is the minimal solution to fix numpygh-22826 with as little change
as possible.
We should fix `getdata()` but I don't want to do that in a bug-fix
release really.

IMO the alternative is to revert numpygh-22046 which would also revert
the behavior noticed in numpygh-22720  (which seems less harmful though).

Closes numpygh-22826
charris pushed a commit to charris/numpy that referenced this pull request Dec 19, 2022
This is the minimal solution to fix numpygh-22826 with as little change
as possible.
We should fix `getdata()` but I don't want to do that in a bug-fix
release really.

IMO the alternative is to revert numpygh-22046 which would also revert
the behavior noticed in numpygh-22720  (which seems less harmful though).

Closes numpygh-22826
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
00 - Bug 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: ma.masked_invalid(a, copy=False) does not modify a
6 participants