I don't know that I like this: in general, python is fairly consistent in just giving you a direct link, so that you can do tests like a is b, and that comes with the responsibility to not just screw things up by assuming you can mutate (as in directories passed through). I think the analogy with data is not quite correct, since that really returns a different type.

Separately, I don't think we should ever suggest to use private attributes.

The other argument for making this change is it gives us the freedom to swap out the implementation for one that computes the mask on the fly - such as how x.imag is x.imag also returns False.

It seems pretty apparent to me that is is a bad way to detect "is a view of the same data" in numpy. Perhaps we need a np.is_full_view = lambda x, y: x.__array_interface__ == y.__array_interface__ or similar

Hmm, looking more at #10270, I see at least the rationale. And I see that __array_finalize__ happily changes the shape of the mask it got from obj. But from an admittedly not in-depth inspection, it is also the only place: everywhere else special care is taken to view or copy the mask properly.

That this PR updates mostly tests seems to indicate that this will break user code. So I share @mhvk's concern.

Also, these are the things that should be discussed on the mailing list before merging, since it's an intentional backwards compat break.

seems to indicate that this will break user code.

Probably mostly user tests. Only code relying on the identity of arr.mask will fail.

should be discussed on the mailing list before merging

Agreed - I figured I'd make the PR first to work out exactly what needs to change

should be discussed on the mailing list before merging

Agreed - I figured I'd make the PR first to work out exactly what needs to change

Yeah, good idea - is a useful strategy especially if the PR is not hard. Easier to discuss a concrete rather than an abstract change.

Something else that is slightly illogical is that this changes only how ma.mask behaves - if you use np.ma.getmask(ma) you would still get the mask directly. Logically, it would seem that, if anything, this should be the reverse. But if one changes the latter to return a view, lots more tests break.

Or, more particularly, a number of additional tests break if one changes this in __array_finalize__ - the one place where really one should not be meddling with the mask shape and have it propagate back to obj! Looking deeper into this: there is actually an incredible hack - it makes shape-setting work by the following rather horrible logic:

1. ma.shape = <shape> calls the base ndarray.shape setter (getset.c, array_shape_set).
2. This internally calls PyArray_Reshape, i.e., effectively new = np.ndarray.reshape(ma, <shape>).
3. Which calls new.__array_finalize__(ma), and therefore mangles ma.mask
4. Upon success, the c code copies new.shape to ma.shape, but does not touch the mask.
5. Since the mask is already changed, this works.

I'm not sure I understand how _mask fits in there - are you sure that the .view(np.ndarray) actually happens?

Sorry that was not clear: I should have written np.ndarray.reshape(ma, <shape>) (will edit above) - this calls __array_finalize__.

I saw a comment with an example of b=MaskedArray(1., mask=masked.mask) fly by but don't know where - this would seem a bug in the initializer, which should always take a new view of the mask.

Fixing __array_finalize__ does not seem to be entirely trivial, sadly. Really, we should be ensuring ma.shape does the right thing, but it's not enough. An initial attempt is at #10314.

Needs rebasing and changing the release note update from 1.15 to 1.17

close/reopen