Oh, I think I know what's going on: in 1.9, a += b ignores __array_priority__ if b is an ndarray subclass (maybe with some slight caveats depending on dtype), but it does respect __array_priority__ if b is not an ndarray subclass (b/c the ufunc machinery ends up treating it like an object array, which is why we missed it before).

Hmm, the case of a non-array is also the only one for which I think support for raising NotImplemented is not crazy. In 1.9,

import astropy.units as u, numpy as np
a=np.arange(10.)
a*=u.m
a
# <Quantity [ 0., 1., 2., 3., 4., 5., 6., 7., 8., 9.] m>

In more-or-less current master, one gets

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-16555c0ea841> in <module>()
----> 1 a*=u.m

TypeError: ufunc 'multiply' output (typecode 'O') could not be coerced to provided output parameter (typecode 'd') according to the casting rule ''same_kind''

I don't particularly mind this eventually becoming impossible, but it seems a bit pointless to force a change now, when we're otherwise continuing to support the old __array_priority__ behaviour. So, maybe remain backwards compatible for now and postpone decisions until __array_priority__ will be deprecated.

I have a use-case very similar to @mhvk's demo there (actually a different unit package, with the same type of behavior). I actually just discovered __array_priority__ and was happy to see that it had the intended effect (i.e., that I could override np.ndarray's insistence that it override the multiplication operator for absolutely everything).

If __array_priority__ is going away, or is currently broken and won't be fixed, what is the alternative to having some type's __rmul__ taking precedence over np.ndarray's __mul__?

support for raising NotImplemented

@mhvk I haven't been following this or related threads, but is that a typo? Should it be either raise NotImplementedError or return NotImplemented? This seems like nitpicking, but this same typo seems to have stalled or killed PRs like https://github.com/scipy/scipy/pull/4066/files#diff-e35a0c376fa44a58cd43220b1660d256R319.

@agriffing: he meant return NotImplemented.
.
@mhvk: yeah, OK, I rather thought so.
.
I think we add a check to the inplace ops like:
if not isinstance(x, np.ndarray) and x.array_priority >
self.array_priority_:
deprecation warning
return NotImplemented
On Jun 26, 2015 7:44 AM, "argriffing" notifications@github.com wrote:

support for raising NotImplemented

@mhvk https://github.com/mhvk I haven't been following this or related
threads, but is that a typo? Should it be either raise NotImplementedError
or return NotImplemented? This seems like nitpicking, but this same typo
seems to have stalled or killed PRs like
https://github.com/scipy/scipy/pull/4066/files#diff-e35a0c376fa44a58cd43220b1660d256R319
.

—
Reply to this email directly or view it on GitHub
#6020 (comment).

@argriffing - yes, returning NotImplemented it should have been.

@njsmith - bit nitpicky, but I'd leave the deprecation warning outside of this issue at least; even though I agree it is risky usage, I think it should be added only when we start deprecating __array_priority__ itself.

@mhvk: hmm, my reasoning is, this isn't a deprecation of
array_priority, it's a deprecation of silently converting inplace to
out-of-place. Even if we changed our minds about array_priority and
decided to keep it in general, then we'd still want to fix this specific
(weird and inconsistent and clearly accidental) behavior.
On Jun 26, 2015 8:07 AM, "Marten van Kerkwijk" notifications@github.com
wrote:

@argriffing https://github.com/argriffing - yes, returning
NotImplemented it should have been.

@njsmith https://github.com/njsmith - bit nitpicky, but I'd leave the
deprecation warning outside of this issue at least; even though I agree it
is risky usage, I think it should be added only when we start deprecating
array_priority itself.

—
Reply to this email directly or view it on GitHub
#6020 (comment).

I may be weird and accidental behaviour that might well be counted on.... It also strikes to one of the main points in the long discussion in #5844, where there was a lot of arguing that for non-array classes, one should just default to standard python logic. If at all possible, I think we should just fix the regression here, and discuss possible deprecation warnings in #6001.

I agree with Martin here

I'm fine with not deprecating __array_priority__ in 1.10. (Just posted to say the same on #6001.) But this bug and #6001 are basically unrelated; they both touch on binop dispatch, but different aspects of dispatch and the issues at stake are different. I want a deprecation warning here so we can fix __array_priority__, because the 1.9 behaviour is simply buggy.

But I don't want to block anything further for that, so tell you what -- as soon as #6001 gets some resolution, I'll post a PR that just restores the 1.9 behavior, and then we can have a whole separate, focused discussion about what to do with __array_priority__ and in-place ops, which won't have to block 1.10.

(Unfortunately I can't post a PR fixing this immediately because I just tried, and discovered it'd have massive conflicts with #6001. So we're waiting on #6001. Fortunately this should only take a few minutes to sort out once #6001 gets resolved.)

OK, this looks like the only thing really blocking 1.11. We should make a decision here, even though 1.10 is already out.

Well, obviously the walls didn't come tumbling down when this regression hit 1.10, though on the other hand it is unclear what fraction of the numpy users is now using 1.10 (e.g., it landed in Debian testing only a few days ago). Overall, I'd still prefer there to be at least one way for ensuring ndarray behaves like other python objects, i.e., that it always returns NotImplemented, so I'd like it to do as @njsmith proposed above, and would suggest not having a deprecation warning until we actually start deprecating __array_priority__ itself.

  if not isinstance(x, np.ndarray) and x.__array_priority__ > self.__array_priority___:
      return NotImplemented

The issue here is a case where we're silently converting inplace ops into out of place ops, which is really inconsistent with numpy's general philosophy (unnumpythonic?). I was fine with deferring that discussion until later, and we definitely should have done a standard deprecation period once we got around to it. But now it appears that we've gone ahead and shipped the fixed behavior in 1.10, and no one noticed. Oops. That's unfortunate, but given this, I'm not really excited about the idea of adding back the broken behavior in 1.11 just so we can deprecate it and remove it again later. Either way, it's not going to be something that downstream packages can count on; changing it now just creates more churn IMO.

(Also, delaying the release because of last minute things like this is just an unfortunate pattern to get into. If no one brought it up during the whole 1.11 cycle, then maybe it can wait until 1.12? That's only a few months away in the new scheme.)

OK, lets kick this can further down the road. We need to get back to the whole __numpy_ufunc__ thing in any case.

Yes, fine by me; I'll probably continue to argue for having an option to tell ndarray to behave pythonic, but it seems fine to not let that be __array_priority__.

Feels like this might still be true, but with the implementation and adoption of __array_function__ its probably blown over and should be reopened if an actual problem. Closing.

@seberg - yes, correct to close; for reference for anyone who stumbles on this: the way to tell numpy to not be greedy is to define __array_ufunc__ = None (this is what we do for our units, so that a *= u.m does make a a Quantity). And if one also needs __array_ufunc__, one can override there.