@mhvk question for you: why do you check for __array_ufunc__ only on the type object in has_non_default_array_ufunc? In practice, I doubt this makes a difference, but it seems slightly simpler and certainly more expected to just check on the object directly.

@shoyer: "expected" is tricky here :-). Most Python users expect that methods are looked up by doing attribute access on the object, like... methods are. But (some) expert Python users will expect that __special__ methods are looked up on the class directly, because that's how all the language's built-in special methods work:

In [1]: class Foo:
   ...:     def __add__(self, other):
   ...:         return "class"
   ...:     

In [2]: f = Foo()

In [3]: f.__add__ = lambda other: "object"

In [4]: f + f
Out[4]: 'class'

In [5]: f.__add__(f)
Out[5]: 'object'

I think this was originally partly an optimization and partly a hold-over from the whole C-level slots system that partly dates back to ancient days of Python when C classes and Python classes were totally different things, but it's standard enough now that PEPs regularly document it as part of the formal semantics for new features, e.g. check out the calls to type here.

OTOH I'm pretty sure numpy doesn't do this for other special attributes like __array_priority__, so I guess the argument from consistency goes both ways...

One interesting question is whether or not doing the lookup directly on the class is faster – object attr lookup is fairly complicated, and this does skip some steps. And lookup speed for special attrs is something we've found worth optimizing in the past.

@njsmith OK, fair enough. It seems totally reasonable to me to do the override lookup on the class for the sake of speed given that this check is in the inner loop of essentially every numpy operation. But in that case, we should document it, and also be consistent in PyUFunc_CheckOverride, where currently we look-up the __array_ufunc__ attribute on the object, not the class. Currently, we look on the class when checking for the existence of an override, but look on the object to actually call it.

The reason I made it look up __array_ufunc__ on the class is a practical one -- only that way can it be compared with ndarray.__array_ufunc__ (on an instance, the method gets wrapped, so one cannot compare equality).

As for then later looking it up on the object: I did consider just keeping the override itself instead of the object defining an override, but one still has to do the subclass check, so it didn't seem quite worth it. That said, I do think there is substantial room for optimization...

Overall, I'm still not sure this is a good idea; see #9011 (comment) and the following comment. That said, much of the clarification of intent in the NEP is great, in particular that if np.ufunc(self, other) is defined and there is a corresponding operator.func(self, other), they should do the same thing.

@shoyer I'd like to put #9021 in, followed by this, and then update the documentation again.

#9021 is in.

@mhvk I think it should be possible to relax this choice in the future if a need to do so turns up. My own sense is that if a class wants to be handled by subclasses only, it should use the python python ops or __array_ufunc__ in its full glory. The proposed mixins should make the latter fairly easy.

@charris - fair enough, it is easier to allow than to disallow later.

I have added a preliminary adjustments to the docs on __array_ufunc__ into this PR, but have split the NEP updates into #9027.

Couple of minor suggestions for documentation, otherwise LGTM.

Having read the revised NEP, I feel this is a fine approach (even if I still slightly prefer the current state), so OK to merge as far as I'm concerned.

I fixed up my commits per @charris's review. I'll wait for CI to pass and then merge.

Beat you to it. Thanks Stephan.