OK, I think this has decent test coverage now and is ready for review.

This test is failing on Python 2.7 with Shippable:

________________________ test_suppress_warnings_module _________________________
[gw1] linux2 -- Python 2.7.12 /root/venv/2.7/bin/python

    def test_suppress_warnings_module():
        # Initial state of module, no warnings
        my_mod = _get_fresh_mod()
        assert_equal(getattr(my_mod, '__warningregistry__', {}), {})
    
        def warn_other_module():
            # Apply along axis is implemented in python; stacklevel=2 means
            # we end up inside its module, not ours.
            def warn(arr):
                warnings.warn("Some warning 2", stacklevel=2)
                return arr
            np.apply_along_axis(warn, 0, [0])
    
        # Test module based warning suppression:
>       assert_warn_len_equal(my_mod, 0)

It looks unrelated.

It looks unrelated.

Correct, the warnings tests are not thread-safe & we run the tests with more than 1 core on shippable CI; I can't remember how much extra time it takes to run the suite on a single core, but that one test will fail stochastically so we should maybe roll back to 1 core, which is a bit of a sin since we get 96 cores on those ARM nodes.

Maybe we could xfail on some hook related to n_cores > 1 for pytest run for that particular test.

for detecting master vs. "child" process with a fixture: pytest-dev/pytest-xdist#47 (comment)

Could you add a release note with a short explanation and a link to the NEP.

but I think it's a little early for that.

I believe there is already some experimental support out there. Was it @mhvk? Might get together and pick a function.

Is it intended that ndarray methods can be overridden?

Eventually I'd like the release note to read "Every core NumPy function now supports __array_function__." But there clearly not true now :).

Is it intended that ndarray methods can be overridden?

No, this is only for functions.

The "subclasses before superclasses" rule coupled with the "special handling for ndarray" rule makes the logic for picking types to call pretty tricky.

In particular, in which order should we try overrides for arguments with types [ndarray, duck_array, ndarray_subclass]? Per the text in the NEP, I suspect we should call ndarray_subclass.__array_function__ before duck_array.__array_function__ because we say we mention "subclasses before superclasses" before "special handling for ndarray", but honestly it's a little ambiguous.

@shoyer - One way of thinking about it is that ndarray actually does not get special treatment - the only reason we skip it in this case is because we already know it will return NotImplemented. I.e., our logic is actually quite simple, but we allow shortcuts.

One way of thinking about it is that ndarray actually does not get special treatment - the only reason we skip it in this case is because we already know it will return NotImplemented. I.e., our logic is actually quite simple, but we allow shortcuts.

Yes, this helps.

@shoyer - most of the diff now seems to be against benchmarks - surely not the intent!

I'll split removing the Benchmark baseclass into another PR.

I removed the unrelated changes to the benchmark suite.

This update has made the code quite a bit clearer; the different split between the actual override call is particularly helpful.
Is it time to start trying this out, e.g., on concatenate?

+1 to merge this as-is as step 1, and to add a tracking issue to follow up with the other two goals in the first comment

see #12028 for a tracking issue

Let's get this started. @shoyer Might want to break the tracking issue down into smaller bits. It would also be good to have some idea of how to gather feedback, and maybe some small demo project (masked arrays, TNG?) or existing project as a test platform. Also might consider long term possibility of JIT for python code, that is, how much does a C implementation actually buy us.

Thanks Stephan.