MAINT: remove `numpy.array_api` module #25911

rgommers · 2024-03-01T14:08:22Z

This submodule was added in NumPy 1.22 under an experimental label. After gaining experience with it, we decided that it was quite useful for testing code in downstream libraries and from end users written against the array API standard, but not for production code. Now that we have array API standard support in the main namespace, we therefore decided to remove this submodule - as documented in NEP 56.

The code from numpy.array_api was moved to the array-api-strict package (see https://github.com/data-apis/array-api-strict) and further improved there. Users are advised to start using that standalone package for testing their code if they want to check that it works not only with numpy but with other libraries that support the array API standard.

This commit also updates the entrypoint (used by for example hypothesis) to point to the main namespace. Users of this entrypoint to obtain NumPy's compliant namespace should not be affected by this change.

asmeurer · 2024-03-01T23:06:52Z

doc/source/reference/array_api.rst

I've linked to this document in quite a few places. Obviously with numpy 2.0, most of what was here is no longer true. Is it worth keeping the contents here around somewhere (maybe in the array-api-strict docs)?

It will remain available at https://numpy.org/doc/1.26/reference/array_api.html#table-of-differences-between-numpy-array-api-and-numpy. How about I add that link to this page?

Added a link to NEP 47, and cross-linked NEP 47 and 56.

I'll also have to follow up on the mailing list about changing the status of NEPs (superseded, accepted).

asmeurer · 2024-03-01T23:08:14Z

doc/source/reference/array_api.rst

-   main ``numpy`` namespace would not need to change in any way to be
-   spec-compatible for these.
+NumPy installs an `entry point <https://packaging.python.org/en/latest/specifications/entry-points/>`__
+that can be used for discovery purposes::


I never noticed that this was actually added to the standard. I should probably add it to array-api-compat and array-api-strict.

It's marked as optional and not in the API docs, since it isn't actually an API but a packaging thing. Maybe we should improve its visibility though?

asmeurer · 2024-03-01T23:09:27Z

Obviously +1 to the overall change here.

I don't know how NEPs work, but should the NEP that proposed numpy.array_api be updated?

rgommers · 2024-03-03T09:20:37Z

I don't know how NEPs work, but should the NEP that proposed numpy.array_api be updated?

Good point, I'll mark it as superseded and add a little note.

mattip · 2024-03-03T09:52:31Z

The macos failure seems relevant

from the log

>       xp = ep.load()

ep         = EntryPoint(name='numpy', value='numpy.array_api', group='array_api')
eps        = {'array_api': [EntryPoint(name='numpy', value='numpy.array_api', group='array_api')], 'babel.checkers': [EntryPoint(na...hinx_theme', group='sphinx.html_themes'), EntryPoint(name='alabaster', value='alabaster', group='sphinx.html_themes')]}
numpy_in_sitepackages = False
xp_eps     = [EntryPoint(name='numpy', value='numpy.array_api', group='array_api')]

numpy/tests/test_public_api.py:482: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/Users/runner/miniconda3/envs/numpy-dev/lib/python3.11/importlib/metadata/__init__.py:202: in load
    module = import_module(match.group('module'))
        match      = <re.Match object; span=(0, 15), match='numpy.array_api'>
        self       = EntryPoint(name='numpy', value='numpy.array_api', group='array_api')
/Users/runner/miniconda3/envs/numpy-dev/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
        level      = 0
        name       = 'numpy.array_api'
        package    = None
<frozen importlib._bootstrap>:1204: in _gcd_import
    ???
        level      = 0
        name       = 'numpy.array_api'
        package    = None
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
        import_    = <function _gcd_import at 0x1031cbd80>
        module     = <object object at 0x103200050>
        name       = 'numpy.array_api'
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = 'numpy.array_api', import_ = <function _gcd_import at 0x1031cbd80>

>   ???
E   ModuleNotFoundError: No module named 'numpy.array_api'

child      = 'array_api'
import_    = <function _gcd_import at 0x1031cbd80>
name       = 'numpy.array_api'
parent     = 'numpy'
parent_module = <module 'numpy' from '/Users/runner/work/numpy/numpy/build-install/usr/lib/python3.11/site-packages/numpy/__init__.py'>
parent_spec = ModuleSpec(name='numpy', loader=<_frozen_importlib_external.SourceFileLoader object at 0x104735110>, origin='/Users/ru...y', submodule_search_locations=['/Users/runner/work/numpy/numpy/build-install/usr/lib/python3.11/site-packages/numpy'])
path       = ['/Users/runner/work/numpy/numpy/build-install/usr/lib/python3.11/site-packages/numpy']
spec       = None

<frozen importlib._bootstrap>:1140: ModuleNotFoundError
=========================== short test summary info ============================
FAILED numpy/tests/test_public_api.py::test_array_api_entry_point - ModuleNotFoundError: No module named 'numpy.array_api'

rgommers · 2024-03-03T15:28:35Z

For the test failure on macOS, what is happening is that the entrypoint is not installed by spin (it doesn't know how to do that), but because the job uses environment.yml there is another numpy package (1.26.4) installed which does install the entrypoint, pointing to the now-removed submodule. This is kinda an invalid situation, so we just need to recognize it and not try to load the entrypoint in that case.

This submodule was added in NumPy 1.22 under an experimental label. After gaining experience with it, we decided that it was quite useful for testing code in downstream libraries and from end users written against the array API standard, but not for production code. Now that we have array API standard support in the main namespace, we therefore decided to remove this submodule - as documented in NEP 56 The code from `numpy.array_api` was moved to the `array-api-strict` package (see https://github.com/data-apis/array-api-strict) and further improved there. Users are advised to start using that standalone package for testing their code if they want to check that it works not only with `numpy` but with other libraries that support the array API standard. This commit also updates the entrypoint (used by for example `hypothesis`) to point to the main namespace. Users of this entrypoint to obtain NumPy's compliant namespace should not be affected by this change. [skip cirrus]

[skip cirrus]

mattip

LGTM, once CI is passing.

mattip · 2024-03-03T16:03:33Z

numpy/tests/test_public_api.py

+        # This isn't a problem except for when running tests with `spin` or an
+        # in-place build.
+        return
+


rgommers · 2024-03-03T16:31:48Z

CI is happy aside from an unrelated connection issue on the FreeBSD job. PyPy is still running but also happy so far and passed on the last run already. Let's call it good. Thanks for the reviews Matti and Aaron!

rgommers added 03 - Maintenance component: numpy.array_api labels Mar 1, 2024

rgommers added this to the 2.0.0 release milestone Mar 1, 2024

This was referenced Mar 1, 2024

Removing numpy.array_api import cubed-dev/cubed#407

Closed

MAINT: convert numpy.array_api to array-api-strict scikit-learn/scikit-learn#28555

Merged

asmeurer reviewed Mar 1, 2024

View reviewed changes

rgommers mentioned this pull request Mar 3, 2024

DOC: Fix some document build errors about rst markups #25915

Merged

rgommers mentioned this pull request Mar 3, 2024

Coordination: last items before branching 2.0.x #25918

Closed

10 tasks

rgommers added 4 commits March 3, 2024 16:30

DOC: update the page on array API standard compatibility

dc1dbdb

DOC: update release notes for array API standard support changes

28f4b29

[skip cirrus]

NEP: mark NEP 47 as superseded; add note with context for future readers

9df3add

rgommers force-pushed the remove-arrayapi-module branch from ba6a867 to 9df3add Compare March 3, 2024 15:45

mattip approved these changes Mar 3, 2024

View reviewed changes

numpy/tests/test_public_api.py

# This isn't a problem except for when running tests with `spin` or an

# in-place build.

return

Copy link

Member

mattip Mar 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool.

rgommers merged commit 8d9218d into numpy:main Mar 3, 2024

rgommers deleted the remove-arrayapi-module branch March 3, 2024 16:31

neutrinoceros mentioned this pull request Mar 4, 2024

TST: skip testing numpy.array_api in numpy 2.0 (module was removed) astropy/astropy#16148

Closed

1 task

Illviljan mentioned this pull request Mar 22, 2024

array api-related upstream-dev failures pydata/xarray#8854

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MAINT: remove `numpy.array_api` module #25911

MAINT: remove `numpy.array_api` module #25911

rgommers commented Mar 1, 2024

asmeurer Mar 1, 2024

rgommers Mar 3, 2024

rgommers Mar 3, 2024

asmeurer Mar 1, 2024

rgommers Mar 3, 2024

asmeurer commented Mar 1, 2024

rgommers commented Mar 3, 2024

mattip commented Mar 3, 2024

rgommers commented Mar 3, 2024

mattip left a comment

mattip Mar 3, 2024

rgommers commented Mar 3, 2024

MAINT: remove numpy.array_api module #25911

MAINT: remove numpy.array_api module #25911

Conversation

rgommers commented Mar 1, 2024

asmeurer Mar 1, 2024

Choose a reason for hiding this comment

rgommers Mar 3, 2024

Choose a reason for hiding this comment

rgommers Mar 3, 2024

Choose a reason for hiding this comment

asmeurer Mar 1, 2024

Choose a reason for hiding this comment

rgommers Mar 3, 2024

Choose a reason for hiding this comment

asmeurer commented Mar 1, 2024

rgommers commented Mar 3, 2024

mattip commented Mar 3, 2024

rgommers commented Mar 3, 2024

mattip left a comment

Choose a reason for hiding this comment

mattip Mar 3, 2024

Choose a reason for hiding this comment

rgommers commented Mar 3, 2024

MAINT: remove `numpy.array_api` module #25911

MAINT: remove `numpy.array_api` module #25911