Skip to content

[NumPy] use NumPy 2.x in CI #158647

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 11 commits into
base: gh/XuehaiPan/370/base
Choose a base branch
from

Conversation

XuehaiPan
Copy link
Collaborator

@XuehaiPan XuehaiPan commented Jul 18, 2025

Stack from ghstack (oldest at bottom):

cc @seemethere @malfet @pytorch/pytorch-dev-infra @mruberry @rgommers

[ghstack-poisoned]
Copy link

pytorch-bot bot commented Jul 18, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158647

Note: Links to docs will display an error until the docs builds have been completed.

❌ 38 New Failures, 1 Unrelated Failure

As of commit bc4e307 with merge base 01f66d0 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the release notes: releng release notes category label Jul 18, 2025
XuehaiPan added a commit that referenced this pull request Jul 18, 2025
ghstack-source-id: 44e9385
Pull-Request: #158647
@XuehaiPan XuehaiPan added module: ci Related to continuous integration module: numpy Related to numpy support, and also numpy compatibility of our operators ciflow/trunk Trigger trunk jobs on your pull request ciflow/inductor keep-going Don't stop on first failure, keep running tests until the end ci-test-showlocals Show local variables on test failures labels Jul 18, 2025
[ghstack-poisoned]
XuehaiPan added a commit that referenced this pull request Jul 18, 2025
ghstack-source-id: a81d9bb
Pull-Request: #158647
[ghstack-poisoned]
XuehaiPan added a commit that referenced this pull request Jul 18, 2025
ghstack-source-id: 83ed021
Pull-Request: #158647
@XuehaiPan XuehaiPan changed the title [NumPy] use NumPy 2.0 in CI [NumPy] use NumPy 2.x in CI Jul 18, 2025
XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Jul 18, 2025
ghstack-source-id: 83ed021
Pull-Request: pytorch#158647
[ghstack-poisoned]
XuehaiPan added a commit that referenced this pull request Jul 18, 2025
ghstack-source-id: bdb0b1c
Pull-Request: #158647
[ghstack-poisoned]
XuehaiPan added a commit that referenced this pull request Jul 19, 2025
ghstack-source-id: ef16bd7
Pull-Request: #158647
[ghstack-poisoned]
XuehaiPan added a commit that referenced this pull request Jul 20, 2025
ghstack-source-id: b7e5e3d
Pull-Request: #158647
[ghstack-poisoned]
XuehaiPan added a commit that referenced this pull request Jul 20, 2025
ghstack-source-id: a205a56
Pull-Request: #158647
XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Jul 20, 2025
ghstack-source-id: a205a56
Pull-Request: pytorch#158647
Copy link
Collaborator

@rgommers rgommers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@XuehaiPan are you trying to use numpy 2.x everywhere in CI? Most of the individual bumps seem fine, however it does seem important to keep some coverage of minimum supported versions of both numpy and other dependencies.

It's too early to drop numpy 1.26.4 support, that will have to stay for quite a while longer since a lot of ML/AI-relevant libraries only fairly recently did releases with numpy >=2.0 support (so this would narrow the window of compatible versions for all those libraries) or even for a few straggelrs aren't compatible at all yet (e.g., sagemaker-python-sdk).

[ghstack-poisoned]
XuehaiPan added a commit that referenced this pull request Jul 20, 2025
ghstack-source-id: 1b64579
Pull-Request: #158647
XuehaiPan added a commit that referenced this pull request Jul 20, 2025
ghstack-source-id: 1b64579
Pull-Request: #158647
@XuehaiPan
Copy link
Collaborator Author

It's too early to drop numpy 1.26.4 support, that will have to stay for quite a while longer since a lot of ML/AI-relevant libraries only fairly recently did releases with numpy >=2.0 support (so this would narrow the window of compatible versions for all those libraries) or even for a few straggelrs aren't compatible at all yet (e.g., sagemaker-python-sdk).

@rgommers FYI, the last numpy 1.x version (1.26) will be EoL in 2 months:

This PR is still a draft. This will not be merged very soon. Additionally, PyTorch 2.8 features are frozen, and they should support NumPy 1.x.

@rgommers
Copy link
Collaborator

rgommers commented Jul 20, 2025

End-of-life for a Python package doesn't really mean much (nor is that even a thing for most projects) - it stays working just fine, it's not like for a distro that security bug fixes stop and it becomes dangerous to use that version.

You have to think about this in terms of compatibility windows. And dropping numpy 1.26 in particular will shorten that window by a lot for many other packages, as you can see from version bumps in this PR. The current approach seems to be to keep the window as wide as possible, and you're switching it to be pretty aggressive here. E.g., support is removed for:

  • numpy versions 1.20-1.26
  • numba versions 0.49 - 0.59
  • tensorboard versions 2.13 - 2.19

Some tightening there seems healthy because some versions are too old, but this probably swings a bit too far in the other direction.

@albanD is there a policy on this similar to the policy for CPython support?

[ghstack-poisoned]
XuehaiPan added a commit that referenced this pull request Jul 21, 2025
ghstack-source-id: 6735ac8
Pull-Request: #158647
@cyyever
Copy link
Collaborator

cyyever commented Jul 21, 2025

We could at least fix some bugs discovered from this PR.

@albanD
Copy link
Collaborator

albanD commented Jul 21, 2025

We don't have a strict policy on this no.
I 100% agree we should not drop numpy 1.26 any time soon. And we should keep testing on the oldest possible numpy version in CI.

That being said, I 100% agree @cyyever that we should improve our numpy 2.X support.
I do think that having a cpu shard on numpy 2.X would help there!

[ghstack-poisoned]
XuehaiPan added a commit that referenced this pull request Jul 26, 2025
ghstack-source-id: daa4cbf
Pull-Request: #158647
[ghstack-poisoned]
XuehaiPan added a commit that referenced this pull request Aug 10, 2025
ghstack-source-id: 7b44dff
Pull-Request: #158647
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-test-showlocals Show local variables on test failures ciflow/inductor ciflow/trunk Trigger trunk jobs on your pull request keep-going Don't stop on first failure, keep running tests until the end module: ci Related to continuous integration module: numpy Related to numpy support, and also numpy compatibility of our operators open source release notes: releng release notes category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants