[build] pin `setuptools>=77` to enable PEP 639 #158104

XuehaiPan · 2025-07-11T05:53:37Z

setuptools v77 adds PEP 639 support for SPDX expression as package.license in pyproject.toml and also deprecated TOML-table-based package.license. That depression will be converted to an error in the future.

Note that setuptools<77 does not support SPDX-based package.license while setuptools>=77 will error for non-SPDX-based package.license in the future. We need an explicit version pin for setuptools>=77 (or pin setuptools<77).

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158104

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 9 New Failures, 2 Cancelled Jobs, 16 Pending, 1 Unrelated Failure

As of commit 75c8c6c with merge base b219ca2 ():

NEW FAILURES - The following jobs have failed:

linux-binary-manywheel / manywheel-py3_10-xpu-build / build (gh)
ninja: build stopped: subcommand failed
linux-binary-manywheel / manywheel-py3_11-cuda12_8-full-test / test (gh)
ImportError: libnvshmem_host.so.3: cannot open shared object file: No such file or directory
linux-binary-manywheel / manywheel-py3_11-xpu-build / build (gh)
ninja: build stopped: subcommand failed
linux-binary-manywheel / manywheel-py3_12-xpu-build / build (gh)
ninja: build stopped: subcommand failed
linux-binary-manywheel / manywheel-py3_13-xpu-build / build (gh)
ninja: build stopped: subcommand failed
linux-binary-manywheel / manywheel-py3_13t-xpu-build / build (gh)
ninja: build stopped: subcommand failed
linux-binary-manywheel / manywheel-py3_14-xpu-build / build (gh)
ninja: build stopped: subcommand failed
linux-binary-manywheel / manywheel-py3_14t-xpu-build / build (gh)
ninja: build stopped: subcommand failed
linux-binary-manywheel / manywheel-py3_9-xpu-build / build (gh)
ninja: build stopped: subcommand failed

CANCELLED JOBS - The following jobs were cancelled. Please retry:

linux-binary-libtorch / libtorch-rocm6_3-shared-with-deps-release-build / build (gh)
##[error]The operation was canceled.
linux-binary-libtorch / libtorch-rocm6_4-shared-with-deps-release-build / build (gh)
##[error]The operation was canceled.

FLAKY - The following job failed but was likely due to flakiness present on trunk:

pull / linux-jammy-py3.9-clang12 / test (crossref, 2, 2, lf.linux.2xlarge) (gh) (similar failure)
cpp/test_jit 1/1 failed!

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 0de80b7 Pull-Request: #158104

[ghstack-poisoned]

ghstack-source-id: a0d315a Pull-Request: #158104

[ghstack-poisoned]

ghstack-source-id: 0f0a345 Pull-Request: #158104

[ghstack-poisoned]

ghstack-source-id: a9749e0 Pull-Request: #158104

[ghstack-poisoned]

ghstack-source-id: 2d2f7fc Pull-Request: #158104

[ghstack-poisoned]

ghstack-source-id: caa071f Pull-Request: #158104

[ghstack-poisoned]

ghstack-source-id: 9a3c1e9 Pull-Request: pytorch#158104

[ghstack-poisoned]

ghstack-source-id: b1f18b3 Pull-Request: pytorch#158104

iremyux · 2025-08-06T11:30:07Z

Adding ciflow/win-arm64 label to trigger Windows Arm64 CI and its test purposes - nothing about this PR specifically. (It should not effect the acceptance of the PR even if it fails.)

[ghstack-poisoned]

ghstack-source-id: 8ffb2cf Pull-Request: pytorch#158104

[ghstack-poisoned]

ghstack-source-id: 79378ae Pull-Request: pytorch#158104

[ghstack-poisoned]

ghstack-source-id: 443804a Pull-Request: pytorch#158104

XuehaiPan · 2025-08-11T13:31:54Z

Since PyTorch 2.8.0 has been released, I wonder if there are any blockers or unresolved concerns about this PR? cc @albanD

albanD · 2025-08-11T18:33:00Z

To me there are two main concers:

I don't understand what we need from pep 639 that justifies all of this.
A lot of version updates and pinning that I'm not sure how to check clearly.

XuehaiPan · 2025-08-12T04:41:02Z

To me there are two main concers:

I don't understand what we need from pep 639 that justifies all of this.

See the PR description #158104 (comment), we will need to pin either <77.0 or >=77.0 shortly. Otherwise, setuptools will complain and fail.

A lot of version updates and pinning that I'm not sure how to check clearly.

I will try to remove the pins as much as I can in this PR, and then get back to you.

[ghstack-poisoned]

ghstack-source-id: 005d39e Pull-Request: pytorch#158104

[ghstack-poisoned]

ghstack-source-id: 8429e5c Pull-Request: pytorch#158104

[ghstack-poisoned]

ghstack-source-id: d199478 Pull-Request: pytorch#158104

[ghstack-poisoned]

ghstack-source-id: 991eb1c Pull-Request: pytorch#158104

[ghstack-poisoned]

ghstack-source-id: f19b9a7 Pull-Request: pytorch#158104

[ghstack-poisoned]

ghstack-source-id: a8e74a0 Pull-Request: pytorch#158104

albanD

I would like to get someone from Infra, either @atalman or @seemethere or @malfet comment on adding a new "pip install -r requirements-build.txt" to all the build/test scripts. It feels to me like these are at least partially redundant and we should remove whatever is trying (and failing) to install these dependencies if we're adding this now.

Also a couple nits to keep everything <80 to avoid changing the build frontend between easy_install and pip.

albanD · 2025-08-12T21:52:55Z

.ci/pytorch/windows/setup_build.bat

@@ -16,7 +17,7 @@ if "%DESIRED_PYTHON%" == "3.9" %PYTHON_EXEC% -m pip install numpy==2.0.2 cmake

 %PYTHON_EXEC% -m pip install pyyaml
 %PYTHON_EXEC% -m pip install mkl-include mkl-static
-%PYTHON_EXEC% -m pip install boto3 ninja typing_extensions setuptools==72.1.0
+%PYTHON_EXEC% -m pip install boto3 ninja typing-extensions setuptools packaging


Why not use the requirements-build.txt file for this one?

albanD · 2025-08-12T21:53:28Z

.github/requirements/pip-requirements-macOS.txt

@@ -26,7 +26,7 @@ pytest-xdist==3.3.1
 pytest==7.3.2
 pyyaml==6.0.2
 scipy==1.12.0
-setuptools==72.1.0
+setuptools==80.9.0


Let's keep it <80

albanD · 2025-08-12T21:54:04Z

pyproject.toml

    # 77.0.0: min version for SPDX expression support for project.license
-    "setuptools>=70.1.0,<80.0",
+    "setuptools>=77.0.0,<80.0",


Aren't these duplicate of requirements-build.txt? Can we remove one of the two?

malfet

It feels like we have an explosion of requirements-something-or-other files in PyTorch repo

May be before we start relying using another one, propose a separate PR that adds comment to each of them explaining when they should vs should not be used

Also, at least for linux, to avoid waisting CI resources, all requirements should be installed as part of the container setup, defined here: .ci/docker/requirements-ci.txt

malfet · 2025-08-12T22:14:07Z

And I know it's way too late to complain, but I'm confused about the purpose of #158111

Requirements should be installed once, if we need custom test configs that lower or raise versions, it should be done differently

Update

ed7683d

[ghstack-poisoned]

XuehaiPan requested review from a team and jeffdaily as code owners July 11, 2025 05:53

pytorch-bot bot added the release notes: releng release notes category label Jul 11, 2025

XuehaiPan added a commit that referenced this pull request Jul 11, 2025

[build] pin setuptools>=77 to enable PEP 639

7d3f9c2

ghstack-source-id: 0de80b7 Pull-Request: #158104

pytorchbot added the open source label Jul 11, 2025

XuehaiPan marked this pull request as draft July 11, 2025 06:27

XuehaiPan added ciflow/binaries Trigger all binary build and upload jobs on the PR ciflow/trunk Trigger trunk jobs on your pull request ciflow/inductor labels Jul 11, 2025

Update

9684ee0

[ghstack-poisoned]

XuehaiPan added a commit that referenced this pull request Jul 11, 2025

[build] pin setuptools>=77 to enable PEP 639

6cd9ce1

ghstack-source-id: a0d315a Pull-Request: #158104

XuehaiPan mentioned this pull request Jul 11, 2025

[BE][Easy] split build system requirements.txt to a separate file #158111

Closed

Update

3902856

[ghstack-poisoned]

XuehaiPan added a commit that referenced this pull request Jul 11, 2025

[build] pin setuptools>=77 to enable PEP 639

d4b29f3

ghstack-source-id: 0f0a345 Pull-Request: #158104

Update

e51f1b8

[ghstack-poisoned]

XuehaiPan added a commit that referenced this pull request Jul 11, 2025

[build] pin setuptools>=77 to enable PEP 639

3ed78aa

ghstack-source-id: a9749e0 Pull-Request: #158104

Update

e097bfa

[ghstack-poisoned]

XuehaiPan added a commit that referenced this pull request Jul 11, 2025

[build] pin setuptools>=77 to enable PEP 639

d70f7bc

ghstack-source-id: 2d2f7fc Pull-Request: #158104

Update

e060ff5

[ghstack-poisoned]

XuehaiPan added module: build Build system issues release notes: build release notes category topic: build labels Jul 11, 2025

XuehaiPan added a commit that referenced this pull request Jul 11, 2025

[build] pin setuptools>=77 to enable PEP 639

2054bfb

ghstack-source-id: caa071f Pull-Request: #158104

XuehaiPan marked this pull request as ready for review July 11, 2025 10:17

Update

b978039

[ghstack-poisoned]

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Jul 31, 2025

[build] pin setuptools>=77 to enable PEP 639

325d383

ghstack-source-id: 9a3c1e9 Pull-Request: pytorch#158104

Update

26f0c1a

[ghstack-poisoned]

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Aug 5, 2025

[build] pin setuptools>=77 to enable PEP 639

5f7d6fb

ghstack-source-id: b1f18b3 Pull-Request: pytorch#158104

iremyux added the ciflow/win-arm64 Trigger Windows Arm64 CI Workflows label Aug 6, 2025

Update

a3b8ea2

[ghstack-poisoned]

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Aug 6, 2025

[build] pin setuptools>=77 to enable PEP 639

cb220af

ghstack-source-id: 8ffb2cf Pull-Request: pytorch#158104

Update

5bfce27

[ghstack-poisoned]

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Aug 10, 2025

[build] pin setuptools>=77 to enable PEP 639

279170e

ghstack-source-id: 79378ae Pull-Request: pytorch#158104

Update

0516a50

[ghstack-poisoned]

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Aug 11, 2025

[build] pin setuptools>=77 to enable PEP 639

04ed5fe

ghstack-source-id: 443804a Pull-Request: pytorch#158104

Update

c310400

[ghstack-poisoned]

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Aug 12, 2025

[build] pin setuptools>=77 to enable PEP 639

703a422

ghstack-source-id: 005d39e Pull-Request: pytorch#158104

Update

dde688d

[ghstack-poisoned]

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Aug 12, 2025

[build] pin setuptools>=77 to enable PEP 639

da5b114

ghstack-source-id: 8429e5c Pull-Request: pytorch#158104

Update

278099c

[ghstack-poisoned]

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Aug 12, 2025

[build] pin setuptools>=77 to enable PEP 639

fe79931

ghstack-source-id: d199478 Pull-Request: pytorch#158104

Update

ba8bd94

[ghstack-poisoned]

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Aug 12, 2025

[build] pin setuptools>=77 to enable PEP 639

e965921

ghstack-source-id: 991eb1c Pull-Request: pytorch#158104

Update

2a4285e

[ghstack-poisoned]

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Aug 12, 2025

[build] pin setuptools>=77 to enable PEP 639

1966be1

ghstack-source-id: f19b9a7 Pull-Request: pytorch#158104

Update

75c8c6c

[ghstack-poisoned]

XuehaiPan added a commit to XuehaiPan/pytorch that referenced this pull request Aug 12, 2025

[build] pin setuptools>=77 to enable PEP 639

a0a2d65

ghstack-source-id: a8e74a0 Pull-Request: pytorch#158104

albanD reviewed Aug 12, 2025

View reviewed changes

malfet requested changes Aug 12, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[build] pin `setuptools>=77` to enable PEP 639 #158104

[build] pin `setuptools>=77` to enable PEP 639 #158104

XuehaiPan commented Jul 11, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Jul 11, 2025 •

edited

Loading

Uh oh!

iremyux commented Aug 6, 2025

Uh oh!

XuehaiPan commented Aug 11, 2025

Uh oh!

albanD commented Aug 11, 2025

Uh oh!

XuehaiPan commented Aug 12, 2025

Uh oh!

albanD left a comment

Uh oh!

albanD Aug 12, 2025

Uh oh!

albanD Aug 12, 2025

Uh oh!

albanD Aug 12, 2025

Uh oh!

malfet left a comment

Uh oh!

malfet commented Aug 12, 2025

Uh oh!

Uh oh!

[build] pin setuptools>=77 to enable PEP 639 #158104

Are you sure you want to change the base?

[build] pin setuptools>=77 to enable PEP 639 #158104

Conversation

XuehaiPan commented Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/158104

❌ 9 New Failures, 2 Cancelled Jobs, 16 Pending, 1 Unrelated Failure

Uh oh!

iremyux commented Aug 6, 2025

Uh oh!

XuehaiPan commented Aug 11, 2025

Uh oh!

albanD commented Aug 11, 2025

Uh oh!

XuehaiPan commented Aug 12, 2025

Uh oh!

albanD left a comment

Choose a reason for hiding this comment

Uh oh!

albanD Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

albanD Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

albanD Aug 12, 2025

Choose a reason for hiding this comment

Uh oh!

malfet left a comment

Choose a reason for hiding this comment

Uh oh!

malfet commented Aug 12, 2025

Uh oh!

Uh oh!

[build] pin `setuptools>=77` to enable PEP 639 #158104

[build] pin `setuptools>=77` to enable PEP 639 #158104

XuehaiPan commented Jul 11, 2025 •

edited

Loading

pytorch-bot bot commented Jul 11, 2025 •

edited

Loading