Skip to content

[Bug]: Poetry installs setuptools-scm and setuptools #26932

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
LehMaxence opened this issue Sep 27, 2023 · 4 comments
Closed

[Bug]: Poetry installs setuptools-scm and setuptools #26932

LehMaxence opened this issue Sep 27, 2023 · 4 comments
Labels
Build
Milestone
v3.8.1

Comments

@LehMaxence
Copy link

LehMaxence commented Sep 27, 2023 •
edited
Loading

Bug summary

When adding matplotlib 3.8.0 to a poetry project, setuptools-scm and setuptools are installed and added to the list of dependencies. This was not the case with versions <= 3.7.2.

poetry version 1.6.1 was used.

Code for reproduction

poetry new poetry-demo
cd poetry-demo
poetry add matplotlib

Actual outcome

Using version ^3.8.0 for matplotlib

Updating dependencies
Resolving dependencies... (0.5s)

Package operations: 17 installs, 0 updates, 0 removals

  • Installing numpy (1.25.2)
  • Installing packaging (23.1)
  • Installing setuptools (68.2.2)
  • Installing six (1.16.0)
  • Installing tomli (2.0.1)
  • Installing typing-extensions (4.8.0)
  • Installing zipp (3.17.0)
  • Installing contourpy (1.1.1)
  • Installing cycler (0.11.0)
  • Installing fonttools (4.42.1)
  • Installing importlib-resources (6.1.0)
  • Installing kiwisolver (1.4.5)
  • Installing pillow (10.0.1)
  • Installing pyparsing (3.1.1)
  • Installing python-dateutil (2.8.2)
  • Installing setuptools-scm (8.0.3)
  • Installing matplotlib (3.8.0)

Writing lock file

Expected outcome

This is the outcome when installing matplotlib 3.7.2 with poetry add matplotlib==3.7.2 :

Updating dependencies
Resolving dependencies... Downloading https://files.pythonhosted.org/packages/72/7d/2ad1b94106f8b1971d1eff0ebb97a81d980c448732a3e624bba281bd274d/matplotlib-3.7.2-cp310-cp310-macosx_10Resolving dependencies... Downloading https://files.pythonhosted.org/packages/72/7d/2ad1b94106f8b1971d1eff0ebb97a81d980c448732a3e624bba281bd274d/matplotlib-3.7.2-cp310-cp310-macosx_10Resolving dependencies... (1.0s)

Writing lock file

Package operations: 13 installs, 0 updates, 0 removals

  • Installing numpy (1.25.2)
  • Installing six (1.16.0)
  • Installing zipp (3.17.0)
  • Installing contourpy (1.1.1)
  • Installing cycler (0.11.0)
  • Installing fonttools (4.42.1)
  • Installing importlib-resources (6.1.0)
  • Installing kiwisolver (1.4.5)
  • Installing packaging (23.1)
  • Installing pillow (10.0.1)
  • Installing pyparsing (3.0.9)
  • Installing python-dateutil (2.8.2)
  • Installing matplotlib (3.7.2)

Additional information

No response

Operating system

Ubuntu, Windows

Matplotlib Version

3.8.0

Matplotlib Backend

No response

Python version

3.9.13

Jupyter version

No response

Installation

None
@oscargus
Copy link
Member

oscargus commented Sep 27, 2023 •
edited
Loading

From a quite quick check there is no obvious reason for this. Both those packages are added a build-only dependencies

matplotlib/pyproject.toml

Lines 1 to 9 in 11aa656

[build-system]
build-backend = "setuptools.build_meta"
requires = [
"certifi>=2020.06.20",
"numpy>=1.25",
"pybind11>=2.6",
"setuptools>=64",
"setuptools_scm>=7",
]

But that has not changed between 3.7.2 and 3.8.0.

This leaves the option that tomli or typing-extensions (which are also added to the list) pull them in(?).

@ksunden
Copy link
Member

ksunden commented Sep 27, 2023 •
edited
Loading

This should only be true for source installs , any chance you did this from an environment with a source install? (Not sure where poetry picks up the dependencies, from local install or from pypi package) (I wrote this question early on, but figured it out as I kept looking)

I spot checked a wheel from pypi, and setuptools does not appear in the METADATA file there.

In a fresh virtualenv, pip does not pull in setuptools/setuptools-scm.

Certainly we have logic which does add setuptools-scm as a dependency (and recently added setuptools explicitly because of version conflicts), but that is only added if .git is found.

matplotlib/setup.py

Lines 346 to 349 in 11aa656

["setuptools_scm>=7", "setuptools>=64"] if (
Path(__file__).with_name(".git").exists() and
os.environ.get("CIBUILDWHEEL", "0") != "1"
) else []

But that logic is not new for 3.8.

From reading the poetry faq it looks like they do the pypi JSON API. and from looking at that, I do see setuptools_scm. Which we don't set anywhere, and I'm not sure where it is pulling that from.

Looking at the versioned json api:

  • 3.7.2 the requires-dist is null
  • 3.7.3 does include setuptools-scm (and list all other reqs)
  • 3.8.0 also does include setuptools-scm (and all other reqs)

This traces back to the PKG-INFO file in the sdist, which setuptools newly started including for sdists in v68.2.0

@ksunden
Copy link
Member

ksunden commented Sep 27, 2023

I think the action item here is to set the environment variable CIBUILDWHEEL=1 for the CI generated sdists (and then upload that one and not a locally generated one, but that part is already in the plans, I think, now that we cannot sign the files for PyPI).

@QuLogic QuLogic added this to the v3.8.1 milestone Oct 7, 2023
@QuLogic QuLogic mentioned this issue Oct 7, 2023
Merged
2 tasks
@QuLogic QuLogic added the Build label Oct 7, 2023
@ksunden
Copy link
Member

ksunden commented Oct 9, 2023

Closed by #27022

@ksunden ksunden closed this as completed Oct 9, 2023
@ksunden ksunden mentioned this issue Oct 9, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Build
Projects
None yet
Milestone
v3.8.1
Development

No branches or pull requests
4 participants
@QuLogic @ksunden @oscargus @LehMaxence