I am in favour of this. The pybind11 work becomes much simpler if we at least replace setup_requires in setup.py with [build-system] in pyproject.toml. I was going to talk about it tomorrow's call.

FWIW Bokeh has just made this transition.

I gave it a quick try to just move over the more basic things and got stuck in that setupext could not be imported from setup.py anymore. I take it that it executes from another directory or something so that it does not find setupext.py.

Copying the contents doesn't seem like the best way to solve it...

A temporary environment is used to build a wheel which is then installed into your working environment. The temporary environment doesn't seem to have access to the full project directory.

I have had success with:

1. Removing the setup_requires section from setup.py

2. New pyproject.toml containing

[build-system]
build-backend = "setuptools.build_meta"
requires = [
    "certifi>=2020.06.20",
    "numpy>=1.19",
    "setuptools_scm>=7",
]

3. Add the following line to setup.py just before import setupext:

sys.path.append(str(Path(__file__).resolve().parent))

so basically botching the sys.path so that it can find setupext.py.

I've seen some problems with numpy versions. My guess is that using pyproject.toml somehow uses a different version of NumPy headers than possibly installed. For example, when installing using 1.22 installed I got an error about not matching API versions, which I didn't get with a plain setup.py install, while upgrading to 1.23 (and reinstalling, so not completely sure) I didn't. Seen similar problems with another package with a much simpler install procedure as well.

Note that in both cases, the actual compilation happened in setup.py, so moving all the way may be a solution.

@oscargus See https://github.com/matplotlib/matplotlib/pull/23829/files#r968896893 for the numpy API version issue.

This was completed as part of the Meson port in #26621.