Skip to content

Handle warning about private API and pybind11 before 2.0.0rc1 #26010

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rgommers opened this issue Mar 12, 2024 · 8 comments · Fixed by #26149
Closed

Handle warning about private API and pybind11 before 2.0.0rc1 #26010

rgommers opened this issue Mar 12, 2024 · 8 comments · Fixed by #26149

Comments

@rgommers
Copy link
Member

In 2.0.0b1 we still have a very verbose and scary-looking warning like this:

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0.dev0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled against NumPy 2.0.
...
pybind11 note: You may see this message if using pybind11,
this is not problematic at pre-release time
it indicates the need for a new pybind11 release.

Full warning:

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0.dev0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled against NumPy 2.0.

If you are a user of the module, the easiest solution will be to
either downgrade NumPy or update the failing module (if available).

NOTE: When testing against pre-release versions of NumPy 2.0
or building nightly wheels for it, it is necessary to ensure
the NumPy pre-release is used at build time.
The main way to ensure this is using no build isolation
and installing dependencies manually with NumPy.
For cibuildwheel for example, this may be achieved by using
the flag to pip:
    CIBW_BUILD_FRONTEND: pip; args: --no-build-isolation
installing NumPy with:
    pip install --pre --extra-index-url https://pypi.anaconda.org/scientific-python-nightly-wheels/simple
in the `CIBW_BEFORE_BUILD` step.  Please compare with the
solutions e.g. in astropy or matplotlib for how to make this
conditional for nightly wheel builds using expressions.
If you do not worry about using pre-releases of all
dependencies, you can also use `--pre --extra-index-url` in the
build frontend (instead of build isolation).
This will become unnecessary as soon as NumPy 2.0 is released.

If your dependencies have the issue, check whether they
have nightly wheels build against NumPy 2.0.

pybind11 note: You may see this message if using pybind11,
this is not problematic at pre-release time
it indicates the need for a new pybind11 release.

We need to finalize this before 2.0.0rc1 in one of two ways:

  1. Turn the warning into a hard error,
  2. Silence the warning.

At #25918 (comment) we discussed a bit. @seberg said:
"That huge warning is seems only semi-useful, it would be nice to make it a hard error by the time we release, otherwise we don't protect users from potentially bad results/crashes due to a module that wasn't recompiled. I don't care much about rc1 specifically it just seems like a plausible/reasonable warning shot for pybind11."

I said: "Well, it must be done either before rc1 or not at all. The way it looks now, it actually seems fairly safe to disable the warning, leaving the compat shim in place for 2.0, and make it a hard error on main in a month or so. Probably best to wait and see how a pybind11 release is coming along, and decide in a week or so?"

The outcome will depend on pybind/pybind11#5009. If pybind11 manages to get a release out soon enough, we may be able to turn the warning into an error. If not, then it'll be too disruptive to do so.

@seberg
Copy link
Member

seberg commented Mar 13, 2024

I am not sure I can agree that silencing the warning is an option. Remember that without the fixes at least in some parts, SciPy would have just returned incorrect results.
I can accept keeping the warning for pybind11s sake, but it seems to me users must have a fighting chance to know that why they may be seeing random nonsense or crashes.

@charris
Copy link
Member

charris commented Mar 24, 2024

We are about ready for 2.0.0rc1.

@rgommers
Copy link
Member Author

This issue is blocking, it needs to be resolved first on the pybind11 side (very close to done).

@mattip
Copy link
Member

mattip commented Mar 25, 2024

Is there a tracking issue we can follow to see the progress?

@neutrinoceros
Copy link
Contributor

I assume @rgommers was talking about pybind/pybind11#5050

@henryiii
Copy link
Contributor

One team member has said they would review, and we are waiting on that review. Hopefully very soon.

@henryiii
Copy link
Contributor

pybind11 2.12 is out!

@rgommers
Copy link
Member Author

Awesome, thanks a lot @henryiii!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants