I missed a comment from @ngoldbaum elsewhere, so I'll wait for at least his input here before touching the copy keyword issue.

I’m going to try to work on a PR to fix the issues with the copy keyword this afternoon. If I can’t figure it out I’ll send in a revert.

@charris I'm having a go at release notes updates now.

@rgommers Great.

Wheel building failed, see #25923

Restarted nightly wheel builds once more.

There is one more PR unless it wrecks havoc. The DType ones were a bit stalled, there is a follow up to bump the itemsize to intp and finalize the dtype flags changes.
(I.e. the actual ABI break, I was quite careful to prepare the break in the first PR, but the back and forth delayed the follow up.)

Thanks @seberg. Let me ask a few questions to really spell it out, because Chuck was already planning to call it good and create the release branch once the copy keyword regression was fixed.

1. You still want to get exactly two things in before 2.0.0rc1: MAINT,API: Make metadata, c_metadata, fields, and names only exist on old-style dtypes #25802 and a to-be-opened PR (which I think addresses the second item of In-flight DType change follow-ups #25820)?
2. Do those two depend on each other?
3. How much time do you estimate you need?
4. Is it necessary to do another round of "build nightlies, then build nightlies of scipy straight after, then test scikit-learn/astropy"? (probably yes?) If so, can we do it once both PRs are in (since shepherding all the nightly builds is pretty time-consuming)?

It shouldn't take long to create the follow up, but right now the most time was always waiting for review (I will skip renaming elsize to itemsize for now to make the change non-huge).

The annoyance is fixing pybind11 and SciPy needs that due to a single line of code:

scipy/spatial/src/distance_pybind.cpp:    desc.element_size = arr.itemsize();

I can even do that, but not quite sure how long review would take.

Any chance we can work around that in SciPy? Getting pybind11 to not only fix/review something but do a release that SciPy can use may be difficult.

Any chance we can work around that in SciPy?

Seems that while pybind11 avoids importing NumPy explicitly, SciPy does not in the module (that I noticed is bad) using pybind11, so using the NumPy provided macros should be trivial, yes.
(I am confused why pybind11 avoids it so much considering SciPy does it...)

Quick question: over the weekend, our JAX CI jobs running against numpy nightly began failing due to this new error:

In [1]: import numpy as np

In [2]: np.array(0.0, copy=False)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[2], line 1
----> 1 np.array(0.0, copy=False)

ValueError: Unable to avoid copy while creating an array.

I know there's been some work on the copy parameter to asarray and __array__: is this intended? If so, I can update our code to work around it. If not, I can open a bug. Thanks!

Yes, that looks expected @jakevdp. For notes on the changes, see

This SciPy PR to adapt to the changes may be useful too: scipy/scipy#20172

The thing that I thought was in the release notes/migration guide: Avoid using array if you have copy=False since asarray is the clearer version. (A long time ago, that had much more overhead, which is probably the main reason why code has it.)

The thing that I thought was in the release notes/migration guide: Avoid using array if you have copy=False since asarray is the clearer version

Good point. It's missing from the migration guide, which is the right place I think. I'll take it along in my next PR for release notes updates.

Thanks!

Looks like the last PRs we need are up!

One thing I'd like to suggest is to do a 2.0.0b1 release, which is likely but not guaranteed to have a stable C API and ABI. Doing an rc1 which has a hard guarantee to be stable within days of merging large breaking changes is probably a bit too risky, so it'd be nice to have a 1-2 week escape hatch.

I imagine the following rollout scenario for releases to PyPI:

1. numpy 2.0.0b1
2. scipy/pandas/matplotlib/etc. make RC1 releases
3. announcement that pip install <your-deps> --pre works from PyPI, give a 3-7 day window to catch major issues
4. numpy 2.0.0rc1 (guaranteed stable now)
5. scipy/pandas/matplotlib/etc. do a final release, compatible with 2.0 and 1.xx
6. numpy 2.0.0rc2 (surely we'll need it for something)
7. numpy 2.0.0

I am happy with a b1 to not signal ABI stability right away. I don't particularly expect ABI changes (as opposed to API ones; the main place where I see a meaningful ABI break would be the new DType API and nobody will release a new DType in the next month).

My impression is that error reports are either immediate, or show up after the first week, so things might move a bit slower than planned. But that is OK as long as something is out there.

We can drop Python 3.9 support in main after the branch, I suggest moving more tests to Python 3.12 where they can hang around for a couple of years.

Looks like we are almost ready.

Yep, I think once the last release notes PR is merged, the branch can be created.

You were right, seems if matplotlib is fine (enough). Doc builds are working again and the website is up to date again.

@seberg are you still planning to make a change in between 2.0.0b1 and 2.0.0rc1 that turns the warning for pybind11 usage of private API into a hard error? Or, since it kinda works for now, do it only in main after pybind11 is fixed up and released?

🤷 that huge warning is seems only semi-useful, it would be nice to make it a hard error by the time we release, otherwise we don't protect users from potentially bad results/crashes due to a module that wasn't recompiled.
I don't care much about rc1 specifically it just seems like a plausible/reasonable warning shot for pybind11.

Well, it must be done either before rc1 or not at all. The way it looks now, it actually seems fairly safe to disable the warning, leaving the compat shim in place for 2.0, and make it a hard error on main in a month or so.

Probably best to wait and see how a pybind11 release is coming along, and decide in a week or so?

The release branch is created and 2.0.0b1 is up on PyPI, so I will close this issue. There are a few things we need to do before 2.0.0rc1, including ones touched upon in this issue, so I will create a new tracking issue for that.

Thanks all!