Add support for NumPy 2.0 #232

bjodah · 2024-06-18T08:23:54Z

Seems like NumPy 2.0 broke some behavior which quantities relied on. This PR allows the test suite to pass for me locally. I have not addressed any deprecation warnings that are new in numpy 2.0. I'm thinking that we can fix those in a subsequent PR.

Let me know what you think.

apdavison · 2024-06-18T09:13:07Z

Hi @bjodah - many thanks for this. To help understand why these changes were needed, I created #233. It seems that almost all the errors are of the form:

ValueError: Unable to avoid copy while creating an array as requested.
If using `np.array(obj, copy=False)` replace it with `np.asarray(obj)` to allow a copy when needed (no behavior change in NumPy 1.x).
For more details, see https://numpy.org/devdocs/numpy_2_0_migration_guide.html#adapting-to-changes-in-the-copy-keyword.

Can you comment on why you used np.asanyarray in some places?

bjodah · 2024-06-18T09:18:41Z

Sure, it's my understanding that asanyarray corresponds to subok=True, and the use of asarray and asanyarray will avoid copying if possible, but unlike the copy=False flag, they will not raise an exception when a copy is needed.

I found this answer on stackoverflow to be helpful:
https://stackoverflow.com/a/52103839/790973

apdavison · 2024-06-18T09:21:41Z

I will leave this PR open until the end of this week in case anyone else has comments, but if everything still looks good by then, I'll merge this and make a new release.

bjodah · 2024-06-18T09:25:05Z

quantities/quantity.py


-        ret = np.array(data, dtype=dtype, copy=copy).view(cls)
+        ret = np.asarray(data, dtype=dtype).view(cls)


After these changes we are no longer using the copy keyword argument passed into __new__. I contemplated dropping it, but I fear that the change would be too disruptive for backwards compatibility. Perhaps changing its default to None and conditionally issuing a DeprecationWarning when it is set to True/False is a viable option? And the deprecation warning would inform the user that the copy-kwarg will be dropped in future release of quantities.

That's a good idea.

drammock · 2024-06-18T15:42:11Z

one other NumPy 2 compat fix would be removing the reference to np.core on this line:

python-quantities/quantities/dimensionality.py

Line 333 in b47419a

p_dict[np.core.umath.clip] = _d_clip

zm711 · 2024-06-18T15:52:12Z

I'm going to link the Neo CI-update so I can see when this is merged to check our tests again :)

NeuralEnsemble/python-neo#1490

bjodah · 2024-06-19T09:25:05Z

Nicely spotted @drammock, hopefully fixed in 6d035c6. Thanks.

I went ahead and fixed all warnings reported during the test run (when using numpy-2.0). We still carry __array_prepare__ which I think is no longer used externally by numpy, but we do call this method from __array_wrap__, so I did not refactor that part.

I seem to have messed up something else for numpy<2 in last commit. Will need to take a closer look a bit later if no one beats me to it.

drammock · 2024-06-19T19:52:13Z

Nicely spotted @drammock, hopefully fixed in 6d035c6. Thanks.

don't thank me, thank our CIs :)

zm711 · 2024-06-27T18:39:38Z

quantities/quantity.py

+            return self.__array_prepare__(obj, context)
+        else:
+            return super().__array_wrap__(obj, context, return_scalar)


I'm confused by this. The previous array wrap would only do something if the object was not a Quantity if the object was already a quantity it would just return the object without any changes. Here you force the array_wrap even in the case of the object being a Quantity. It seems like for the case of some failing tests the objects are losing their dimensionality and this seems like it could be a place where a Quantity should be returned, no?

I'm not super familiar with this deep level of array wrapping so just thought maybe a discussion of this could help point us to the actual test failure issue.

@bjodah just wanted to bump my message. Does this seem like it could be the problem with the tests failing?

bjodah · 2024-07-24T16:17:46Z

Sounds plausible. I'm on vacation at the moment, and then I'll be moving, so I won't have much time the upcoming weeks to look further into this. If someone wants to take over with a new PR (based on this branch or master branch), then feel free to do so.

…

On Wed, Jul 24, 2024, 14:41 Zach McKenzie ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In quantities/quantity.py <#232 (comment)> : > + return self.__array_prepare__(obj, context) + else: + return super().__array_wrap__(obj, context, return_scalar) @bjodah <https://github.com/bjodah> just wanted to bump my message. Does this seem like it could be the problem with the tests failing? — Reply to this email directly, view it on GitHub <#232 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADWUMA3JMNEHGEOFEZ7IVTZN6OF5AVCNFSM6AAAAABJPQSL22VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDCOJWGYZDQNJTGU> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

zm711 · 2024-07-24T17:28:58Z

Thanks @bjodah, I built off of your PR in #235 and got it working for both. Enjoy the vacation and moving :)

bjodah · 2024-07-24T17:33:20Z

That's great to hear! And thanks.

…

On Wed, Jul 24, 2024, 19:29 Zach McKenzie ***@***.***> wrote: Thanks @bjodah <https://github.com/bjodah>, I built off of your PR in #235 <#235> and got it working for both. Enjoy the vacation and moving :) — Reply to this email directly, view it on GitHub <#232 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADWUMGXZTDP4XWWJ4HL72LZN7P7DAVCNFSM6AAAAABJPQSL22VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBYGU2DQMZQHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

mscheltienne · 2024-08-27T12:29:31Z

quantities/dimensionality.py

@@ -9,6 +9,8 @@
 from .registry import unit_registry
 from .decorators import memoize

+_np_version = tuple(map(int, np.__version__.split('.')))


This is now breaking the tests in MNE:

_np_version = tuple(map(int, np.__version__.split('.'))) E ValueError: invalid literal for int() with base 10: 'dev0+git20240824'

x-ref: mne-tools/mne-python#12815

Are you all testing against a development version of NumPy? I don't think @bjodah or I thought about that in these PRs for parsing the numpy version. I guess a version parse instead would be better if you want to defend against dev version testing.

MNE is testing against stable and the development version of numpy, all my projects are also tested against both, and I'm probably not the only one ;)
Quick fix in #238 but maybe you want something a bit more elaborate 😃

Add support for NumPy 2.0

ed43021

bjodah mentioned this pull request Jun 18, 2024

New numpy error on import bjodah/chempy#234

Closed

bjodah commented Jun 18, 2024

View reviewed changes

drammock mentioned this pull request Jun 18, 2024

NumPy 2.0 compat problems in doc build mne-tools/mne-python#12673

Merged

zm711 mentioned this pull request Jun 18, 2024

Support numpy 2.0 SpikeInterface/spikeinterface#3033

Closed

slayoo mentioned this pull request Jun 19, 2024

fix compatibility with NumPy 2.0.0 (dependes on a fix in python-quantities package) open-atmos/PySDM#1344

Open

bjodah force-pushed the support-for-numpy-2 branch from 52788ed to 63a64b2 Compare June 19, 2024 09:09

Deprecate Quantities(... copy=) kwarg. Fix warnings.

6d035c6

bjodah force-pushed the support-for-numpy-2 branch from 63a64b2 to 6d035c6 Compare June 19, 2024 09:18

zm711 reviewed Jun 27, 2024

View reviewed changes

zm711 mentioned this pull request Jun 27, 2024

numpy.core is deprecated as of Numpy 2.0 #234

Open

zm711 mentioned this pull request Jul 24, 2024

Add numpy 2.0 compatibility #235

Merged

apdavison merged commit 6d035c6 into python-quantities:master Aug 27, 2024
20 of 44 checks passed

mscheltienne reviewed Aug 27, 2024

View reviewed changes

Moritz-Alexander-Kern mentioned this pull request Oct 28, 2024

[Fix] neo v0.13.4 deprecated copy NeuralEnsemble/elephant#646

Merged

Moritz-Alexander-Kern mentioned this pull request Dec 10, 2024

[Feature] Support for Numpy >= 2.0 NeuralEnsemble/elephant#652

Open


		ret = np.array(data, dtype=dtype, copy=copy).view(cls)
		ret = np.asarray(data, dtype=dtype).view(cls)

Add support for NumPy 2.0 #232

Add support for NumPy 2.0 #232

Uh oh!

Conversation

bjodah commented Jun 18, 2024

Uh oh!

apdavison commented Jun 18, 2024

Uh oh!

bjodah commented Jun 18, 2024

Uh oh!

apdavison commented Jun 18, 2024

Uh oh!

bjodah Jun 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apdavison Jun 18, 2024

Choose a reason for hiding this comment

Uh oh!

drammock commented Jun 18, 2024

Uh oh!

zm711 commented Jun 18, 2024

Uh oh!

bjodah commented Jun 19, 2024

Uh oh!

drammock commented Jun 19, 2024

Uh oh!

zm711 Jun 27, 2024

Choose a reason for hiding this comment

Uh oh!

zm711 Jul 24, 2024

Choose a reason for hiding this comment

Uh oh!

bjodah commented Jul 24, 2024 via email

Uh oh!

zm711 commented Jul 24, 2024

Uh oh!

bjodah commented Jul 24, 2024 via email

Uh oh!

Uh oh!

mscheltienne Aug 27, 2024

Choose a reason for hiding this comment

Uh oh!

zm711 Aug 27, 2024

Choose a reason for hiding this comment

Uh oh!

mscheltienne Aug 27, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bjodah Jun 18, 2024 •

edited

Loading