The failures here are (will be) as the relevant parts of npy_static_pydata are not cleared on module deallocation, so e.g. npy_cpu_dispatch_tracer_init() and other functions checking the global state fail. We can add the m_clear/m_free slots to the moduledef, but I'm not sure which bits we should reset.

The other option is to keep treating those variables as global and instead re-add the same object to the new module. This may cause other problems though, as I don't believe it's the intended execution model.

There also appears to be a SIGSEV around the call to dtypemeta_wrap_legacy_descriptor() in arraytypes.c.src, but I'm not familiar enough with the templating process here.

cc @ngoldbaum if you'd be able to suggest anything?

A

Nice, will be great to make progress here!

We have a lot of global state, but much of it can probably just stay intact within the same interpreter (some cannot, e.g. because new singletons are created but this was always incorrect, so for a first start...).

So without multi-stage init, CPython would not run the module initially again if the so was already loaded (so deleting it from sys.modules and importing again doesn't trigger a second init)?

One problem with dtypemeta_wrap_legacy_descriptor is that it mutates global singleton instances. The actual problem should be in dtypemeta_wrap_legacy_descriptor in dtypemeta.c.

There is a check to catch types that are set to bad values, I honestly don't remember why it is there, and the loop definitely looks bad (it should just look up the type number of them descr and not loop!).
So it may just be replacing the loop with PyArray_DescrFromType(descr->type_num);.

So long we don't actually support subinterpreters, the old type object should actually be good to use. @AA-Turner maybe you can try applying something like this:

diff --git a/numpy/_core/src/multiarray/dtypemeta.c b/numpy/_core/src/multiarray/dtypemeta.c
index 0b1b0fb391..adb268097c 100644
--- a/numpy/_core/src/multiarray/dtypemeta.c
+++ b/numpy/_core/src/multiarray/dtypemeta.c
@@ -1075,6 +1075,11 @@ dtypemeta_wrap_legacy_descriptor(
     _PyArray_LegacyDescr *descr, PyArray_ArrFuncs *arr_funcs,
     PyTypeObject *dtype_super_class, const char *name, const char *alias)
 {
+    if (Py_TYPE(descr) != NULL && NPY_DTYPE(descr)->singleton == (PyArray_Descr *)descr) {
+        /* We seem to already have created a type, multiple inits? */
+        return 0;
+    }
+
     int has_type_set = Py_TYPE(descr) == &PyArrayDescr_Type;
 
     if (!has_type_set) {

(but there will likely be a long tail of these type of things, and of course the real fix will be to move the singletons into proper module state!)

Linux Qemu tests / riscv64 (pull_request) failure unrelated: E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/dists/jammy-updates/universe/binary-riscv64/Packages.gz File has unexpected size (1163769 != 1163765). Mirror sync in progress? [IP: 91.189.91.104 80]

Yeah we get spurious failures due to network issues quite a bit, safe to ignore.

Also, wow, awesome, all the CI passes otherwise 🎉

Also, wow, awesome, all the CI passes otherwise 🎉

Before this is merged, one question. I've had to change test_reloading as multiarray is the first extension module imported, and hence we are now hitting the defensive error in

Is it worth adding a special case for where we detect re-initialisation? I've not done so currently because it is less work, but I'd be remiss for not asking.

A

Would be nice to avoid printing out that monster if it isn't too hard, considering that it is -- for once -- really unrelated.

I've updated _core/__init__.py to avoid raising the wall-of-text error for the reinitialisation exception, hopefully this works.

A