Thanks @andfoy, that's super helpful!

Tests are now passing locally in my fork: https://github.com/andfoy/scikit-image/actions/runs/9900010082/job/27349985531

Thanks again, this looks good to me.

Now, how do we shore up our test suite to actually create parallel faults if they are present? I suspect our current suite simply won't catch those.

Well, the usual first step is to identify which classes or functions may have a shared state (usually a big culprit for concurrency issues).

I did a similar exercise throughout scipy.interpolate scipy/scipy#20671 and scipy.spatial scipy/scipy#20655, where we found several cases of classes that had a shared state.

However, this is a very time consuming exercise that should be conducted with planning ahead, i.e., first making sure that the most popular functionality of scikit-image is tested for concurrency first.

I don't know how much budget we can spend here from the current Quansight labs project, maybe @rgommers knows more about how we can help the project to improve its parallelism support.

Well, the usual first step is to identify which classes or functions may have a shared state (usually a big culprit for concurrency issues).

Agreed. I'm not sure that there is much shared/global state - if there is, maintainers often know where it's located.

In scikit-image I believe there's a lot of Cython and Pythran code, and no C/C++ code (right?). Pythran code is fairly constrained in how it can be written - I believe individual numerical kernels are safe. For Cython code that's written in C-like ways, it can have thread-safety issues. I suggest auditing it quickly for odd patterns. And maybe @stefanv can suggest a couple of the most heavily used and complex functions to write some new parallel tests for? That tends to be pretty quick to do, and if they all pass that gives a good amount of confidence (plus it's a test pattern that's then easy for future contributors to emulate).

At that point I would stop unless scikit-image-specific bugs show up. I'm hoping that that won't be the case. Given the amount of Cython code and there being both known and unknown issues in Cython, I think it's perhaps more likely to hit Cython (or NumPy, or CPython) issues than problems in scikit-image itself.

I'd also suggest opening a tracking issue to capture the plans/progress, that's easier to find that a merged PR.

Should we also publish a nightly wheel for packages that test against us?

Thanks for guidance on testing! We do have parallel tests for some of our Cython code, and hopefully that catches the worst of it. All of our C++ code comes via Cython, but we do have some C code for phase unwrapping (unwrap_2d_ljmu.c and unwrap_3d_ljmu.c); I'll add some parallel testing to those functions.

How does Cython's freethreading_compatible flag interact with the PYTHON_GIL variable; do we need to modify our Meson build? Maybe @ngoldbaum has thoughts.

Should we also publish a nightly wheel for packages that test against us?

I can open a follow up PR to upload free-threaded nightly wheels to the Scientific Python Anaconda distribution channel

How does Cython's freethreading_compatible flag interact with the PYTHON_GIL variable; do we need to modify our Meson build?

The freethreading-compatible sets the default. PYTHON_GIL=0 overrides the default (so if an extension is not yet marked as safe, it's forced to run without the GIL anyway). Using PYTHON_GIL=0 may be useful for the next little while, until all extension modules in all test dependencies are marked as safe and those nightly wheels have propagated.

xref numpy/numpy#26950 for the pattern to use the new freethreading-compatible directive.

@stefanv @lagru 140a4dc preemptively marks all scikit-image modules as free-threading safe, however, for this to be true, it would require an extensive lookup on each of scikit-image's modules to guarantee that they are indeed safe. I don't know if you prefer to that to be done in a separate PR?

We only have one source of C code:

./restoration/unwrap_2d_ljmu.c
./restoration/unwrap_3d_ljmu.c

If, in addition, we can guarantee that Cython and Pythran codes are safe, then I think we're good (I don't currently have enough knowledge to perform such a check).

@stefanv, thanks for the references! After looking at them, they should not have any issues regarding C module registration, since they are being included as part of other Cython source.

From my side, this PR is ready

Two approvals and the one CI failure is unrelated. Can this PR be merged?

Should we also publish a nightly wheel for packages that test against us?

That does seem like a good idea. In other projects we've done that as a next step after adding regular CI. And in NumPy we've now merged the default and free-threaded wheel builds; 2.1.0 should publish cp313t wheels to PyPI.

Thanks for helping us get this testing in! And sorry for the small stall at the end. It's easy to forget PRs when waiting on the CI. 🙈

Let me open a followup PR to upload nightly wheels