[MRG] FIX SparseCoder with readonly parallel mmap #11346
Conversation
7010f33 to
62d2b7f
Compare
| # if they are larger than 1MB. The 4 accounts for float32 | ||
| # data type | ||
| n_samples = int(2e6) // (4 * n_features) | ||
| data = np.random.rand(n_samples, n_features).astype(np.float32) |
There was a problem hiding this comment.
Alternatively can just do data.setflags(write=False). That way having a lot of data is not necessary to generate the same problem. Did something similar in PR ( #11342 ) in case it helps.
There was a problem hiding this comment.
Unfortunately, I believe it's not possible to use this as in this particular case it's an internal data structure Xy (inderectly derived from the data) that is being mutated.
We could write a unittest for the private function _gram_omp tough.
|
I have added a short unittest based on @jakirkham's suggestion to use readonly flags. I am not sure if we should keep the On the + side, it's a good integration test that actually reflects the original problem from the user's point of view. On the - side, it's a bit slow to run (2.26s on my laptop) and I don't think we can easily make it run faster while still reproducing the original problem. |
|
CI was green everywhere, I merged. |
Fixes #5956.
I reused a faster version of the non-regression test of #5998 but the fix is more minimal.
I will directly merge if CI is green.