-
-
Notifications
You must be signed in to change notification settings - Fork 11.1k
Description
Describe the issue:
memmap
segfaults on python 3.13t when two threads try to open the same file (that didn't previously exist) with mode='w+'.
Expected behaviour
Ideally, all threads read and write on the same file.
This should be in theory possible with three syscalls:
open(fname, O_CREAT)
- resize, unless another thread has already resized it
- mmap
If the above was for some reason impossible (I particularly expect trouble on NFS and similar filesystems), the segfault should be replaced with a meaningful exception (that doesn't leak resources).
Reproduce the code example:
- deploy numpy through https://github.com/rgommers/pixi-dev-scipystack/
- apply patch (see DEV: stop monkeypatching and use of
autouse=True
fixtures by default in test suite #29090):
--- a/numpy/conftest.py
+++ b/numpy/conftest.py
@@ -147,9 +147,10 @@ def check_fpu_mode(request):
def add_np(doctest_namespace):
doctest_namespace['np'] = numpy
+
@pytest.fixture(autouse=True)
-def env_setup(monkeypatch):
- monkeypatch.setenv('PYTHONHASHSEED', '0')
+def env_setup():
+ os.environ['PYTHONHASHSEED'] = '0'
- pixi run test-nogil -- -k TestMemmap --parallel-threads=32
Error message:
All tests that use tmp_path
segfault; e.g.
numpy/numpy/_core/tests/test_memmap.py
Lines 62 to 67 in ff1d6cc
def test_open_with_filename(self, tmp_path): | |
tmpname = tmp_path / 'mmap' | |
fp = memmap(tmpname, dtype=self.dtype, mode='w+', | |
shape=self.shape) | |
fp[:] = self.data[:] | |
del fp |
These tests are thread-unsafe due to Quansight-Labs/pytest-run-parallel#14, which causes all threads to call memmap on the same file name at once.
Python and NumPy Versions:
python 3.13t
numpy git tip 2025-06-04
Runtime Environment:
32-core x86_64 Linux host
/tmp
is a ext4 mountpoint.