Skip to content

BUG: memmap multithreaded segfault on on-disk collision  #29126

@crusaderky

Description

@crusaderky

Describe the issue:

memmap segfaults on python 3.13t when two threads try to open the same file (that didn't previously exist) with mode='w+'.

Expected behaviour

Ideally, all threads read and write on the same file.
This should be in theory possible with three syscalls:

  1. open(fname, O_CREAT)
  2. resize, unless another thread has already resized it
  3. mmap

If the above was for some reason impossible (I particularly expect trouble on NFS and similar filesystems), the segfault should be replaced with a meaningful exception (that doesn't leak resources).

Reproduce the code example:

  1. deploy numpy through https://github.com/rgommers/pixi-dev-scipystack/
  2. apply patch (see DEV: stop monkeypatching and use of autouse=True fixtures by default in test suite #29090):
--- a/numpy/conftest.py
+++ b/numpy/conftest.py
@@ -147,9 +147,10 @@ def check_fpu_mode(request):
 def add_np(doctest_namespace):
     doctest_namespace['np'] = numpy
 
+
 @pytest.fixture(autouse=True)
-def env_setup(monkeypatch):
-    monkeypatch.setenv('PYTHONHASHSEED', '0')
+def env_setup():
+    os.environ['PYTHONHASHSEED'] = '0'
  1. pixi run test-nogil -- -k TestMemmap --parallel-threads=32

Error message:

All tests that use tmp_path segfault; e.g.

def test_open_with_filename(self, tmp_path):
tmpname = tmp_path / 'mmap'
fp = memmap(tmpname, dtype=self.dtype, mode='w+',
shape=self.shape)
fp[:] = self.data[:]
del fp

These tests are thread-unsafe due to Quansight-Labs/pytest-run-parallel#14, which causes all threads to call memmap on the same file name at once.

Python and NumPy Versions:

python 3.13t
numpy git tip 2025-06-04

Runtime Environment:

32-core x86_64 Linux host
/tmp is a ext4 mountpoint.

Metadata

Metadata

Assignees

No one assigned

    Labels

    00 - Bug39 - free-threadingPRs and issues related to support for free-threading CPython (a.k.a. no-GIL, PEP 703)

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions