Skip to content

Support free threaded builds #292

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
geertj opened this issue Oct 19, 2024 · 1 comment · May be fixed by #301
Open

Support free threaded builds #292

geertj opened this issue Oct 19, 2024 · 1 comment · May be fixed by #301

Comments

@geertj
Copy link

geertj commented Oct 19, 2024

Importing lz4 in a free-threaded Python re-enables the GIL with the following warning.

$ python
Python 3.13.0 experimental free-threading build (main, Oct 19 2024, 12:34:19) [GCC 11.4.1 20230605 (Red Hat 11.4.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import lz4
<frozen importlib._bootstrap>:488: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'lz4._version', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.
@rgommers
Copy link

rgommers commented Feb 18, 2025 •
edited
Loading

Free-threading support would be great. I had a look at what it would take. The README says:
"The bindings drop the GIL when calling in to the underlying LZ4 library, and is thread safe. An extensive test suite is included." and there aren't any open thread-safety issues - which looks like good news to me. The TODOs here should be something like:

  • Audit Python bindings and declare them free-threading compatible (xref https://py-free-threading.github.io/porting/#updating-extension-modules).
  • Run the test suite with pytest-run-parallel to find potential issues, and fix them.
  • Run the test suite under ThreadSanitizer. Should be feasible, since there are no runtime dependencies aside from CPython itself.
  • Add cp313t-* to CI to build free-threading wheels.

Note that this is the first time I've looked at this repo, so I might be missing known issues or code that needs closer inspection. Any suggestions here will be very useful.

Our team at Quansight working on free-threading support in widely used open source projects is happy to help and work on this. Please do let us know if you'd prefer us to hold off for any reason.

@andfoy andfoy linked a pull request Mar 17, 2025 that will close this issue
Open
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Enable CPython free-threaded wheel builds andfoy/python-lz4
2 participants
@rgommers @geertj