Skip to content

[3.13] gh-87135: Hang non-main threads that attempt to acquire the GIL during finalization (GH-105805) #137827

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: 3.13
Choose a base branch
from

Conversation

gpshead
Copy link
Member

@gpshead gpshead commented Aug 15, 2025

Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This backport documents that the PyThread_exit_thread public C API should not be used (deprecated in 3.14) as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option.

(cherry picked from commit 8cc5aa4)


📚 Documentation preview 📚: https://cpython-previews--137827.org.readthedocs.build/

…the GIL during finalization (pythonGH-105805)

Instead of surprise crashes and memory corruption, we now hang threads that attempt to re-enter the Python interpreter after Python runtime finalization has started. These are typically daemon threads (our long standing mis-feature) but could also be threads spawned by extension modules that then try to call into Python. This marks the `PyThread_exit_thread` public C API as deprecated as there is no plausible safe way to accomplish that on any supported platform in the face of things like C++ code with finalizers anywhere on a thread's stack. Doing this was the least bad option.

(cherry picked from commit 8cc5aa4)

Co-authored-by: Jeremy Maitin-Shepard <jeremy@jeremyms.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
@gpshead gpshead force-pushed the backport-8cc5aa4-3.13 branch from 4f77306 to 977b0d4 Compare August 15, 2025 16:28
@gpshead gpshead marked this pull request as ready for review August 15, 2025 16:30
@gpshead gpshead added type-bug An unexpected behavior, bug, or error interpreter-core (Objects, Python, Grammar, and Parser dirs) 3.13 bugs and security fixes labels Aug 15, 2025
@gpshead gpshead self-assigned this Aug 15, 2025
@gpshead gpshead force-pushed the backport-8cc5aa4-3.13 branch from 977b0d4 to f412737 Compare August 15, 2025 16:34
@gpshead gpshead added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Aug 15, 2025
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @gpshead for commit f412737 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F137827%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Aug 15, 2025
Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot say it LGTM, but it looks less bad than anything else. ☹️ It seems there are no good solutions here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.13 bugs and security fixes awaiting merge interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants