Skip to content

gh-137400: Fix a crash when disabling profiling across all threads #137471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Aug 11, 2025
Merged

gh-137400: Fix a crash when disabling profiling across all threads #137471

merged 1 commit into from
Aug 11, 2025
+46 −2

Conversation

colesbury
Copy link
Contributor

@colesbury colesbury commented Aug 6, 2025 •
edited by bedevere-app bot
Loading

The PyEval_SetProfileAllThreads function and other related functions had a race condition on tstate->c_profilefunc that could lead to a crash when disable profiling or tracing on all threads while another thread is starting to profile or trace a a call.

There are still potential crashes when threads exit concurrently with profiling or tracing be enabled/disabled across all threads.

@colesbury
pythongh-137400: Fix a crash when disabling profiling across all threads
5e5e4f1 
The `PyEval_SetProfileAllThreads` function and other related functions
had a race condition on `tstate->c_profilefunc` that could lead to a
crash when disable profiling or tracing on all threads while another
thread is starting to profile or trace a a call.

There are still potential crashes when threads exit concurrently with
profiling or tracing be enabled/disabled across all threads.
@bedevere-app bedevere-app bot mentioned this pull request Aug 6, 2025
Open
@godlygeek godlygeek mentioned this pull request Aug 6, 2025
Closed
@colesbury
Copy link
Contributor Author

colesbury commented Aug 7, 2025 •
edited
Loading

Closing in favor of the more complete PR: #137518

@colesbury colesbury closed this Aug 7, 2025
@colesbury colesbury reopened this Aug 7, 2025
@colesbury colesbury marked this pull request as ready for review August 7, 2025 18:55
@colesbury colesbury added topic-free-threading needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Aug 7, 2025
@colesbury colesbury requested review from hugovk and pablogsal August 7, 2025 18:57
@colesbury colesbury added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Aug 7, 2025
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @colesbury for commit 5e5e4f1 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F137471%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Aug 7, 2025
@colesbury colesbury merged commit 3626928 into python:main Aug 11, 2025
107 checks passed
@miss-islington-app
Copy link

Thanks @colesbury for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

@colesbury colesbury deleted the gh-137400-profileall branch August 11, 2025 15:41
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Aug 11, 2025
@colesbury @miss-islington
pythongh-137400: Fix a crash when disabling profiling across all thre…
77f9ca6 
…ads (pythongh-137471)

The `PyEval_SetProfileAllThreads` function and other related functions
had a race condition on `tstate->c_profilefunc` that could lead to a
crash when disable profiling or tracing on all threads while another
thread is starting to profile or trace a a call.

There are still potential crashes when threads exit concurrently with
profiling or tracing be enabled/disabled across all threads.
(cherry picked from commit 3626928)

Co-authored-by: Sam Gross <colesbury@gmail.com>
@miss-islington-app
Copy link

Sorry, @colesbury, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 362692852f13cdd1d33cc7ed35c0cbac7af1a785 3.13

@bedevere-app
Copy link

bedevere-app bot commented Aug 11, 2025

GH-137648 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Aug 11, 2025
colesbury added a commit to colesbury/cpython that referenced this pull request Aug 11, 2025
@colesbury
[3.13] pythongh-137400: Fix a crash when disabling profiling across a…
88e3cea 
…ll threads (pythongh-137471)

The `PyEval_SetProfileAllThreads` function and other related functions
had a race condition on `tstate->c_profilefunc` that could lead to a
crash when disable profiling or tracing on all threads while another
thread is starting to profile or trace a a call.

There are still potential crashes when threads exit concurrently with
profiling or tracing be enabled/disabled across all threads.
(cherry picked from commit 3626928)

Co-authored-by: Sam Gross <colesbury@gmail.com>
@bedevere-app
Copy link

bedevere-app bot commented Aug 11, 2025

GH-137649 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Aug 11, 2025
colesbury added a commit that referenced this pull request Aug 11, 2025
@colesbury
[3.13] gh-137400: Fix a crash when disabling profiling across all thr…
e236568 
…eads (gh-137471) (gh-137649)

The `PyEval_SetProfileAllThreads` function and other related functions
had a race condition on `tstate->c_profilefunc` that could lead to a
crash when disable profiling or tracing on all threads while another
thread is starting to profile or trace a a call.

There are still potential crashes when threads exit concurrently with
profiling or tracing be enabled/disabled across all threads.
(cherry picked from commit 3626928)
hugovk pushed a commit that referenced this pull request Aug 12, 2025
@miss-islington @colesbury
[3.14] gh-137400: Fix a crash when disabling profiling across all thr…
4ebd928 
…eads (gh-137471) (#137648)

Co-authored-by: Sam Gross <colesbury@gmail.com>
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Android 3.14 (tier-3) has failed when building commit 4ebd928.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1794/builds/435) and take a look at the build logs.
  4. Check if the failure is related to this commit (4ebd928) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1794/builds/435

Summary of the results of the build (if available):

Click to see traceback logs 
remote: Enumerating objects: 24, done.        
remote: Counting objects:   4% (1/24)        
remote: Counting objects:   8% (2/24)        
remote: Counting objects:  12% (3/24)        
remote: Counting objects:  16% (4/24)        
remote: Counting objects:  20% (5/24)        
remote: Counting objects:  25% (6/24)        
remote: Counting objects:  29% (7/24)        
remote: Counting objects:  33% (8/24)        
remote: Counting objects:  37% (9/24)        
remote: Counting objects:  41% (10/24)        
remote: Counting objects:  45% (11/24)        
remote: Counting objects:  50% (12/24)        
remote: Counting objects:  54% (13/24)        
remote: Counting objects:  58% (14/24)        
remote: Counting objects:  62% (15/24)        
remote: Counting objects:  66% (16/24)        
remote: Counting objects:  70% (17/24)        
remote: Counting objects:  75% (18/24)        
remote: Counting objects:  79% (19/24)        
remote: Counting objects:  83% (20/24)        
remote: Counting objects:  87% (21/24)        
remote: Counting objects:  91% (22/24)        
remote: Counting objects:  95% (23/24)        
remote: Counting objects: 100% (24/24)        
remote: Counting objects: 100% (24/24), done.        
remote: Compressing objects:   7% (1/13)        
remote: Compressing objects:  15% (2/13)        
remote: Compressing objects:  23% (3/13)        
remote: Compressing objects:  30% (4/13)        
remote: Compressing objects:  38% (5/13)        
remote: Compressing objects:  46% (6/13)        
remote: Compressing objects:  53% (7/13)        
remote: Compressing objects:  61% (8/13)        
remote: Compressing objects:  69% (9/13)        
remote: Compressing objects:  76% (10/13)        
remote: Compressing objects:  84% (11/13)        
remote: Compressing objects:  92% (12/13)        
remote: Compressing objects: 100% (13/13)        
remote: Compressing objects: 100% (13/13), done.        
remote: Total 13 (delta 11), reused 0 (delta 0), pack-reused 0 (from 0)        
From https://github.com/python/cpython
 * branch                    3.14       -> FETCH_HEAD
Note: switching to '4ebd928b0096ca1dd78d5d3e6c787dbc40a43c11'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 4ebd928b009 [3.14] gh-137400: Fix a crash when disabling profiling across all threads (gh-137471) (#137648)
Switched to and reset branch '3.14'

configure: WARNING: no system libmpdec found; falling back to bundled libmpdec (deprecated and scheduled for removal in Python 3.16)
configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100  192k  100  192k    0     0  1474k      0 --:--:-- --:--:-- --:--:-- 1474k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100 42455  100 42455    0     0   318k      0 --:--:-- --:--:-- --:--:--  318k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

  6 5041k    6  333k    0     0  2019k      0  0:00:02 --:--:--  0:00:02 2019k
100 5041k  100 5041k    0     0  24.6M      0 --:--:-- --:--:-- --:--:--  135M
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100 1257k  100 1257k    0     0  9634k      0 --:--:-- --:--:-- --:--:-- 9634k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100  635k  100  635k    0     0  5066k      0 --:--:-- --:--:-- --:--:-- 5066k
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0

100  479k  100  479k    0     0  2199k      0 --:--:-- --:--:-- --:--:-- 2199k
../../configure: line 4071: pkg-config: command not found
configure: WARNING: no system libmpdec found; falling back to bundled libmpdec (deprecated and scheduled for removal in Python 3.16)
configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

../../Python/fileutils.c:460:1: warning: unused function 'decode_current_locale' [-Wunused-function]
  460 | decode_current_locale(const char* arg, wchar_t **wstr, size_t *wlen,
      | ^~~~~~~~~~~~~~~~~~~~~
../../Python/fileutils.c:679:1: warning: unused function 'encode_current_locale' [-Wunused-function]
  679 | encode_current_locale(const wchar_t *text, char **str,
      | ^~~~~~~~~~~~~~~~~~~~~
2 warnings generated.
../../Modules/_localemodule.c:148:1: warning: unused function 'is_all_ascii' [-Wunused-function]
  148 | is_all_ascii(const char *str)
      | ^~~~~~~~~~~~
1 warning generated.
../../Modules/_hacl/Lib_Memzero0.c:66:6: warning: "Your platform does not support any safe implementation of memzero -- consider a pull request!" [-W#warnings]
   66 |     #warning "Your platform does not support any safe implementation of memzero -- consider a pull request!"
      |      ^
1 warning generated.

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  8784  100  8784    0     0  67630      0 --:--:-- --:--:-- --:--:-- 68093
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  2894  100  2894    0     0   9811      0 --:--:-- --:--:-- --:--:--  9843
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 43504  100 43504    0     0   696k      0 --:--:-- --:--:-- --:--:--  708k

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kumaraditya303 kumaraditya303 kumaraditya303 approved these changes

@hugovk hugovk Awaiting requested review from hugovk

@pablogsal pablogsal Awaiting requested review from pablogsal

Assignees
No one assigned
Labels
topic-free-threading
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@colesbury @bedevere-bot @kumaraditya303