Skip to content

GH-132532: Add new DSL macros to better declare semantics of exits at ends of instructions/uops. #137098

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 6 commits into from
Aug 9, 2025
Merged

GH-132532: Add new DSL macros to better declare semantics of exits at ends of instructions/uops. #137098

merged 6 commits into from
Aug 9, 2025

Conversation

markshannon
Copy link
Member

@markshannon markshannon commented Jul 25, 2025 •
edited
Loading

This PR changes adds the ability to de-opt at the end an instruction.
Exits from tier 2 jump to the next instruction in tier 1.
In the case of deopting due to the eval breaker being set, _Py_HandlePending is called before resuming tier 1.

I am taking this approach instead of checking the eval breaker before each call, as this does not cause any observable behavior change. For comparison see https://github.com/python/cpython/compare/main...faster-cpython:cpython:check-periodic-before-call?expand=1, noting the necessary changes to tests and threading module.

Although this approach adds a bit more complexity to the optimizer, it doesn't need any changes to the library or tests.

@bedevere-app bedevere-app bot mentioned this pull request Jul 25, 2025
Open
@markshannon markshannon marked this pull request as ready for review July 29, 2025 11:01
@markshannon markshannon mentioned this pull request Jul 29, 2025
Closed
brandtbucher
brandtbucher approved these changes Aug 7, 2025
View reviewed changes
Copy link
Member

@brandtbucher brandtbucher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems correct. My only issue is with all of the new names... naming is hard.

I feel like the chosen names are a bit hard to parse and reason about, and suggested some alternatives. But they're just suggestions, nothing blocking.

@markshannon markshannon merged commit af15e1d into python:main Aug 9, 2025
69 checks passed
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL8 Refleaks 3.x (tier-3) has failed when building commit af15e1d.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/75/builds/3149) and take a look at the build logs.
  4. Check if the failure is related to this commit (af15e1d) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/75/builds/3149

Failed tests:

  • test.test_multiprocessing_fork.test_processes

Failed subtests:

  • test_interrupt - test.test_multiprocessing_fork.test_processes.WithProcessesTestProcess.test_interrupt

Summary of the results of the build (if available):

==

Click to see traceback logs 
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 588, in test_interrupt
    exitcode = self._kill_process(multiprocessing.Process.interrupt)
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 570, in _kill_process
    self.assertEqual(join(), None)
                     ~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 250, in __call__
    return self.func(*args, **kwds)
           ~~~~~~~~~^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/multiprocessing/process.py", line 156, in join
    res = self._popen.wait(timeout)
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/multiprocessing/popen_fork.py", line 44, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
               ~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 566, in handler
    raise RuntimeError('join took too long: %s' % p)
RuntimeError: join took too long: <Process name='Process-2' pid=1263447 parent=1263444 started daemon>


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/multiprocessing/process.py", line 320, in _bootstrap
    self.run()
    ~~~~~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 518, in _sleep_some_event
    time.sleep(100)
    ~~~~~~~~~~^^^^^
KeyboardInterrupt
k


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 588, in test_interrupt
    exitcode = self._kill_process(multiprocessing.Process.interrupt)
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 570, in _kill_process
    self.assertEqual(join(), None)
                     ~~~~^^
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 250, in __call__
    return self.func(*args, **kwds)
           ~~~~~~~~~^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/multiprocessing/process.py", line 156, in join
    res = self._popen.wait(timeout)
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/multiprocessing/popen_fork.py", line 44, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
               ~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-rhel8-s390x.refleak/build/Lib/test/_test_multiprocessing.py", line 566, in handler
    raise RuntimeError('join took too long: %s' % p)
RuntimeError: join took too long: <Process name='Process-1713' pid=1262658 parent=1245698 started daemon>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@brandtbucher brandtbucher brandtbucher approved these changes

Assignees
No one assigned
Labels
skip news
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@markshannon @bedevere-bot @brandtbucher