gh-131591: Implement PEP 768 #131937
gh-131591: Implement PEP 768 #131937
Conversation
pablogsal
commented
Mar 31, 2025
•
- Issue: Implement PEP 768 – Safe external debugger interface for CPython #131591
Fix syntax error
The PyRuntime section can be found in either the main executable (python.exe) or the Python DLL (pythonXY.dll, where X and Y represent the major and minor version numbers). Scan the modules of the process to locate the PyRuntime address in one of these modules.
Ensure that the caller and target processes have matching architectures before proceeding. Attaching to a 64-bit process from a 32-bit process will fail when using CreateToolhelp32Snapshot, and attempting to attach to a 32-bit process from a 64-bit process may cause issues during PE file parsing. To avoid potential errors abort the operation if the architectures differ.
This reverts commit 746ecfc. That commit isn't correct. It conflates the return from `IsWow64Process` (whether the process is running under 32-bit emulation on a 64-bit process) with whether the process is 64-bit. A 32-bit process on 32-bit Windows would have `IsWow64Process` set our `isWow64` flag to `FALSE`, and we'd then incorrectly return `TRUE` from `is_process64Bit`.
…_support External debugger Windows support
This is useful because debuggers will write to this variable remotely, and it's helpful for it to have a well known size rather than one that could vary per platform.
Require the flags for turning this on to be set to exactly 1 to avoid accidentally triggering remote debugging in the case of heap corruption. Make a heap copy of the script path before using it to avoid the buffer being overwritten while we're still using it by another debugger.
This avoids the need to reopen it by (wide character) path.
Previously it was leaked if `PyObject_AsFileDescriptor` failed.
I think this was a copy paste error.
The remote process must be a compatible version. Explain our compatibility rules.
We can clean this up by freeing strings as soon as we're done with them, and remove some duplicate code.
This was documented as working, but in fact we assumed that the path would always be a unicode string. Factor the version that only accepts a unicode string into a helper function, and have the caller call `os.fsdecode` on the input path, which accepts either a byte string or a unicode string and always returns a unicode string.
Previously we were returning the number of bytes, which we didn't need and would never be different than the argument that was passed into the function.
|
Working on my review now - have a few minor changes so far, so don't merge without me |
|
Addressed the docs review round from @zooba. @godlygeek is working on your second round |
|
Had to fix also some merge conflicts :( |
|
@zooba I think we have addressed everything. |
|
Solved more merge conflicts |
| env = os.environ.copy() | ||
| env['PYTHON_DISABLE_REMOTE_DEBUG'] = '1' | ||
|
|
||
| _, out, err = assert_python_failure('-c', f'import os, sys; sys.remote_exec(os.getpid(), "{script}")', **env) |
There was a problem hiding this comment.
Random thought - should we special-case (or disallow) calling remote_exec on ourselves? It seems like it could be useful, but there's probably overheads involved that could be skipped. Alternatively, just saying "it's not for this" and blocking it also seems fine to me, though it'd complicate the tests.
There was a problem hiding this comment.
Nah, it actually requires no work whatsoever on our part, which is very cool! And we use it all the time for testing and developing since having to attach debuggers to two or even 3 processes is much much harder.
I am happy to keep it unless you feel strongly
| PyExc_RuntimeError, | ||
| "Memory reading is not supported on this platform"); | ||
| return -1; | ||
| Py_UNREACHABLE(); |
There was a problem hiding this comment.
Does this do something if it is actually reached? I'd guess Py_FatalError is the only real option, since it doesn't know how to return an error code.
There was a problem hiding this comment.
They are equivalent. This one crashes with the funny message from xkcd:
Lines 152 to 179 in 275056a
There was a problem hiding this comment.
I don't know about the others, but it's not runtime-safe for execution to actually reach __assume(0) - it implies genuine lack of reachability, and UB if it's reached (which probably crashes, but it's not guaranteed). Most of the time we won't get the funny messages.
It shouldn't matter in this case, because we have Windows code everywhere you've used it, but we should more clearly decide how we want to handle Py_UNREACHABLE.
There was a problem hiding this comment.
I think this is the case, we should only reach this I we really screwed up the guards
There was a problem hiding this comment.
Maybe we can blow up at compile time?
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
|
I'm addressed some of the last comments. We will be filing smaller PRs to improve the error messages and other smaller details raised during the review, but I think the best route is landing this PR as it's already 2k of changes that are attracting merge conflicts like crazy. Thanks a lot everyone for the reviews and congrats to @ivonastojanovic and @godlygeek |
|
P.S. Congrats also means "more work is coming" 😉 |
|
|
Ah, yes, the inevitable fall of the build bots. Will address this immediately |
Co-authored-by: Ivona Stojanovic <stojanovic.i@hotmail.com> Co-authored-by: Matt Wozniski <godlygeek@gmail.com>