gh-135953: Profile a module or script with sampling profiler #136777
Conversation
5a351da to
1bf048a
Compare
|
🤖 New build scheduled with the buildbot fleet by @pablogsal for commit 1bf048a 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136777%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
There was a problem hiding this comment.
Pull Request Overview
This PR extends the sampling profiler to support profiling modules and scripts by launching them in subprocesses, in addition to the existing PID-based profiling. The change improves the usability of the profiler by allowing users to profile Python programs from startup rather than only attaching to existing processes.
- Adds
-mand script filename arguments as mutually exclusive alternatives to the existing-p PIDoption - Updates argument parsing to handle the new target modes with proper validation
- Implements subprocess management with graceful termination handling
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| Lib/profile/sample.py | Adds argument parsing for module and script modes, implements subprocess launching and management |
| Lib/test/test_sample_profiler.py | Updates existing CLI tests to use -p flag and adds comprehensive test coverage for new module/script functionality |
Comments suppressed due to low confidence (1)
Lib/test/test_sample_profiler.py:1637
- The test uses
contextlib.chdir()to change directories for module discovery, but doesn't test the case where the module is not found or the directory change fails. Consider adding a test case for module resolution errors.
contextlib.chdir(tempdir.name),
Lib/profile/sample.py
Outdated
| if not(args.pid or args.module or args.script): | ||
| parser.error( | ||
| "You must specify either a process ID (-p), a module (-m), or a script to run." | ||
| ) |
There was a problem hiding this comment.
The condition not(args.pid or args.module or args.script) is redundant since the mutually exclusive group is already marked as required=True. The argparse library will automatically enforce that one of these options is provided.
| if not(args.pid or args.module or args.script): | |
| parser.error( | |
| "You must specify either a process ID (-p), a module (-m), or a script to run." | |
| ) | |
| # The mutually exclusive group already enforces that one of these arguments is required. |
Copilot uses AI. Check for mistakes.
Lib/profile/sample.py
Outdated
| process = subprocess.Popen(cmd) | ||
|
|
||
| try: | ||
| exit_code = process.wait(timeout=0.1) |
There was a problem hiding this comment.
The hardcoded timeout of 0.1 seconds is a magic number. Consider defining this as a named constant or making it configurable, as some programs may have longer startup times.
| exit_code = process.wait(timeout=0.1) | |
| exit_code = process.wait(timeout=DEFAULT_PROCESS_WAIT_TIMEOUT) |
Copilot uses AI. Check for mistakes.
| if process.poll() is None: | ||
| process.terminate() | ||
| try: | ||
| process.wait(timeout=2) |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
Thanks @AA-Turner, I think I addressed all your comments. |
Add `-m` and `filename` arguments to the sampling profiler to launch the specified Python program in a subprocess and start profiling it. Previously only a PID was accepted, this can now be done by passing `-p PID`.
These args are already mutually exclusive, but we need to check if at least on module argument has been passed.
In this case the subprocess will go into zombie state until we can poll it. We can simply assume this is the case if it's still detected as running when we get a ValueError.
Improve the return value check to be able to raise a ProcessLookupError when the remote process is not available. Mach uses composite error values where higher error values indicate specific subsystems. We can use the err_get_code function to mask the higher bits to make our error checking more robust in case the subsystem bits are set. For example, in some situations if the process is in zombie state, we can get KERN_NO_SPACE (0x3) but the actual return value is 0x10000003 which indicates a specific subsystem, thus we need to use err_get_code to extract the error value. This also improves how KERN_INVALID_ARGUMENT is handled to check whether we got a generic invalid argument error, or if the process is no longer accessible.
758af04 to
289d868
Compare
289d868 to
0338ee1
Compare
|
🤖 New build scheduled with the buildbot fleet by @pablogsal for commit 0338ee1 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136777%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
|
@lkollar Seems |
|
why is the |
|
Oh, I think this is because we are sampling before the other process calls |
a7fa762 to
2576f8b
Compare
e40aa09 to
3847fff
Compare
|
🤖 New build scheduled with the buildbot fleet by @pablogsal for commit 3847fff 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F136777%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
Add
-mandfilenamearguments to the sampling profiler to launch the specified Python program in a subprocess and start profiling it. Previously only a PID was accepted, this can now be done by passing-p PID.