gh-135335: flush stdout/stderr in forkserver after preloading modules #135338
Conversation
…odules If a preloaded module writes to stdout or stderr, and the stream is buffered, child processes will inherit the buffered data after forking. Attempt to prevent this by flushing the streams after preload.
...or at least produce more useful output if they fail
| p.start() | ||
| p.join() | ||
| else: | ||
| assert 'a' in sys.modules |
There was a problem hiding this comment.
There is a problem with this test, see
https://github.com/python/cpython/actions/runs/15558229444/job/43803979297?pr=135338
Each test should be written in such a way that make -j test command can be runned w/o errors.
There was a problem hiding this comment.
Yes, for some reason it is failing to import the module on MacOS. Note the Ubuntu failure looks like an intermittent race condition in resource tracker, unrelated to the MacOS failure, or indeed to this change at all.
Unfortunately I don't currently have access to a MacOS system, and it is difficult to debug without that. Fork server just swallows the exceptions on import failure, so there are no diagnostics available from the failed CI runs. I'm arranging to get access to such a system sometime over the next few days, which will allow me to dump those errors and debug it.
I considered pushing some additional diagnostics out to gather more information, but I didn't want to clutter the PR, confuse the history, and add extra load to CI.
Another option would be to wait for #126631 to be fixed, which would allow rewriting the test to just use __main__ and drop creating and importing the module.
If a preloaded module writes to stdout or stderr, and the stream is buffered, child processes will inherit the buffered data after forking. Attempt to prevent this by flushing the streams after preload.