Skip to content

gh-134876: Add fallback for when process_vm_readv fails with ENOSYS #134878

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 7, 2025

Conversation

cakemanny
Copy link
Contributor

@cakemanny cakemanny commented May 29, 2025

This adds a fallback /proc/[pid]/mem from the proc(5) filesystem when process_vm_readv and process_vm_writev are not compiled into the kernel.

Regarding tests, these are covered by ./python -m test --match 'test_remote_pdb', but only on affected systems.
Do say if this merits a NEWS entry, I wasn't sure because it's a fix for not yet released python versions.

@cakemanny cakemanny requested a review from pablogsal as a code owner May 29, 2025 10:57
@python-cla-bot
Copy link

python-cla-bot bot commented May 29, 2025

All commit authors signed the Contributor License Agreement.

CLA signed

@bedevere-app
Copy link

bedevere-app bot commented May 29, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@cakemanny
Copy link
Contributor Author

I think the those tests are only failing because compilation fails on gcc-10. I'll have a look and see if I can obtain a copy and will update accordingly.

@cakemanny cakemanny force-pushed the fallback-to-proc-mem branch from 938b91c to 90b08e6 Compare May 29, 2025 12:28
@bedevere-app
Copy link

bedevere-app bot commented May 29, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app
Copy link

bedevere-app bot commented May 29, 2025

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@cakemanny cakemanny force-pushed the fallback-to-proc-mem branch from 90b08e6 to cbea7a2 Compare May 29, 2025 21:29
@bedevere-app
Copy link

bedevere-app bot commented May 29, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@cakemanny cakemanny force-pushed the fallback-to-proc-mem branch from cbea7a2 to 5fd8ff6 Compare May 29, 2025 21:44
@bedevere-app
Copy link

bedevere-app bot commented May 29, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@cakemanny cakemanny force-pushed the fallback-to-proc-mem branch from 5fd8ff6 to ef8ae45 Compare May 29, 2025 22:05
@bedevere-app
Copy link

bedevere-app bot commented May 29, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@cakemanny cakemanny force-pushed the fallback-to-proc-mem branch from ef8ae45 to f1eb6c0 Compare May 29, 2025 22:37
@bedevere-app
Copy link

bedevere-app bot commented May 29, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@pablogsal
Copy link
Member

@cakemanny one small comment: we squash merge so there is no need to rebase. It's easier to review if you just add commits on top so we don't need to review the full thing every iteration :)

@cakemanny
Copy link
Contributor Author

I have made the requested changes; please review again

@bedevere-app
Copy link

bedevere-app bot commented May 31, 2025

Thanks for making the requested changes!

@pablogsal: please review the changes made to this pull request.

@pablogsal
Copy link
Member

Excellent job @cakemanny 🚀

@pablogsal pablogsal added the needs backport to 3.14 bugs and security fixes label Jun 7, 2025
@pablogsal pablogsal merged commit ac9c343 into python:main Jun 7, 2025
46 of 49 checks passed
@miss-islington-app
Copy link

Thanks @cakemanny for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 7, 2025
…OSYS (pythonGH-134878)

(cherry picked from commit ac9c343)

Co-authored-by: Daniel Golding <goldingd89@gmail.com>
@bedevere-app
Copy link

bedevere-app bot commented Jun 7, 2025

GH-135240 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Jun 7, 2025
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x (tier-1) has failed when building commit ac9c343.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/345/builds/11555) and take a look at the build logs.
  4. Check if the failure is related to this commit (ac9c343) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/345/builds/11555

Failed tests:

  • test_tokenize
  • test.test_concurrent_futures.test_thread_pool
  • test.test_multiprocessing_forkserver.test_misc
  • test_math

Failed subtests:

  • test_lock - test.test_multiprocessing_forkserver.test_misc.TestForkAwareThreadLock.test_lock
  • test_random_files - test.test_tokenize.TestRoundtrip.test_random_files
  • test_sumprod_stress - test.test_math.MathTests.test_sumprod_stress

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_tokenize.py", line 2027, in test_random_files
    self.check_roundtrip(f)
    ~~~~~~~~~~~~~~~~~~~~^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_tokenize.py", line 1860, in check_roundtrip
    self.assertEqual(tokens2_from2, tokens2)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Lists differ: [(68,[121084 chars]'), (55, '.'), (1, '_box'), (4, ''), (6, ''), (6, ''), (0, '')] != [(68,[121084 chars]'), (55, '.'), (1, '_box')]


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/_test_multiprocessing.py", line 5658, in test_lock
    new_size = r.recv()
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/connection.py", line 256, in recv
    buf = self._recv_bytes()
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/connection.py", line 447, in _recv_bytes
    buf = self._recv(4)
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/multiprocessing/connection.py", line 416, in _recv
    raise EOFError
EOFError


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_math.py", line 1471, in test_sumprod_stress
    self.assertEqual(
    ~~~~~~~~~~~~~~~~^
        run(baseline_sumprod, *args),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        run(sumprod, *args),
        ^^^^^^^^^^^^^^^^^^^^
        args,
        ^^^^^
    )
    ^
AssertionError: Tuples differ: (None, <class 'float'>, '181.59375') != (None, <class 'float'>, '161.59375')


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_tokenize.py", line 2027, in test_random_files
    self.check_roundtrip(f)
    ~~~~~~~~~~~~~~~~~~~~^^^
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_tokenize.py", line 1860, in check_roundtrip
    self.assertEqual(tokens2_from2, tokens2)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Lists differ: [(68,[23523 chars]'\n')] != [(68,[23523 chars]'\n'), (1, 'then'), (55, '='), (1, 'time'), (5[3888 chars] '')]

pablogsal pushed a commit that referenced this pull request Jun 7, 2025
…NOSYS (GH-134878) (#135240)

gh-134876: Add fallback for when process_vm_readv fails with ENOSYS (GH-134878)
(cherry picked from commit ac9c343)

Co-authored-by: Daniel Golding <goldingd89@gmail.com>
@cakemanny
Copy link
Contributor Author

The failures on that buildbot don't look like they should be related to my changes.
I saw in https://buildbot.python.org/#/builders/345/builds/11547 test_math had a re-run recently, so I guess that one could be flaky sometimes?
And test_multiprocessing_forkserver is listed under expected transient failures: https://devguide.python.org/testing/buildbots/index.html#transient-failures .

test_tokenize failure is rather puzzling though.
Unless remote debugging is being used as part of the test infrastructure I guess it also cannot be related?

I think all I would be able to do right now would be to keep an eye open on subsequent runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants