Skip to content

python -m pdb -p fails when CONFIG_CROSS_MEMORY_ATTACH not set in kernel config #134876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
cakemanny opened this issue May 29, 2025 · 2 comments
Closed
Labels
3.14 bugs and security fixes 3.15 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error

Comments

@cakemanny
Copy link
Contributor

cakemanny commented May 29, 2025

Bug report

Bug description:

It's possible that a Linux kernel is built with the kernel config variable CONFIG_CROSS_MEMORY_ATTACH not set. (This is the case for the kernel that comes with Flatcar Container Linux and for CoreOS, the former of which is used for kubernetes nodes by the cloud provider we use).

In this case, using python -m pdb <some python PID> fails and it looks like this:

root@bc6b29d0894d:/src/cpython# ./python -c $'import time;\nwhile True: time.sleep(1)' &
[1] 17106
root@bc6b29d0894d:/src/cpython# ./python -m pdb -p $!
OSError: [Errno 38] Function not implemented

The above exception was the direct cause of the following exception:

OSError: process_vm_readv failed for PID 17106 at address 0xaaaab1f8d038 (size 760, partial read 0 bytes): Function not implemented

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/src/cpython/Lib/runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
                     "__main__", mod_spec)
  File "/src/cpython/Lib/runpy.py", line 88, in _run_code
    exec(code, run_globals)
    ~~~~^^^^^^^^^^^^^^^^^^^
  File "/src/cpython/Lib/pdb.py", line 3609, in <module>
    pdb.main()
    ~~~~~~~~^^
  File "/src/cpython/Lib/pdb.py", line 3540, in main
    attach(opts.pid, opts.commands)
    ~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/cpython/Lib/pdb.py", line 3424, in attach
    sys.remote_exec(pid, connect_script.name)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Failed to read debug offsets structure from remote process

This is a single command to repro on such a system

docker run -i --rm python:3.13 bash << 'EOF'  # the python version here is irrelevant, the image just has the right deps
newresolv=$(sed $'/nameserver/i\\\nnameserver 1.1.1.1' /etc/resolv.conf); echo "$newresolv" > /etc/resolv.conf  # resolv is wrong in my repro VM
mkdir src && cd src
git clone https://github.com/python/cpython.git --depth=10  # add --branch=3.14 to test 3.14
cd cpython
./configure --with-pydebug && make -j $(nproc)
./python -c $'import time;\nwhile True: time.sleep(1)' &
./python -m pdb -p $!
EOF

Further relevant version info for the above repro output:

root@bc6b29d0894d:/src/cpython# uname -a
Linux bc6b29d0894d 6.6.88-flatcar #1 SMP PREEMPT Tue Apr 29 23:08:45 -00 2025 aarch64 GNU/Linux

root@bc6b29d0894d:/src/cpython# ./python -VV
Python 3.15.0a0 (heads/main:e64395e, May 29 2025, 09:16:22) [GCC 12.2.0]

root@bc6b29d0894d:/src/cpython# git show | head -1
commit e64395e8eb8d3a9e35e3e534e87d427ff27ab0a5

root@bc6b29d0894d:/src/cpython# zcat /proc/config.gz  | grep CONFIG_CROSS_MEMORY_ATTACH
# CONFIG_CROSS_MEMORY_ATTACH is not set

Diagnosis:
When CONFIG_CROSS_MEMORY_ATTACH is not set, the syscall handler for process_vm_readv (and also process_vm_writev) is not compiled into the kernel and so it returns ENOSYS as errno.

If it's ok, I'd also like to / be happy to offer a PR to fix.
Specifically I'd go with the approach that I discovered someone contributed to py-spy, of falling back to the /proc/[pid]/mem file in the proc filesystem. I could even open such a PR pretty soon.

CPython versions tested on:

3.14, 3.15, CPython main branch

Operating systems tested on:

Linux

Linked PRs

@ZeroIntensity
Copy link
Member

(Hit the assign button on accident)

@ZeroIntensity ZeroIntensity added interpreter-core (Objects, Python, Grammar, and Parser dirs) 3.14 bugs and security fixes 3.15 new features, bugs and security fixes labels May 29, 2025
@gaogaotiantian
Copy link
Member

@pablogsal

miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jun 7, 2025
…OSYS (pythonGH-134878)

(cherry picked from commit ac9c343)

Co-authored-by: Daniel Golding <goldingd89@gmail.com>
pablogsal pushed a commit that referenced this issue Jun 7, 2025
…NOSYS (GH-134878) (#135240)

gh-134876: Add fallback for when process_vm_readv fails with ENOSYS (GH-134878)
(cherry picked from commit ac9c343)

Co-authored-by: Daniel Golding <goldingd89@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.14 bugs and security fixes 3.15 new features, bugs and security fixes interpreter-core (Objects, Python, Grammar, and Parser dirs) type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

4 participants