Support attach-to-PID on macOS on ARM

[nkdanta1]

I'm trying to attach a Python debugger to a process using the debugpy module, but I'm getting the following error:

Unable to attach to process in arch: aarch64 (did not find attach_aarch64.so in /usr/local/lib/python3.8/site-packages/debugpy/_vendored/pydevd/pydevd_attach_to_process).
E+00000.011: Code injection into PID=7 failed:
             Traceback (most recent call last):
               File "/usr/local/lib/python3.8/site-packages/debugpy/server/cli.py", line 391, in attach_to_pid
                 add_code_to_python_process.run_python_code(
               File "/usr/local/lib/python3.8/site-packages/debugpy/_vendored/pydevd/pydevd_attach_to_process/add_code_to_python_process.py", line 400, in run_python_code_linux
                 raise RuntimeError('Could not find .so for attach to process.')
             RuntimeError: Could not find .so for attach to process.

The command I'm running is:
python3 -m debugpy --listen 0.0.0.0:5678 --pid 7

I'm not sure what could be causing this error or how to fix it. Any help would be greatly appreciated.

Environment

debugpy: v1.6.6
Host OS: Ventura 13.1
Docker: v23.0.1
Python: v3.8.12

Actual Behavior

See description

Expected Behavior

The command injects a debugger into the process.

Steps to Reproduce

1. Start a Python Docker container on an ARM Apple Silicon computer.
2. Run python3 -m debugpy --listen 0.0.0.0:5678 --pid <pid> inside container.

[int19h]

Attach-to-PID is not supported on ARM.

#286 is the item tracking this for Linux, but I think we should have a separate one on macOS, as there may be differences in implementation.

[nkdanta1]

thanks @int19h for the faster response. Is there any workaround for ARM currently?

[int19h]

If you can run arbitrary Python code in the target process, or can edit the code that runs normally, you can use attach-to-TCP via import debugpy etc.

[nkdanta1]

Hi @int19h, is there any news on this enhancement?

[sevetseh28]

I'm having a similar issue on a Macbook Pro with the M2 Pro chip (ARM64).
I'm developing in a devcontainer with Ubuntu 22.04 and I get:

--- Starting attach to pid: 25047 ---
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
0x0000ffff874de844 in __GI___select (nfds=nfds@entry=0, readfds=readfds@entry=0x0, writefds=writefds@entry=0x0, exceptfds=exceptfds@entry=0x0, timeout=timeout@entry=0xffffdadd73d8) at ../sysdeps/unix/sysv/linux/select.c:69
69	../sysdeps/unix/sysv/linux/select.c: No such file or directory.
warning: File "/root/.pyenv/versions/3.9.18/bin/python3.9-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
To enable execution of this file add
	add-auto-load-safe-path /root/.pyenv/versions/3.9.18/bin/python3.9-gdb.py
line to your configuration file "/root/.config/gdb/gdbinit".
To completely disable this security protection add
	set auto-load safe-path /
line to your configuration file "/root/.config/gdb/gdbinit".
For more information about this security protection see the
"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
	info "(gdb)Auto-loading safe path"

**The target architecture is set to "auto" (currently "aarch64").**

.....
<skipped for brevity>
.....

subprocess.CalledProcessError: Command 'gdb --nw --nh --nx --pid 25047 --batch --eval-command='set scheduler-locking off' --eval-command='set architecture auto' --eval-command='call (void*)dlopen("/root/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_linux_amd64.so", 2)' --eval-command='sharedlibrary attach_linux_amd64' --eval-command='call (int)DoAttach(0, "import codecs;import json;import sys;decode = lambda s: codecs.utf_8_decode(bytearray(s))[0] if s is not None else None;script_dir = decode([4....

So in my case, it's trying to use attach_linux_amd64.so which is wrong.
I'd love to see this prioritised now that Apple is about to release the third generation of Silicon chips, attaching to running process is, in my view, a basic debugging use case. I have many programs that launch a subprocess to which I need to attach in order to continue debugging.

The TCP approach will work because it's platform-agnostic, but does require code change. I'm going to stick with that for now.

[int19h]

@sevetseh28, can you clarify the scenario? We generally assume that TCP attach is the default scenario since it's more reliable (Python runtime really isn't designed for this kind of thing, so the way it has to be implemented is incredibly hacky). You shouldn't need to do any code changes so long as you control the launch - debugpy CLI should cover all cases. Attaching by PID is really mostly meant for Python embedded in some host process. But if there's another scenario where debugpy CLI is not an option, it would help prioritize this higher.

[sevetseh28]

@sevetseh28, can you clarify the scenario? We generally assume that TCP attach is the default scenario since it's more reliable (Python runtime really isn't designed for this kind of thing, so the way it has to be implemented is incredibly hacky). You shouldn't need to do any code changes so long as you control the launch - debugpy CLI should cover all cases. Attaching by PID is really mostly meant for Python embedded in some host process. But if there's another scenario where debugpy CLI is not an option, it would help prioritize this higher.

Yes, my usecase involves debugging inside a devcontainer using VSCode.

In order to do that I'm using the official Python extension for VSCode which seems to use debugpy behding the scenes.
debugpy is not actually installed in my program's virtualenv but used by VSCode's Python extension.

My main Python program creates other processes (subprocess.Popen) which also run Python code.
These child processes are the processes that I need to debug by attaching to their PIDs.

If I want to use debugpy over TCP I first need to install it in my virtualenv, and then write this in my code:

import debugpy
# Enable the debugger to listen on port 5678
debugpy.listen(('0.0.0.0', 5678))
# Pause execution until debugger is attached
debugpy.wait_for_client()

Thanks!

[int19h]

So long as you start the main process using python -m debugpy ..., or use the "launch" debug configuration from VSCode, the child processes should be automatically connected to in this scenario, unless you have "subProcess": false in your config. If it doesn't auto-attach to the children in this setup, that would be a bug in its own right.

[sevetseh28]

So long as you start the main process using python -m debugpy ..., or use the "launch" debug configuration from VSCode, the child processes should be automatically connected to in this scenario, unless you have "subProcess": false in your config. If it doesn't auto-attach to the children in this setup, that would be a bug in its own right.

I actually had tried that. I have a launch configuration for the main process with subProcess: true but for some reason it's not auto-attaching to the subprocess. That's why I came all the way down to manually attaching to the PID and ended up here. Where do you think the issue lies in that case?
I can see the command VSCode is running is this one:

python/debugpy/adapter/../../debugpy/launcher 36129 -- /app/checker/debug_wrapper.py --verbose --daemon 
2023-08-29 12:19:51.399 [info] DAP Server launched with command: /app/virtualenv/env3/bin/python /root/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/lib/python/debugpy/adapter
2023-08-29 12:19:51.518 [info] Send text to terminal:  cd /app/checker ; /usr/bin/env /app/virtualenv/env3/bin/python /root/.vscode-server/extensions/ms-python.python-2023.14.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher 58935 -- /app/checker/debug_wrapper.py --verbose --daemon 

Thank you!

[int19h]

@sevetseh28 We detour process creation on API level (basically, hijack stuff like fork, CreateProcess etc), so long as it happens in the process that's running Python, so it should work even if you're using ctypes or calling into native code that's spawning processes. But one other possibility is that you have a process chain where the main Python process spawns another process that is not Python, and then this latter one is spawning more Python subprocesses; we wouldn't be able to debug this middle process in that scenario, and thus wouldn't detect those Python subprocesses.

The command line looks fine, but most of the interesting bits aren't visible there. Can you enable logging ("logToFile": true), and then take a look at debugpy*.log files that it generates in the same directory where the extension is installed (you should see that path in the terminal when doing launch - it's the parent folder of debugpy)? If you can share those, that would help diagnose this. Probably best to do this in a separate issue, though, so that we can track the underlying bug properly.