GDB crashes debugging x86 binary under Rosetta 2 + Apple Virtualization Framework

Question

I am trying to debug a x86 program on Apple Silicon using GDB. I would like to be able to debug using the Apple Virtualization Framework (AVF) for performance reasons.

First, I setup a aarch64 VM using AVF. I did this using UTM, and using update-binfmts to pass through x86 binaries to Rosetta. I followed the steps here to register that passthrough.

Here you can see I have a aarch64 host powered by avf.

retep@avf:~$ uname -a                                                                    
Linux avf 5.15.0-83-generic #92-Ubuntu SMP Mon Aug 14 09:34:05 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

I am running Sonoma 14.0 on M1, which should provide the latest avf release + Rosetta as of this writing. Specifically, this appears to be Rosetta-312 (checked by directly running /media/rosetta/rosetta).

In this environment, I am able to run statically and dynamically linked x86 binaries (after installing the appropriate libraries).

retep@avf:~$ file static                                                                 
static: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=13fb542bf99f84822dc4e496c1339e102ba14d0c, for GNU/Linux 3.2.0, not stripped

retep@avf:~$ file dynamic                                                                
dynamic: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=ba40fbd2ec5ff1cf3a27e42149cb14889adc37af, for GNU/Linux 3.2.0, not stripped

retep@avf:~$ ./static                                                                   
Hello, world
                                                                       
retep@avf:~$ ./dynamic                                                                  
Please enter the flag: wrong                                                             
That flag is incorrect.

I then followed this blog post which explains that you cannot directly run gdb on the program since you would be debugging the rosetta runtime. Indeed, this fails:

retep@avf:~$ gdb ./static
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04) 12.1

Reading symbols from ./static...                                                         
(No debugging symbols found in ./static)                                                 
(gdb) run                                                                                  
Starting program: /home/retep/static                                                     
Cannot access memory at address 0x4cc258                                                 
Cannot access memory at address 0x4cc250                                                 
Cannot access memory at address 0x4cc258                                                 
Warning:                                                                                 
Cannot insert breakpoint -3.                                                             
Cannot access memory at address 0x451e10

Instead, I use the feature outlined in the post, ROSETTA_DEBUGSERVER_PORT:

retep@avf:~$ ROSETTA_DEBUGSERVER_PORT=1234 ./static
<waits for GDB connection>

and in a seperate terminal,

retep@avf:~$ gdb-multiarch

(gdb) set architecture i386:x86-64
The target architecture is set to "i386:x86-64".
(gdb) file static
Reading symbols from static...
(No debugging symbols found in static)
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
Remote communication error.  Target disconnected.: Connection reset by peer

I receive this error message:

rosetta error: ptrace seize failed with error 1
 Trace/breakpoint trap

Questions

How can I debug this error? Is this something fixable (i.e., I have set this up incorrectly if I want to debug)? Is what I want to do impossible - if so, why?

Answer 1

I've found that it's a Ubuntu (not only) default limitation of ptrace for security:

By default, on Ubuntu, a process can only attach to another process if it has a predefined relationship with it. An example would be a parent process that has started a child process. This is a restriction to ptrace and is valid for non-root users only. It is designed to ensure higher security to malware attacks. The root user can still attach to any process.

This affects unsolicited Dynamic Attachment debugging.

This behavior is defined by the sysctl value in /proc/sys/kernel/yama/ptrace_scope, which is set to 1, by default.

To relax the restriction and allow processes to attach to other processes, this value must be set to 0. This allows processes that have the same user id to attach to each other. Execute the following to remove the restriction:

https://www.microfocus.com/documentation/enterprise-developer/ed80/ED-Eclipse/GUID-C1576E22-D5BC-420C-825D-562A39C7C2DB.html

echo 0 > /proc/sys/kernel/yama/ptrace_scope