Is there a way to get the complete function calls chain from a function in the C standard library all the way down to the system call?

Question

I'm on an Ubuntu x86_64 system. I know we have the strace command to trace the system calls in our programs. However I'd like to know if there is a way ( other than inspecting the GNU C library source codes ) to get the complete function calls chain. For example when I do :

printf("Hello World\n");

I'd like to know the complete function call chain from printf all the way down to the write system call ( not the wrapper function )

Answer 1

Is there a way to get the complete function calls chain from a function in the C standard library all the way down to the system call?

Debugger is for that. Example debugging echo program:

$ gdb echo
(gdb) catch syscall 1
Catchpoint 1 (syscall 'write' [1])
(gdb) r
Starting program: /usr/bin/echo 
Downloading separate debug info for /lib64/ld-linux-x86-64.so.2
Downloading separate debug info for system-supplied DSO at 0x7ffff7fc8000                                                               
Downloading separate debug info for /usr/lib/libc.so.6                                                                                  
[Thread debugging using libthread_db enabled]                                                                                           
Using host libthread_db library "/usr/lib/libthread_db.so.1".

Catchpoint 1 (call to syscall write), 0x00007ffff7e9d034 in write () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff7e9d034 in write () from /usr/lib/libc.so.6
#1  0x00007ffff7e1dd4d in _IO_file_write () from /usr/lib/libc.so.6
#2  0x00007ffff7e1c014 in ?? () from /usr/lib/libc.so.6
#3  0x00007ffff7e1ce19 in _IO_do_write () from /usr/lib/libc.so.6
#4  0x00007ffff7e1d353 in _IO_file_overflow () from /usr/lib/libc.so.6
#5  0x0000555555556741 in putchar_unlocked (__c=10) at /usr/include/bits/stdio.h:110
#6  main (argc=<optimized out>, argv=<optimized out>) at src/echo.c:275

Answer 2

You could use bcc's stackcount to do this. It relies on eBPF to aggregate stack traces in the kernel. To trace the write(2) syscall for your process, you can run:

stackcount-bpfcc -p [pid_of_your_process] t:syscalls:sys_enter_write

On Ubuntu, you can install it with apt install bpfcc-tools. See https://github.com/iovisor/bcc/ for more details.