x86_64 printf segfault after brk call

Question

While i was trying do use brk (int 0x80 with 45 in %rax) to implement a simple memory manager program in assembly and print the blocks in order, i kept getting segfault. After a while i could only reproduce the error, but have no idea why is this happening:

    .section    .data
    helloworld:
    .ascii  "hello world"
    .section    .text
    .globl      _start
_start:

    push    %rbp
    mov     %rsp, %rbp

    movq    $45, %rax  
    movq    $0, %rbx   #brk(0) should just return the current break of the programm
    int     $0x80

    #incq    %rax #segfault
    #addq    $1, %rax #segfault
    movq    $0, %rax #works fine?
    #addq    $1, %rax #segfault again?

    movq    $helloworld, %rdi
    call    printf

    movq    $1, %rax #exit
    int     $0x80

In the example here, if the commented lines are uncommented, i have a segfault, but some commands (like de movq $0, %rax) work just fine. In my other program, the first couple printf work, but the third crashes... Looking for other questions, i heard that printf sometimes allocates some memory, and that the brk shouldn't be used, because in this case it corrupts the heap or something... I'm very confused, does anyone know something about that?

EDIT: I've just found out that for printf to work you need %rax=0.

Answer 1

Your next problem is that the x86_64 system call calling convention uses %rdi, %rsi, %rdx, %r10, %r8 and %r9 to pass up to six arguments in that order (rather than the %ebx, %ecx, %edx, %esi, %edi, %ebp used in x86_32).

Therefore, you have to put brk's first argument in %rdi:

movq    $12, %rax  
movq    $0, %rdi
syscall

On kernels that support it, using int 0x80 will actually invoke 32-bit system calls with 32-bit call numbers and register assignments (for compatibility). If you had a kernel like that then your code snippet should have worked. If you did not, then your program would die as soon as it executed int 0x80.

Answer 2

EDIT: I've just found out that for printf to work you need %rax=0. ”

This hit the point. Besides the x86_64 compatible problem, %rax must be set to correct value for printf() to work.

According to x86_64 ABI Documentation (http://x86-64.org/documentation/abi.pdf), Section 3.5.7 Variable Argument Lists:

When a function taking variable-arguments is called, %rax must be set to the total number of floating point parameters passed to the function in vector registers.

Since printf is a var_arg function, you need to explicitly told the C library you don't want to pass any float type to it, aka. mov 0, %rax.

Answer 3

Your immediate problem is that you are using the wrong system call numbers: 45 is a SYS_brk on i*86, but on x86_64 45 is SYS_recvfrom. Likewise, SYS_exit is 60 on x86_64. You can find out the right number like so:

echo "#include <syscall.h>" | gcc -xc - -E -dD | egrep '__NR_(brk|exit) '
#define __NR_brk 12
#define __NR_exit 60

echo "#include <syscall.h>" | gcc -xc - -E -dD -m32 | egrep '__NR_(brk|exit) '
#define __NR_exit 1
#define __NR_brk 45

Your second problem is that int $0x80 is not a standard way to invoke a system call on x86_64; you should use syscall instead.

As nneonneo correctly pointed out, your third problem is that arguments to the system call on x86_64 are passed in %rdi, %rsi, etc. and not in %rbx.

With the changes above, I get (with printf commented out):

strace ./a.out
execve("./a.out", ["./a.out"], [/* 68 vars */]) = 0
brk(0)                                  = 0x15b9000
_exit(0)                                = ?

Compare that to your original program (without printf):

execve("./a.out", ["./a.out"], [/* 68 vars */]) = 0
recvfrom(0, NULL, 0, 0, NULL, NULL)     = 29184000
write(6291768, NULL, 0 <unfinished ... exit status 0>

Answer 4

Since no one else pointed this out: No, it is not safe to call sbrk with a nonzero argument, or brk at all, unless the calling function is (part of) the one and only implementation of malloc in the current executable image. More specifically, if you change the size of the brk area from outside the malloc implementation, you are very likely to corrupt malloc's internal data structures, which will cause your program to crash the next time it uses malloc or free. Moreover, any C library function that is not on the short list of async-signal-safe functions (list is near the end of that document) is allowed to call malloc under the hood, and printf in particular definitely does do that on Linux.

P.S. Even if you are coding in assembly language, you really should use the C library's shims to make system calls; this would have insulated you from the difference in system call numbers between x86-32 and x86-64, and would help in other ways as well, like setting errno and ensuring that you use the most efficient trap sequence available for the processor.