cause of: AddressSanitizer: SEGV on unknown address (null pointer)

Question

I need some advice how to identify the source of the segfault.

compiled with ASAN:

==21093==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f09d744d882 bp 0x000000001000 sp 0x62100001c538 T0)
ASAN:DEADLYSIGNAL
AddressSanitizer: nested bug in the same thread, aborting.

started with gdb:

Program received signal SIGSEGV, Segmentation fault.    
0x00007ffff5eeb882 in __memset_avx2_erms () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff5eeb882 in __memset_avx2_erms () from /usr/lib/libc.so.6
#1  0xbebebebebebebebe in ?? ()
#2  0xbebebebebebebebe in ?? ()
...

1. Edit:

the output above was compiled for 64bit (x86_64), on 32bit following output is generated:

==8361==ERROR: AddressSanitizer failed to allocate 0x200000 (2097152) bytes of SizeClassAllocator32 (error code: 12)
==8361==Process memory map follows:
    0x00200000-0x00300000
    0x00400000-0x00500000
...
    0xf7791000-0xf7792000   /lib32/ld-2.24.so
    0xf7800000-0xffd00000
    0xffe34000-0xffe55000   [stack]
==8361==End of process memory map.
==8361==AddressSanitizer CHECK failed: ../../../../../src/libsanitizer/sanitizer_common/sanitizer_common.cc:180 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
ERROR: Failed to mmap

2. EDIT:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff5eeb882 in __memset_avx2_erms () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff5eeb882 in __memset_avx2_erms () from /usr/lib/libc.so.6
#1  0xbebebebebebebebe in ?? ()
#2  0xbebebebebebebebe in ?? ()
#3  0xbebebebebebebebe in ?? ()
#4  0xbebebebebebebebe in ?? ()
...
(gdb) record instruction-history
17798      0x00007ffff5eeb8b6 <__memset_avx2_unaligned_erms+22>:    cmp    $0x40,%rdx
17799      0x00007ffff5eeb8ba <__memset_avx2_unaligned_erms+26>:    ja     0x7ffff5eeb8ca <__memset_avx2_unaligned_erms+42>
17800      0x00007ffff5eeb8ca <__memset_avx2_unaligned_erms+42>:    cmp    $0x800,%rdx
17801      0x00007ffff5eeb8d1 <__memset_avx2_unaligned_erms+49>:    ja     0x7ffff5eeb870 <__memset_avx2_erms>
17802      0x00007ffff5eeb870 <__memset_avx2_erms+0>:   vzeroupper 
17803      0x00007ffff5eeb873 <__memset_avx2_erms+3>:   mov    %rdx,%rcx
17804      0x00007ffff5eeb876 <__memset_avx2_erms+6>:   movzbl %sil,%eax
17805      0x00007ffff5eeb87a <__memset_avx2_erms+10>:  mov    %rdi,%rdx
17806      0x00007ffff5eeb87d <__memset_avx2_erms+13>:  rep stos %al,%es:(%rdi)
17807      0x00007ffff5eeb87f <__memset_avx2_erms+15>:  mov    %rdx,%rax

not sure what that means/why this causes a segfault?

Answer 1

I need some advice how to identify the source of the segfault.

The GDB stack trace is typical of stack overflow similar to:

int main()
{
  char buf[1];
  memset(buf, 0xbe, 1<<20);
}

It is surprising that AddressSanitizer didn't catch that overflow.

I would try to debug it with the GDB branch trace support, as described here.

P.S. If you can construct a minimal example, Address Sanitizer developers will be interested in it.

Answer 2

Is it being built and run on different machines/environments?

I observe such segfaults for the executable compiled with asan when it is built and run on different environments/machines (don't observe if the lib versions are same). i.e. without asan the app runs fine on different machine.

In my case, when I run an app with address sanitizer on different machine:

./dummy_logger
ASAN:SIGSEGV
=================================================================
==18213==ERROR: AddressSanitizer: SEGV on unknown address 0x00000000 (pc 0xf7f45e60 bp 0x1ffff000 sp 0xffab0a4c T16777215)
    #0 0xf7f45e5f in _dl_get_tls_static_info (/lib/ld-linux.so.2+0x11e5f)
    #1 0xf7a59d1c  (/usr/lib/i386-linux-gnu/libasan.so.2+0xacd1c)
    #2 0xf7a4ddbd  (/usr/lib/i386-linux-gnu/libasan.so.2+0xa0dbd)
    #3 0xf7f438ea  (/lib/ld-linux.so.2+0xf8ea)
    #4 0xf7f34cb9  (/lib/ld-linux.so.2+0xcb9)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV ??:0 _dl_get_tls_static_info
==18213==ABORTING

And works fine on the machine where it was compiled.