Automatic test for our program picked up very rare problem with 1 module. Test consists of running under valgrind (valgrind-3.18.1) a sample program using our library. Sample program seems to finish correctly, but later valgrind starts complaining in following fashion:
[FooTest] (... redacted program stdout ...)
[FooTest] ==2==
[FooTest] ==2== HEAP SUMMARY:
[FooTest] ==2== in use at exit: 473,112 bytes in 4,756 blocks
[FooTest] ==2== total heap usage: 27,292 allocs, 22,536 frees, 29,352,230 bytes allocated
[FooTest] ==2==
[FooTest]
[FooTest] Memcheck: mc_main.c:5765 (vgMemCheck_is_valid_aligned_word): Assertion 'VG_IS_WORD_ALIGNED(a)' failed.
[FooTest]
[FooTest] host stacktrace:
[FooTest] ==2== at 0x58042F3A: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest] ==2== by 0x58043067: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest] ==2== by 0x5804320B: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest] ==2== by 0x58010FC1: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest] ==2== by 0x580020C4: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest] ==2== by 0x58002427: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest] ==2== by 0x58002895: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest] ==2== by 0x58002B42: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest] ==2== by 0x5800437F: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest] ==2== by 0x5800F2C2: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest] ==2== by 0x580B2214: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest] ==2== by 0x580E4D53: ??? (in /usr/libexec/valgrind/memcheck-amd64-linux)
[FooTest]
[FooTest] sched status:
[FooTest] running_tid=1
[FooTest]
[FooTest]
[FooTest] Note: see also the FAQ in the source distribution.
[FooTest] It contains workarounds to several common problems.
[FooTest] In particular, if Valgrind aborted or crashed after
[FooTest] identifying problems in your program, there's a good chance
[FooTest] that fixing those problems will prevent Valgrind aborting or
[FooTest] crashing, especially if it happened in m_mallocfree.c.
[FooTest]
[FooTest] If that doesn't help, please report this bug to: www.valgrind.org
[FooTest]
[FooTest] In the bug report, send all the above text, the valgrind
[FooTest] version, and what OS and version you are using. Thanks.
I am completely unable to reproduce it locally, on build server it only occurred rarely. Compiling and running sample program with address or ub sanitizer didn't show any problems. Reviewing the code didn't find any issues.
What does this error mean? Is valgrind even complaing about my program, or is it valgrind crash? In any case do you have any pointers how to find the issue?
EDIT
System on build server where it happens (occasionally) is some containerized Linux x86_64 (possibly Alma Linux 9, but I don't know whether valgrind is from official repo or not).
I checked valgrind code around mc_main.c:5765 but didn't understand what it does. I also verified that latest version of valgrind didn't modify the method that asserts. I saw the FAQ but didn't find anything that helps me.
Sample program feeds some string to rapidjson library and later one of the values in json goes to openssl's EVP_DecodeBlock. The value in question happens to be invalid base64, but from what I can tell both rapidjson and openssl should handle it fine.
EDIT2
Colleague helped me find most likely debug symbols for valgrind. I'm not sure they are perfect match, but at least part around mc_main.c:5765 seems to fit. Reminder - this is valgrind-3.18.1
addr2line -fpCie ./usr/lib/debug/usr/libexec/valgrind/memcheck-amd64-linux.debug 0x58042F3A 0x58043067 0x5804320B 0x58010FC1 0x580020C4 0x58002427 0x58002895 0x58002B42 0x5800437F 0x5800F2C2 0x580B2214 0x580E4D53 | column -t
show_sched_status_wrk at coregrind/m_libcassert.c:406
report_and_quit at coregrind/m_libcassert.c:477
vgPlain_assert_fail at coregrind/m_libcassert.c:543
vgMemCheck_is_valid_aligned_word at memcheck/mc_main.c:5765 (discriminator 1)
heuristic_reachedness.constprop.0 at memcheck/mc_leakcheck.c:685
lc_push_without_clique_if_a_chunk_ptr at memcheck/mc_leakcheck.c:954
lc_scan_memory at memcheck/mc_leakcheck.c:1365
scan_memory_root_set at memcheck/mc_leakcheck.c:2182
vgMemCheck_detect_memory_leaks at memcheck/mc_leakcheck.c:2391
mc_fini at memcheck/mc_main.c:8025
shutdown_actions_NORETURN at coregrind/m_main.c:2242 (discriminator 2)
run_a_thread_NORETURN at coregrind/m_syswrap/syswrap-linux.c:201