MSVC Address Sanitizer - Any reason to use it in Release builds?

Question

Microsoft recently brought Address Sanitizer (ASan) to Microsoft Visual Studio 2019, and I've been experimenting with it. This question is specific to C and C++. My question is this: Is there any reason to have ASan enabled for Release builds, as opposed to having it enabled only for Debug builds? Having ASan turned on is devastating to the performance and memory usage of my program. (CPU performance worse than halved, memory usage tripled.) Therefore my hope is that if I enable ASan just to check for potential issues, and it does not detect any problems in a Debug build, then I can safely assume that there would not be any problems in the Release build that would have otherwise been caught by ASan? Certainly we are not meant to leave ASan enabled in Release/Production builds?

Thanks for any insight.

Answer 1

Therefore my hope is that if I enable ASan just to check for potential issues, and it does not detect any problems in a Debug build, then I can safely assume that there would not be any problems in the Release build that would have otherwise been caught by ASan?

That's not likely (especially for larger complex programs). Consider a function like:

int array[250];

int test1(uint16_t a) {
    return array[ (1016 / (a+1)) & 0xFF];
}

You can test it with lots of values of a (e.g. 0, 2, 3, 4, ...) and it will be fine, and it will pass the address sanitizer's test during debug. Then, after release, it might crash due to a different (untested) value of a (e.g. 1) that triggers the "array index out of range" bug.

To guarantee that something like that can't happen you could test every possible combination of parameters for every function (which would be ludicrously expensive); but (in the presence of global variables/state) that might still not be enough, and (in the presence of multiple threads) it won't find hidden race conditions (e.g. code that almost always works because threadA finishes before threadB, that crashes when subtle differences in timing cause threadB to finish before threadA).

Certainly we are not meant to leave ASan enabled in Release/Production builds?

Correct - it's too expensive to leave enabled.

The general idea is to minimize the risk of bugs (because you can't guarantee that all bugs were found and eliminated). Without much testing you might be able to say that you're "80% sure" there's no bugs in the released version; and with some rigorous testing with Asan (and optimizations) enabled you might be able to say that you're "95% sure" there's no bugs in the released version.

What this really comes down to is economics (money). It's a "cost of software bugs (lost sales, support costs, etc)" vs. "cost of finding bugs" compromise; where something like a computer game might be fine with less testing (because users just assume that games crash occasionally anyway) and something like a life support machine (where bugs cause deaths) you might refuse to use C or C++ in the first place.

The other thing worth considering is that users don't care much why your code failed - if it failed because of a bug then the user will be annoyed; and if it failed because Asan was enabled and detected a bug then the user will be equally annoyed. Leaving Asan enabled in release builds would help the user submit better information in a bug report, but most people don't submit bug reports (they just sit there annoyed; wondering if it was a temporary hardware glitch because they're too cheap to buy ECC RAM, or if it was a bug in the OS or a driver; and then restart the software).

Answer 2

Hard facts up front: A program's shape - broadly speaking - falls into one of two categories.

1. Its logical representation expressed as its source code.
2. Its binary representation, translated by a compiler (and linker) into something a machine can execute.

1. is what the programmer intended the program to do, and 2. is what a program translated that intent into.

Ideally, 1. is perfectly within a programming language's specification, and either representation has identical observable behavior. In reality, 1. is wrong, and 2. does whatever.

The exercise is thus: Evaluate how badly 2. is broken. To do that, you absolutely want a 2. that's as close to what your clients will run as ever possible. Fuzzing a 2. compiled as a Release configuration with ASan enabled gets you there. If you don't feel like reading along, fuzz this target as hard as humanly imaginable, and go your merry way.

That covers the unquestionable facts. What follows is strong opinions on what you (we, us) should really be doing. Stop reading, if you don't like opinions, strong opinions, or ethics.

---

<Ramblings, not quite done rambling, yet>