Combine unit test with benchmarkDotNet?

Question

Is there a way to combine Unit Testing with BenchmarkDotNet? The idea would be that I would like to be able to write my unit tests using the AAA-pattern. However, when running it as a benchmark-test using dotnetBenchmark, I would like to be able to not include the arrange and assert part of the benchmark-test (not include it in the code being benchmarked, since the arrange and act has nothing to do with the execution of my actual code). By being able to benchmark my "unit tests" this way, I would not need to write separate benchmark tests and unit test, but be able to base them upon each other (no need for duplication of arrange or act).

Answer 1

There is no way for BenchmarkDotNet to execute only selected parts of the compiled code, moreover it's not recommended to treat benchmarks as unit tests.

Here is what I wrote about this matter in Microbenchmark Design Guidelines created for the .NET Team:

When writing Unit Tests, we ideally want to test all methods and properties of the given type. We also test both the happy and unhappy paths. The result of every Unit Test run is a single value: passed or failed.

Benchmarks are different. First of all, the result of a benchmark run is never a single value. It's a whole distribution, described with values like mean, standard deviation, min, max and so on. To get a meaningful distribution, the benchmark has to be executed many, many times. This takes a lot of time. With the current recommended settings used in dotnet/performance repository, it takes on average six seconds to run a single benchmark. The public surface of .NET Standard 2.0 API has tens of thousands of methods. If we had 1 benchmark for every public method, it would take two and a half days to run the benchmarks. Not to speak about the time it would take to analyze the results, filter the false positives, etc..

This is only one of the reasons why writing Benchmarks is different than writing Unit Tests.

The goal of benchmarking is to test the performance of all the methods that are frequently used (hot paths) and should be performant. The focus should be on the most common use cases, not edge cases.