How to benchmark .Net classes started from external code

Question

I intend to use BenchmarkDotNet to test some methods inside various project.

As it may be as easy as adding the Benchmark attribute to the methods, I have considerable doubt about using it in the project I work on.

The project is consisting of 2 parts:

• native core
• .Net features

The native part initializes the .Net part and calls its methods via interop.

So using a test project and adding references to the .Net projects and starting the benchmark application will not work as the native code should be started first.

Any idea, if I can somehow skip the native part and make BenchmarkDotNet test only the methods? Or should I look for another benchmarking approach?

Answer 1

Looking at the specification for BenchmarkDotNet it seems you can mark any method as benchmark, it doesn't have to be the actual methods that your native app calls. Similar to unit tests, you can write benchmarks that call methods in your .NET code with valid parameters that would otherwise come from the native core.

I would suggest creating a separate Benchmark project similar to how you'd have a separate Test project.

Answer 2

• Create a new public class
• Create benchmark method, call the thing that you want to measure. Mark it with [Benchmark] attribute.

If it's enough to run the initialization code once (for many benchmark iterations):

• Create a new method, implement the initialization and mark it with [GlobalSetup] attribute
• BenchmarkDotNet is going to create a new instance of your class, call the setup method once and afterward start the benchmarking of your code.

If it's not enough to run the initialization once and it's required to be called every time before benchmarked method call:

• Create a new method, implement the initialization and mark it with [IterationSetup] attribute. Set the run strategy to RunStrategy.Monitoring.
• BenchmarkDotNet is going to create a new instance of your class, call the setup method once before calling the benchmark and repeat it many times.

You can read more about setups and cleanups in our official docs