OutOfMemoryException when comparing .NET list initialized vs not initialized one using BenchmarkDotNet

Question

I want to create a benchmark showing the difference in performance between an initialized and an uninitialized .net list, but I'm having some trouble. I tried different ways of writing this benchmark but without success. Every time I run the code, I get the following:

System.OutOfMemoryException: Array dimensions exceeded supported range

ListInitializationBenchmark.cs

using BenchmarkDotNet.Attributes;

namespace list_benchmark
{
    public class ListInitializationBenchmark
    {
        private List<int> notInitializedList; // Not initialized list
        private List<int> initializedList; // Initialized list with capacity

        [Params(1000, 10000, 100000)]
        public int BatchSize { get; set; }

        [GlobalSetup]
        public void GlobalSetup()
        {
            notInitializedList = new List<int>(); // Not initialized list
            initializedList = new List<int>(BatchSize); // Initialized list with capacity
        }

        [Benchmark]
        public void ProcessNotInitializedList()
        {
            for (int i = 0; i < BatchSize; i++)
            {
                notInitializedList.Add(i);
            }
        }

        [Benchmark]
        public void ProcessInitializedList()
        {
            for (int i = 0; i < BatchSize; i++)
            {
                initializedList.Add(i);
            }
        }
    }
}

Any idea on how can I resolve this problem?

Answer 1

BenchmarkDotNet will run your code thousands of times to get a good indication of how long it takes. However, your code uses the same instance of the list for each execution meaning it will very quickly run out of space. You need to keep the list and its initialisation inside the individual tests or change your setup to be execute per iteration. For example:

[Benchmark]
public void ProcessNotInitializedList()
{
    List<int> notInitializedList = new List<int>();

    for (int i = 0; i < BatchSize; i++)
    {
        notInitializedList.Add(i);
    }
}

[Benchmark]
public void ProcessInitializedList()
{
    List<int> initializedList = new List<int>(BatchSize);

    for (int i = 0; i < BatchSize; i++)
    {
        initializedList.Add(i);
    }
}

Or this:

[IterationSetup] // Iteration, not global setup
public void GlobalSetup()
{
    notInitializedList = new List<int>(); // Not initialized list
    initializedList = new List<int>(BatchSize); // Initialized list with capacity
}