I have a situation where I'm allocating a large chunk of memory, but in many smaller sub-arrays. I noticed the performance is significantly worse once I pass a threshold of around 85,000 bytes per array. I'm assuming the performance drop is because the smaller arrays are being allocated on the "small object heap" (SOH) rather than the "large object heap" (LOH) at that point.
| # of Arrays | Size of Each Array (bytes) | Allocation Time |
|---|---|---|
| 1,154 | 532,480 | 377ms |
| 1,319 | 465,920 | 412ms |
| 1,539 | 339,360 | 439ms |
| 1,847 | 332,800 | 435ms |
| 2,308 | 266,240 | 446ms |
| 3,077 | 199,680 | 491ms |
| 4,616 | 133,120 | 514ms |
| 9,231 | 66,560 | 4420ms |
Note, in all cases, the total memory allocated is around 586MB, but the allocation takes an order-of-magnitude longer in the final case.
My first thought to quickly resolve this performance issue was to somehow tell the C# runtime that I want those arrays in the large object heap even though they're smaller than the threshold. I assume this would bring the allocation time in line with the rest of the cases.
However, I can't seem to find if there's a way to do this. It looks like no one has ever wanted an object to go on the large object heap before. So, I ask: is it possible to flag these arrays in some way to force them onto the large object heap, even though they're smaller than the 85,000 byte threshold?
(A way to lower the 85,000 byte threshold to, say, 65,000 bytes, would also solve my problem, but I couldn't find a way to do that either!)
After reading https://learn.microsoft.com/en-us/dotnet/standard/garbage-collection/large-object-heap, I assume that the small objects, starting in heap generation 0, are subject to more frequent garbage collections. Those that are not collected because they are still referenced may be moved for compacting.
Objects on the large object heap are kinda-treated like generation 2 objects and less frequently collected to begin with. And when they are, the remaining ones are not moved because it's considered too expensive. (After reading in your comment that you already keep a reference to each array, the issue with small objects is probably this moving for compaction.)
Solution: Prevent garbage collection of your small objects by keeping references to them, and pin them in order to prevent moves for compacting when other objects are collected.
(Whether essentially disabling garbage collection is doable or advisable depends on the circumstances. You are essentially on your own with memory management and must, when the occasion arises, unpin (and, if due, un-reference) your objects to make them available for collection or compaction, or you'll run into memory or fragmentation problems.)