Why does windbg> !EEHeap -gc show a much smaller managed heap than VMMAP.exe?

Question

I have a C# application whose memory usage increases overtime. I've taken periodic user mode dumps and after loading sos, run !EEHeap -gc to monitor the managed heap size. In windbg/sos I've seen it start ~14MB and grow up to 160MB, then shrink back to 15MB, but the applications "Private Bytes" never decreases significantly. I have identified the activity that cauases the increase in "Private Bytes", so I can control when the memory growth occurs.

I tried running Vmmap.exe and noticed it reports a managed heap of ~360MB, took a quick dump and using windbg/sos/eeheap -gc I only see 15MB.

Why am I seeing such different values? Is the managed heap really what vmmap.exe reports?

How can I examine this area of the managed heap in windbg?

Answer 1

You can't break into a .NET application with WinDbg and then run VMMap at the same time. This will result in a hanging VMMap. You can also not do it in the opposite direction: start VMMap first, then break into WinDbg and then refresh the values in VMMap.

Therefore the values shown by VMMap are probably never equal, because the numbers are from a different point in time. Different points in time could also mean that the garbage collector has run. If the application is not changing so much, the values should be close.

In my tests, the committed part of the managed heap in VMMap is the sum of !eeheap -gc and !eeheap -loader, which sounds reasonable.

Given the output of !eeheap -gc, we get the start of the GC heap at generation 2 (11aa0000) and a size of only 3.6 MB.

Number of GC Heaps: 1
generation 0 starts at 0x0000000011d110f8
generation 1 starts at 0x0000000011cd1130
generation 2 starts at 0x0000000011aa1000
...
GC Heap Size          0x374a00(3623424)

!address gives the details:

...
+        0`11aa0000        0`11ef2000        0`00452000 MEM_PRIVATE MEM_COMMIT  PAGE_READWRITE                     <unknown>  
         0`11ef2000        0`21aa0000        0`0fbae000 MEM_PRIVATE MEM_RESERVE                                    <unknown>  
         0`21aa0000        0`21ac2000        0`00022000 MEM_PRIVATE MEM_COMMIT  PAGE_READWRITE                     <unknown>  
         0`21ac2000        0`29aa0000        0`07fde000 MEM_PRIVATE MEM_RESERVE                                    <unknown>
+        0`29aa0000        0`6ca20000        0`42f80000             MEM_FREE    PAGE_NOACCESS                      Free 
...

Although not documented, I believe that a new segment starts at 11aa0000, indicated by the + sign. The GC segment ends at 29aa0000, which is also the starting point of the next segment. Cross check: .NET memory should be reported as <unknown> in the last column - ok.

The total GC size (reserved + committed) is

?29aa0000-11aa0000
Evaluate expression: 402653184 = 00000000`18000000

which is 402 MB or 393.216 kB, which in my case is very close to 395.648 kB reported by VMMap.

If you have more GC heaps, the whole process needs more effort. Therefore I typically take the shortcut, which is ok if you know that you don't have anything else than .NET that calls VirtualAlloc(). Type !address -summary and then look at the first <unknown> entry:

--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Free                                    144      7ff`d8a09000 (   7.999 Tb)           99.99%
<unknown>                               180        0`1a718000 ( 423.094 Mb)  67.17%    0.01%
...

Answer 2

Thank you very much for the detailed answer. Much appreciated.

I'm clear on windbg vs VMmap access/control of the program. Since I can cause the leak by an external action, I'm pretty sure that since I quiesce the activity, memory won't grow much between samples.

I had been relying on the last line of output from !eeheap -gc:

GC Heap Size: Size: 0xed7458 (15561816) bytes.

I think this number must be the amount of managed heap in use (with un-free'ed objects in it). I summed all the "size" bytes reported by "!eeheap -gc" for each SOH and LOH and it matches the above value.

I ran VMmap, took a snap shot and quit VMmap. Then I attached to the process with windbg. Your technique of using !address was most enlightening. I'm using a 12 processor server system, so there are SOH's and LOH's for each processor, i.e 12 to sum. Taking your lead, the output from "!eeheap -gc" has the segments for all of the heaps. I feed them all into "!address " and summed their sizes (plus the size reported by !eeheap -loader ). The result was 335,108K which is within the variation I'd expect to see within the time elapsed (within 600K). The VMmap Managed Heap seems to be the total amount of all of the memory segments committed for use by the managed heap (I didn't check the Reserved numbers). So now I see why the total reported by "!eeheap -gc" is so much less than what VMmap shows. Thanks!