My Java program reads data from a stream and creates an in-memory cache of parts of it. At some point it throws an OutOfMemoryError, and I've caused it to create a heap dump at that time so that I can see what causes the issue. But when I load the heap dump I see that about half of the memory is unused: I've started the VM with -Xmx8000m, and the heap dump, when loaded into Eclipse Memory Analyzer or VirtualVM only shows about 4GB in use. The dump file itself however is around 8GB in file size.
What is also odd is that both tools report a lot of int arrays of size int[262136] as "unreferenced objects", i.e. garbage. There is about 4GB of those - so that really points to them not being garbage but being the reason for the OOM.. My code does not create in arrays of this size, at all, btw.
Why do I get this OOM, and what is the matter with those int[] arrays?
I am running on a Java 11 JDK, but the same issue also occurs on Java 14.
This is a problem with Java's garbage collector, and it was very hard to find.
The default garbage collector for these versions is the G1 collector. This garbage collector divides the available memory into fixed size memory regions. These are always a power of 2 big, starting at 1MB, and depend on the -Xmx max memory parameter.
Those int[262136] arrays are a trick that the gc uses to somehow mark these regions as Java objects. This int array takes exactly 1mb of space, so it has the size of the region. It marks them as being unreferenced so most tools do not see them, or mark them as garbage. This is highly misleading as it seems to be the cause of the OOM issue.
The real reason for the OOM is that the caching code allocates (and releases) objects that are considered "Humongous objects" by the G1 garbage collector. It has huge problems with reclaiming or moving these objects, and this apparently causes memory fragmentation - which in turn causes the OOM even though enough memory appears to be available. For some reason the gc logging does not give any indication that this could be an issue 8-(.
A good test to see whether this is the cause of your issue is to run the same program with either the old "mark and sweep GC" (by adding the parameter -XX:+UseConcMarkSweepGC to the Java command line; this works best, but this gc has been removed starting from Java 15), or by trying with the parallel GC (by adding -XX:+UseParallelGC).
To solve this either use one of the above GC's, or play with the -XX:G1HeapRegionSize parameter. Set it to a larger power-of-2 size (like 2m, 4m, 16m) to see if that fixes the issue.
Some more information on this can be found on the site of jxray.com, a heap dump analysis tool: https://jxray.com/documentation#humongous_objs, and in an Oracle article about the G1 collector at https://www.oracle.com/technical-resources/articles/java/g1gc.html.