Why is heap memory going up together with metaspace in Java 8?

Question

I'm doing a little test to understand how metaspace memory (Java 8 onwards) works. When I create 100,000 classes dynamically the metaspace memory is growing (obviously) but heap memory is going up too. Can someone explain to me why this occurs?

PS: I'm running the test with 128 MB of heap and 128 MB of metaspace.

@Test
public void metaspaceTest() throws CannotCompileException, InterruptedException {

ClassPool cp = ClassPool.getDefault();
System.out.println("started");

for (int i = 0; i <= 100000; i++) {
    Class c = cp.makeClass("br.com.test.GeneratedClass" + i).toClass();
    Thread.sleep(1);
    if (i % 10000 == 0) {
    System.out.println(i);
    }
}

System.out.println("finished");
}

See the images below:

Answer 1

I did a study of your code especially ClassPool#makeClass. There are few points I noticed which are causing heap space to increase as metaspace increases.

1. It cache the classes created by method makeClass inside a hashtable

protected Hashtable classes;

So, for a 10th of million of classes, it has the entry for each of these. Hence, Heap Space increases too and it's not GC as hashtable reference is still used by your for loop and continuously updated hence not eligible for gc.

2. CtNewClass creates the new instance of the class and it has the constructor definition as below:

     CtNewClass(String name, ClassPool cp, boolean isInterface, CtClass superclass) {
       super(name, cp);
       this.wasChanged = true;
       String superName;
       if (!isInterface && superclass != null) {
           superName = superclass.getName();
       } else {
           superName = null;
       }
   
       this.classfile = new ClassFile(isInterface, name, superName);
       if (isInterface && superclass != null) {
           this.classfile.setInterfaces(new String[]{superclass.getName()});
       }
   
       this.setModifiers(Modifier.setPublic(this.getModifiers()));
       this.hasConstructor = isInterface;
   }
   

In code above line this.classfile = new ClassFile(isInterface, name, superName); actually creates new ConstPool instance for each class i.e. new HashMap instances for each instance and these reserve memory on heap space.

HashMap classes; //from ConstPool class

HashMap strings; //from ConstPool class

Also, this creates two new ArrayLists. Observe this.fields = new ArrayList(); and this.methods = new ArrayList(); statement in above constructor. Also, a new linked list this.attributes = new LinkedList();.

Hence, the conclusion is the ClassPool has its cache management which takes the good amount of heap space. Then each class has its own set of collections to manage properties, constants etc.

Hope it helps!

Answer 2

Your class pool uses heap memory. It's got hashtables and lists and other things. It also uses java reflection code which uses heap memory. Heap memory that is not gc'ed is probably all those data structures in class pool, a couple hashtables, linked lists, array lists, pools, etc... For instance, every single class you create is stored in a hashtable by the class pool. That is a 100,000 element hash table.

Secondly, if there are any static initializers in the classes you are creating, that will use heap memory. Static initializers include the static field initializations and the static block code.