Type information for primitive types in polymorphic types

Question

Given the following object:

object Foo {
  val bar: List[Int] = List(1, 2, 3)
}

When we compile this file to JVM bytecode, because of type erasure and due to the fact that Java does not support primitive types as parameters for generic types, this gets translated into a List<Object>.

We can see this by compiling and inspecting the .class with javap -l:

public static com.yuvalitzchakov.github.Foo$ MODULE$;
  descriptor: Lcom/yuvalitzchakov/github/Foo$;
  flags: ACC_PUBLIC, ACC_STATIC

public scala.collection.immutable.List<java.lang.Object> bar();
  descriptor: ()Lscala/collection/immutable/List;
  flags: ACC_PUBLIC
  Code:
    stack=1, locals=1, args_size=1
        0: aload_0
        1: getfield      #19                 // Field bar:Lscala/collection/immutable/List;
        4: areturn
      LineNumberTable:
        line 4: 0
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       5     0  this   Lcom/yuvalitzchakov/github/Foo$;
    Signature: #17                          // ()Lscala/collection/immutable/List<Ljava/lang/Object;>;

But, if we compile this into a JAR file and later take it as a dependency in a different Scala project and try to set Foo.bar to a different value, the Scala compiler will infer this type as List[Int], not List[Object]:

After browsing around the .class file, I could not find the information on the type parameter which would allow the Scala compiler to successfully infer this as List[Int].

Where is this metadata stored such that we can refer to this type as an actual List[Int] instead of List[Object]?

Answer 1

The JVM class file format allows compilers to put custom attributes into a class file, see Section 4.7.1 of the Java Virtual Machine Specification. Among other things, the Scala compiler puts information about the Scala signature of names into the class files it generates, so that on a later compiler run, it can read this information again. Java Virtual machines are required to ignore attributes they don't understand, so this doesn't make a difference at runtime.

I didn't find a specification of the binary format for the annotations, but if you want to dig into the implementation, I found:

• serialization of types in https://github.com/scala/scala/blob/2.13.x/src/compiler/scala/tools/nsc/symtab/classfile/Pickler.scala
• deserialization of types in https://github.com/scala/scala/blob/2.13.x/src/reflect/scala/reflect/internal/pickling/UnPickler.scala

For Scala 3.0, it is even planned to store the full abstract syntax tree including the information generated by the type checker in class files using the new "tasty" format. Tasty stands for "typed abstract syntax trees". The basic idea is to serialize the abstract syntax tree after the type checking phase and put it into the class files. Later compiler runs can then load the full abstract syntax of dependencies. This can allow not just type checking, but also cross-module inlining and other global optimizations.

Tasty is planned to become a universal interchange format for Scala abstract syntax trees, also for the communication between the compiler and integrated development environments and for meta programming.

If you want to dig into the implementation, maybe the files in https://github.com/lampepfl/dotty/tree/master/compiler/src/dotty/tools/dotc/core/tasty are good start.