class defined in trait cannot be serialized

Question

I tried to cache some objects in binaries these days. But when I try to serialize my case classes defined in a trait, exception happens as bellow:

Exception in thread "main" java.io.NotSerializableException: tests.DataStore$ at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1185) at java.base/java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1553) at java.base/java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1510) at java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1433) at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179) at java.base/java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:349) at services.cache.core.redis.ScalaCacheTest$.serialize(ScalaCacheTest.scala:19) at services.cache.core.redis.ScalaCacheTest$.main(ScalaCacheTest.scala:26) at services.cache.core.redis.ScalaCacheTest.main(ScalaCacheTest.scala)

The same class works fine if I move it out of the trait. It's similar to Converting object to byte array throws NotSerializableException. Unfortunately, the case classes under trait are auto-generated codes(by slick codegen), I cannot change that. Anyone can help to make this case work? Here is my code sample:

package test

import java.io.{ByteArrayOutputStream, ObjectOutputStream}


trait Tables {
  case class User(id: Int, name: String)
}

object DataStore extends Tables {
  // it works if we move the User class here.
}

object ScalaCacheTest {

  def serialize(value: Serializable): Array[Byte] = {
    val buf = new ByteArrayOutputStream()
    val os = new ObjectOutputStream(buf)
    os.writeObject(value)
    os.close()
    buf.toByteArray
  }

  def main(args: Array[String]) = {
     val user = DataStore.User(9527, "star")
    serialize(user)
    println("tests pass")
  }
}

Answer 1

You get that exception because the class that is being serialized is a member of a trait, which is not marked as Serializable.

Objects in Scala are treated in a similar way: they behave like an anonymous instance of some unknown class, but the difference is that definitions inside an object are static members (they are defined using the static keyword in Java), whereas the same definitions would be instance members in a class/trait.

The consequence of this behavior is that an instance of a static (nested) class can be created without creating an instance of its outer class:

trait Tables extends Serializable {
  case class User(id: Int, name: String)
}

  val o1 = new Tables() {}
  val o2 = new Tables() {}

  val user: o1.User = o1.User(9527, "star")
  val user2: o2.User = o2.User(9527, "star")

Whereas:

object DataStore {
  case class User(id: Int, name: String)
}

val user3: DataStore.User = DataStore.User(9527, "star")
val user4: DataStore.User = DataStore.User(9527, "star")

If you would decompile these 2 code samples, you'll see something like this:

public class User implements Product, Serializable
{
  // other code
  public User(final Tables $outer, final int id, final String name) {
        this.id = id;
        this.name = name;
        if ($outer == null) {
            throw null;
        }
        this.$outer = $outer;
        Product.$init$((Product)this);
    }
}

And in the second example:

public static class User implements Product, Serializable
{
  // other code
  public User(final int id, final String name) {
        this.id = id;
        this.name = name;
        Product.$init$((Product)this);
    }
}

Notice the difference between them:

Inner classes associated with outer instances do not have zero-argument constructors (constructors of such inner classes implicitly accept the enclosing instance as a prepended parameter)

Because in Java an inner class can access the members of it's outer enclosing class (including private members), this internal manipulation of the code is enforced by the Scala compiler.

Even if Scala does not allow inner classes to access private fields of their outer classes, at compile-time it has to insert enclosing instances in the inner classes' constructors, to be compatible with the JVM's bytecode.

In Scala, this is somewhat hinted, as in the first example, class User is called a path-dependent type because it depends on the type of the outer class/trait, which is trait Tables.

To create a User, you have to create an instance of the enclosing trait first. Thus, to serialize a User instance, the enclosing instance must also be serialized, so your trait, or the (anonymous) class that extends it, has to extend Serializable too. Example:

import java.io.{ByteArrayOutputStream, ObjectOutputStream}

trait Tables extends Serializable {
  case class User(id: Int, name: String)
}

trait Tables2 {
  case class User(id: Int, name: String)
}

object DataStore {
  // it works if we move the User class here.
}

object ScalaCacheTest extends App {

  def serialize(value: Serializable): Array[Byte] = {
    val buf = new ByteArrayOutputStream()
    val os = new ObjectOutputStream(buf)
    os.writeObject(value)
    os.close()
    buf.toByteArray
  }

  val o1: Tables = new Tables() {}
  val o2: Tables2 with Serializable = new {} with Tables2 with Serializable

  val user: o1.User = o1.User(9527, "star")
  val user2: o2.User = o2.User(9527, "star")
  serialize(user)
  serialize(user2)
  println("tests pass")
}

In the end, be aware that the Java Object Serialization Specification, discourages serialization of inner classes:

Note: Serialization of inner classes (i.e., nested classes that are not static member classes), including local and anonymous classes, is strongly discouraged for several reasons. Because inner classes declared in non-static contexts contain implicit non-transient references to enclosing class instances, serializing such an inner class instance will result in serialization of its associated outer class instance as well.