Java upgrade with Caucho Hessian v3/v4

Question

Problem:
We are currently using Caucho Hessian 3.2.0 in combination with Java 8 and Java 11 applications. This works fine, but we want to migrate to Java 17 and we get following systen.out message:

java.lang.reflect.InaccessibleObjectException: Unable to make field private java.lang.String java.lang.StackTraceElement.classLoaderName accessible: module java.base does not "opens java.lang" to unnamed module @78e03bb5
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
    at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
    at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
    at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
    at com.caucho.hessian.io.JavaDeserializer.getFieldMap(JavaDeserializer.java:299)
    at com.caucho.hessian.io.JavaDeserializer.<init>(JavaDeserializer.java:77)
    at com.caucho.hessian.io.StackTraceElementDeserializer.<init>(StackTraceElementDeserializer.java:60)
    at com.caucho.hessian.io.SerializerFactory.<clinit>(SerializerFactory.java:627)
    at com.caucho.hessian.io.AbstractHessianOutput.findSerializerFactory(AbstractHessianOutput.java:95)
    at com.caucho.hessian.io.Hessian2Output.writeObject(Hessian2Output.java:486)
    at Main.serializeWithHessian(Main.java:23)
    at Main.main(Main.java:13)

There is a work-around by adding the following JVM param:

--add-opens java.base/java.lang=ALL-UNNAMED

We dont want to use this flag in all our productive applications, so we tried to upgrade Hessian to its newest version 4.0.66. Basicly this works for the most common use cases but doesn't work when we serialize data having same object instances multiple times.

I created a small example application: https://github.com/MatWein/hessian-test

This application reads some java serialized test data from its classpath and tries to serialize and deserialize it with Hessian. If you run this application using Hessian 3.2.0 it works for all Java versions (8, 11 and 17 if you set the JVM flag above) in run&debug modes. But I cannot get it work with Hessian 4.0.66 because following error occures:

Exception in thread "main" com.caucho.hessian.io.HessianFieldException: de.test.TestData.field29: de.test.PaymentType cannot be assigned from null
    at com.caucho.hessian.io.FieldDeserializer2FactoryUnsafe.logDeserializeError(FieldDeserializer2FactoryUnsafe.java:538)
    at com.caucho.hessian.io.FieldDeserializer2FactoryUnsafe$ObjectFieldDeserializer.deserialize(FieldDeserializer2FactoryUnsafe.java:169)
    at com.caucho.hessian.io.UnsafeDeserializer.readObject(UnsafeDeserializer.java:237)
    at com.caucho.hessian.io.UnsafeDeserializer.readObject(UnsafeDeserializer.java:148)
    at com.caucho.hessian.io.Hessian2Input.readObjectInstance(Hessian2Input.java:2202)
    at com.caucho.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2123)
    at com.caucho.hessian.io.CollectionDeserializer.readLengthList(CollectionDeserializer.java:93)
    at com.caucho.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2050)
    at Main.deserializeWithHessian(Main.java:32)
    at Main.main(Main.java:15)
Caused by: java.lang.IndexOutOfBoundsException: Index 13 out of bounds for length 13
    at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
    at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
    at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:266)
    at java.base/java.util.Objects.checkIndex(Objects.java:361)
    at java.base/java.util.ArrayList.get(ArrayList.java:427)
    at com.caucho.hessian.io.Hessian2Input.readObject(Hessian2Input.java:1810)
    at com.caucho.hessian.io.FieldDeserializer2FactoryUnsafe$ObjectFieldDeserializer.deserialize(FieldDeserializer2FactoryUnsafe.java:165)
    ... 8 more

ATTENTION: We have a different runtime behaviour between running and debugging the app (dont know why, but Hessian seems to use Unsafe class, Weak&Soft references and so on).

For me, it looks like an internal bug in Hessian. It is using the class com.caucho.hessian.util.IdentityIntMap on serialization site, but when deserializing the client cannot find the object reference index.

Question: Is there any way to get Hessian 4.0.66 working with Java 8, 11 and 17 (or at least with Java 11 and 17)?

Answer 1

Finally, found the issue after lot of debugging. The issue is with Enum fields having null values.

In Hessian 3.x.x, it is allowing de-serialization of null value by default for Enum type as checked.

But in Hessian 4.x.x, they have changed the approach of evaluating enum values and didn't allow the de-serialization of null value for Enum type as checked.

Note: I have used 4.0.66 Hessian. It will work with all versions of (JDK 8, 11, 17) (already tested)

Fix:

1. First solution: use transient keyword for PaymentType, GenderType & ActivationState as they are coming with null values and should be avoided being part of de-serialization.

Note: Before putting transient for Enum fields, please make sure to select fields that you think are not needed as part of serialization and you are sure that data will be coming as null for those Enum fields when de-serialization will happen.

public class TestData implements Serializable {
    private static final long serialVersionUID = 8316361431446795589L;

    private transient ActivationState field18;
    private transient PaymentType field29;
    private transient GenderType field34;


    ... // getters and setters
}

Output with JDK 11:

/Users/anisb/Library/Java/JavaVirtualMachines/corretto-11.0.19/Contents/Home/bin/java -javaagent:/Applications/IntelliJ IDEA CE.app/Contents/lib/idea_rt.jar=59693:/Applications/IntelliJ IDEA CE.app/Contents/bin -Dfile.encoding=UTF-8 -classpath /Users/anisb/hessian-test/target/classes:/Users/anisb/.m2/repository/com/caucho/hessian/4.0.66/hessian-4.0.66.jar Main
[de.test.TestData@12f41634, de.test.TestData@7a3d45bd, de.test.TestData@87f383f, de.test.TestData@e720b71, de.test.TestData@27f723, de.test.TestData@612fc6eb, de.test.TestData@4d3167f4, de.test.TestData@4923ab24, de.test.TestData@50d0686, de.test.TestData@1a3869f4, de.test.TestData@3aeaafa6, de.test.TestData@13c27452, de.test.TestData@1060b431, de.test.TestData@7b69c6ba, de.test.TestData@46daef40, de.test.TestData@a38d7a3, de.test.TestData@11758f2a, de.test.TestData@5ed828d, de.test.TestData@670b40af, de.test.TestData@44c8afef, de.test.TestData@262b2c86, de.test.TestData@ed9d034, de.test.TestData@612679d6, de.test.TestData@76a3e297, de.test.TestData@4eb7f003, de.test.TestData@371a67ec, de.test.TestData@63440df3, de.test.TestData@6121c9d6, de.test.TestData@77f99a05, de.test.TestData@5f3a4b84, de.test.TestData@eafc191]

Process finished with exit code 0

Output with JDK 8:

/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/bin/java -javaagent:/Applications/IntelliJ IDEA CE.app/Contents/lib/idea_rt.jar=59763:/Applications/IntelliJ IDEA CE.app/Contents/bin -Dfile.encoding=UTF-8 -classpath /Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/charsets.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/ext/cldrdata.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/ext/dnsns.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/ext/jaccess.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/ext/localedata.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/ext/nashorn.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/ext/sunec.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/ext/zipfs.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/jce.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/jfr.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/jsse.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/management-agent.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/resources.jar:/Users/anisb/Library/Java/JavaVirtualMachines/corretto-1.8.0_362/Contents/Home/jre/lib/rt.jar:/Users/anisb/hessian-test/target/classes:/Users/anisb/.m2/repository/com/caucho/hessian/4.0.66/hessian-4.0.66.jar Main
[de.test.TestData@7fbe847c, de.test.TestData@442d9b6e, de.test.TestData@4c98385c, de.test.TestData@1c655221, de.test.TestData@1b9e1916, de.test.TestData@1edf1c96, de.test.TestData@368102c8, de.test.TestData@1963006a, de.test.TestData@4f8e5cde, de.test.TestData@5eb5c224, de.test.TestData@1b701da1, de.test.TestData@58d25a40, de.test.TestData@53e25b76, de.test.TestData@ee7d9f1, de.test.TestData@3f8f9dd6, de.test.TestData@45fe3ee3, de.test.TestData@6996db8, de.test.TestData@759ebb3d, de.test.TestData@73a8dfcc, de.test.TestData@7e774085, de.test.TestData@504bae78, de.test.TestData@ba8a1dc, de.test.TestData@ea30797, de.test.TestData@4cdf35a9, de.test.TestData@17d10166, de.test.TestData@484b61fc, de.test.TestData@726f3b58, de.test.TestData@3b764bce, de.test.TestData@aec6354, de.test.TestData@15615099, de.test.TestData@5fcfe4b2]

Process finished with exit code 0

Output with JDK 17:

/Library/Java/JavaVirtualMachines/zulu-17.jdk/Contents/Home/bin/java -javaagent:/Applications/IntelliJ IDEA CE.app/Contents/lib/idea_rt.jar=59813:/Applications/IntelliJ IDEA CE.app/Contents/bin -Dfile.encoding=UTF-8 -classpath /Users/anisb/hessian-test/target/classes:/Users/anisb/.m2/repository/com/caucho/hessian/4.0.66/hessian-4.0.66.jar Main
[de.test.TestData@12bb4df8, de.test.TestData@11028347, de.test.TestData@39a054a5, de.test.TestData@2f333739, de.test.TestData@1f89ab83, de.test.TestData@14899482, de.test.TestData@3caeaf62, de.test.TestData@7382f612, de.test.TestData@21588809, de.test.TestData@2a33fae0, de.test.TestData@7b3300e5, de.test.TestData@71bc1ae4, de.test.TestData@6ed3ef1, de.test.TestData@77468bd9, de.test.TestData@e73f9ac, de.test.TestData@383534aa, de.test.TestData@311d617d, de.test.TestData@7b1d7fff, de.test.TestData@61064425, de.test.TestData@16f65612, de.test.TestData@6bc168e5, de.test.TestData@3b192d32, de.test.TestData@2437c6dc, de.test.TestData@2e5c649, de.test.TestData@707f7052, de.test.TestData@299a06ac, de.test.TestData@7c53a9eb, de.test.TestData@1055e4af, de.test.TestData@2aae9190, de.test.TestData@ed17bee, de.test.TestData@136432db]

Process finished with exit code 0

2. Second solution: Make sure to send the non-null values in those appropriate Enum fields to avoid facing the below error.

   Exception in thread "main" com.caucho.hessian.io.HessianFieldException: <package>.<Class>.<field-name>: <package>.<Enum> cannot be assigned from null
   

Update:

3. Third Solution: I found a way to fix the issue without a code change. After adding jackson-databind dependency in pom.xml of the app, Hessian surprisingly started to deserialize with null values for Enum type.

Just add this dependency in POM:

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>{{version}}</version>
</dependency>

4. Forth Solution: You can use @HessianUnshared (that you suggested) annotation will fix the issue.