I am trying to compile and load dynamically generated Java code during runtime. Since both ClassLoader::defineClass and Unsafe::defineAnonymousClass have serious drawbacks in this scenario, I tried using hidden classes via Lookup::defineHiddenClass instead. This works fine for all classes that I tried to load, except for those that call lambda expressions or contain anonymous classes.
Calling a lambda expression throws the following exception:
Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
... 1 more
Executing code that instantiates an anonymous class throws the following error:
Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
Reason:
Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
Current Frame:
bci: @5
flags: { }
locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
Bytecode:
0000000: bb00 1159 2ab7 0013 4cb1
at java.base/java.lang.ClassLoader.defineClass0(Native Method)
at java.base/java.lang.System$2.defineClass(System.java:2193)
at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
This is a short example that recreates the problem:
import java.lang.invoke.MethodHandles;
public class HiddenClassLambdaTest {
/** This class is to be loaded and executed as hidden class */
public static final class LambdaRunner implements Runnable {
@Override public void run() {
Runnable runnable = () -> System.out.println("Success");
runnable.run();
}
}
public static void main(String[] args) throws Throwable {
// Path to the class file of the nested class defined above
String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
// Class file content of the LambdaRunner class
byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
lambdaRunnerInstance.run();
}
}
I've already tried compiling and running the code with different JDKs, using different ways to create new instances of the hidden class, searching for bugs at https://bugs.openjdk.java.net/, messing with the bytecode itself and several other things. I am not an expert on Java internals, so I am not sure whether I have not understood the JEP that introduced hidden classes correctly.
Am I doing something wrong, is this just impossible or is this a bug?
Edit: The JEP states
Migration should take the following into account: To invoke private nestmate instance methods from code in a hidden class, use invokevirtual or invokeinterface instead of invokespecial. Generated bytecode that uses invokespecial to invoke a private nestmate instance method will fail verification. invokespecial should only be used to invoke private nestmate constructors.
This might be the problem for the anonymous class. Is there a way to compile the code such that invokespecial is avoided in the bytecode?
You can not turn arbitrary classes into hidden classes.
The documentation of
defineHiddenClass
contains the sentenceWhat it doesn’t spell out explicitly is that this is the only place where a type resolution ever ends up at the hidden class.
But it has been said unambiguously in bug report JDK-8222730:
Which we can check. Even a simple case like
will already fail. Note that it is a special case that the attempt to resolve the original class name
LambdaRunner
within the hidden class will not fail, as you used an existing class as template. So you get anIncompatibleClassChangeError
or aVerifierError
due to mismatches between the hidden class and the existingLambdaRunner
class. When you don’t use a class definition of an existing class, you’d get aNoClassDefFoundError
.The same applies to
As the cited bug report said, neither field nor methods can refer to the hidden class in their signature.
A less intuitive example is
which will fail depending on the compiler and options, as when the
StringConcatFactory
is used, the behavior is like an invocation of a method having all non-constant parts as parameters and returning aString
. So this is another case of having the hidden class in a method signature.Lambda expressions are special, as a class like
gets compiled similar to
which doesn’t have the hidden class in the method signature, but has to refer to the method holding the body of the lambda expression as a
MethodReference
. Within the constant pool, the description of this method refers to its declaring class using thethis_class
entry. So it gets redirected to the hidden class as described in the documentation.But the construction of the
MethodType
as part of theMethodReference
does not use this information to load aClass
like a class literal would do. Instead, it tries to load the hidden class through the defining class loader, which fails with theNoClassDefFoundError
you have posted.This seems to be related to JDK-8130087 which suggests that ordinary method resolution differs from the way,
MethodType
works, which can makeMethodType
fail where just invoking the method would work.But it’s possible to demonstrate that even fixing this issue wouldn’t solve the general problem:
This bypasses the problem described above and calls the
LambdaMetafactory
manually. When being redefined as hidden class, it will print:which shows that all obstacles have been circumvented, but when it comes to the actual invocation from the generated
Runnable
to the method holding the lambda body, it will fail due to the fact that the target class is hidden. A JVM with eager resolution of symbolic references might fail earlier, i.e. the example might not printgot runnable
then.Unlike the old JVM anonymous classes, there is no way to link to a hidden class, not even from another hidden class.
The bottom line is, as said at the beginning, you can not turn arbitrary classes into hidden classes. Lambda expressions are not the only feature not working with hidden classes. It’s not a good idea to try and get surprised. Hidden classes should only be used in conjunction with bytecode generators carefully using only features known to work.