Spring Kafka 2.9.3 NoClassDefFound - AlterIsrRequest$Builder on EmbeddedKafka Start up

Question

Trying to upgrade the springboot application which uses spring kafka to version 2.9.3 from 2.8.4.

I have followed the instructions here to pulldown kafka-client version 3.2.3 as a separate dependency.

My dependencies looks like this

implementation("org.springframework.kafka:spring-kafka:2.9.3") {
    exclude group: "org.apache.kafka", module: "kafka-streams"
    exclude group: "org.apache.kafka", module: "kafka-clients"
}
implementation("org.apache.kafka:kafka-clients:3.2.3")
testImplementation("org.springframework.kafka:spring-kafka-test:2.9.3")

When I try to kick off a test which makes use of EmbeddedKafka. I am getting this NoClassDefFound error

Caused by: java.lang.NoClassDefFoundError: org/apache/kafka/common/requests/AlterIsrRequest$Builder
    at kafka.server.KafkaServer.startup(KafkaServer.scala:313)
    at kafka.utils.TestUtils$.createServer(TestUtils.scala:177)
    at kafka.utils.TestUtils$.createServer(TestUtils.scala:172)
    at kafka.utils.TestUtils.createServer(TestUtils.scala)
    at org.springframework.kafka.test.EmbeddedKafkaBroker.afterPropertiesSet(EmbeddedKafkaBroker.java:350)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1863)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1800)
    ... 87 more
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.common.requests.AlterIsrRequest$Builder
    at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
    ... 94 more

I think I am missing a dependency?

Answer 1

You have a version mis-match between the kafka-clients and other kafka jars. All kafka jars must be the same version since the server uses classes from the clients jar.

I don't see AlterIsrRequest in jars since 3.1.2 - so it looks like, somehow, you have older kafka server jars on the class path.

spring-kafka-test:2.9.3 should pull in 3.2.3 transitively.

Since you are using Spring Boot, it will pull in 3.1.2 instead.

See the documentation (https://docs.spring.io/spring-kafka/docs/2.9.3/reference/html/#update-deps) for how to override the dependencies when using Spring Boot.