We are using Vert.x web server (https://vertx.io/) which takes in requests, processes them and sends them to our Kafka cluster.
Everything works fine till we send messages to 3 different topics. When we add another topic in the configuration, the memory usage keeps increasing and the cpu usage also increases to almost 100% possibly due to GC.
Our assumption is that adding the 4th topic is causing a memory leak because of which the CPU utilization peaks trying to run GC. But we couldn't find any open issue regarding this.
What could be the possible reasons for this problem? Is this an open bug in vert.x?
We use the Vertx Kafka client to send the messages to our Kafka cluster. The cluster has 3 brokers and 3 zookeepers running. Each topic has 10 partitions each.
The topic name of the newly added topic doesn't matter. We tried with different names and it still caused the same issue.
The lingerMs is set to 8 minutes and acks is set to 0.
The messages are Json string messages without any schema.
The traffic received by the newly added topic should also be lesser than the first 3 topics.