I'm using Prometheus to collect metrics on my application which is written in Java 8 (without Spring). I use Kafka Stream to read and Kafka Producer to publish.
I want to expose metrics on Kafka client used by the application such as counting the messages sent to each topic, number of records read, latencies, number of publishes, etc...
I tried using Prometheus built in Collectors such as Counter and Histogram. They work fine but the problem is that I have to record each method that sends or read data from Kafka which is either a lot of code duplication or a huge refactor to the application.
I wonder if anyone else encountered this problem and has a better solution.
The builtin Kafka metrics include record bytes sent, and consumer offset lag. If you want to count number of records per client instance, or latency of some processing, that's not causing any duplicated code, since it's not otherwise included by Kafka.