What is the best way to write an Apache Kafka producer with a steady but adjustable output.
Example: The producer should send constant 1000 messages/sec to the broker. During runtime the output should be adjustable to either 10 or 10000 messages/sec.
One approach would be to set up a scheduler which runs each second and batch sends the predefined amount of messages.
Addition: Since this producer should be part of a performance testing framework the amount of messages that need to be sent is quite high. How would someone handle very high loads? Would it be beneficial to use Akka for that?
Target language is Scala, but example code in any language is very welcome.
In java this can be implemented by using guava's RateLimiter in your producer code, where in you can define the rate at which producer can produce messages to kafka broker.