How to group messages so that they go to the same partition every time?

53 Views Asked by At

We are using java, Akka framework and azure event hub. We are using akka framework's groupedWeightedWithin to group messages based on the total number of bytes or duration. I don't know how to set the partition key so that a particular device's messages goes to the same partition all the time. Before implementing batching we were setting device id as the partition key to guarantee that. Can someone suggest how to group messages and calculate the partition key so that each message goes to the same partition every time.

1

There are 1 best solutions below

0
On

In the producer config, you can set partitioner.class to a Partitioner implementation to return whatever integer you want for your given record

Otherwise, just set the key of the ProducerRecord to equivalent values, and they'll already go to the same partition (as of Kafka 3.3 you need to set partitioner.ignore.keys=true)