Kafka Consumers subscribed to different topics in same consumer group

1k Views Asked by At

I am starting with kafka , have a question on the consumer groups. We have an application where we want different consumers from same group subscribing to different topics. The grouping is done based on some business criteria . To be specific consumer 1 from group A and consumer 2 from group A are subscribed to Topic 1 and Topic 2 each with 10 partitions. Does this mean that there consumer 1 can scale to 10 and consumer 2 also can scale to 10 since they are subscribed to different topics. Is this a correct design

1

There are 1 best solutions below

0
On

yes, since within a topic kafka try to assign partitions to consumers as equal as possible. The key is topic:consumer_group_id so it doesn't matter another_topic:same_consumer_group_id - it's another key, and consumer with topic:consumer_group_id could be scaled to partitions number