Hi I have 2 consumers on a kafka topic. Both of them belong to different consumer groups but consuming from the same topic. I am using Go kafka library to consume the messages through both the consumers.
This problem only occurs when I see a number of events on this topic. for ex when 1000 events are seen on this topic for some reason one consumer is receiving around 600 and the other around 200. This number varies from time to time. In normal circumstances when there is 1 event, It is always consumed by both of them.
And I also noticed that the missing message on one consumer is seen on the other and vice versa. Though there might be messages missing on both of them as well. This eliminates the producer not producing the messages.
Is there anyway in which consumer groups get mixed up. I'm not sure what exactly is going on.
Does anyone know what's going on and how I can debug this further?
Thanks in advance.