We're using a python event hub consumer client that reads from a topic containing 32 partitions.
We've enabled checkpointing as per the documentation (that seems to work). Events get consumed via a singular client.
The velocity of these small json payloads is less than 1000 per day.
The problem we run into is that the consumer seemingly seems to skip some events (roughly 10-15% at times). Simple logging of the event body verifies that not all events that go into the queue come out the other end.
When we restart the application, the missed events get picked up.
When setting up a new consumer group and just printing all events held by the retention period also shows no missing events.
We're at our wit's end trying to figure out what's going on.
with eh_client:
eh_client.receive_batch(
on_event_batch=get_msg, #print event
on_error=handle_error, # print error
starting_position="-1",
max_batch_size=1
)