I'm working with a high-throughput Kafka-based messaging system using the .NET client, where we aim to guarantee both in-order and exactly-once delivery of messages.
While the Kafka Streams API offers the processing.guarantee option with exactly_once_v2, it's still experimental in the .NET client. Currently, we have enable.idempotence set to true, ensuring idempotency and preserving message order during the service's runtime. However, the challenge arises when we need to restart the service, potentially leading to the loss of some messages.
Is there any strategy or workaround within the Kafka .NET client to ensure both in-order and exactly-once message delivery, even after service restarts? Any insights or alternative approaches to achieve this goal would be greatly appreciated.
Thank you!