Unclear behavior when using async Kafka writer and RequiredAcks set to RequireNone in segmentio/kafka-go

168 Views Asked by At

When using segmentio/kafka-go

It is unclear what is the difference in the behavior when writing with async mode and setting the RequiredAcks to different values: RequireNone, RequireOne, and RequireAll (0, 1, -1).

Specifically, let's have the following scenario:

  • async Kafka writer, writing data in batches for multiple topics (topic is added to each message)
  • callback function to handle the response
  • required acks set to RequireNone
  • number of replicas set to 3
  • number of retries set to 10

Which responses will I get on my callback function:

  1. If I don't get an error response can I assume the data successfully persisted on the leading broker?
  2. will I get all the errors? or some errors I can get only when setting RequireOne?
  3. will I get replica failures from the other brokers?
  4. when I get an error response is it after it retried 10 times?
  5. is there any advantage to setting the required acks to 1 if I don't care about the order?
0

There are 0 best solutions below