I have a seemingly simple question about the sarama Go language Kafka library: When a Kafka broker is closed due to an idle timeout (controlled by the connections.max.idle.ms config), should the producer automatically re-establish the connection to the broker when/by sending a new message or not?
I was not able to find an answer in any of the resources I checked:
I am asking because it does not seem to reconnect automatically - now should I try to avoid the timeout (either by increasing the connections.max.idle.ms config or by sending heartbeat messages) or should I raise an issue with sarama?
I also had a look at the sarama logs for clues about what actually happens here (testing scenario, sending messages every 10mins) but, being new to both Kafka and Go, I couldn't make much out of it:
...
2021/02/03 07:55:46 producer/broker/0 starting up
2021/02/03 07:55:46 producer/broker/0 state change to [open] on <topic>/0
2021/02/03 07:55:46 Connected to broker at <hostIP>:30001 (registered as #0)
2021/02/03 08:05:46 client/metadata fetching metadata for all topics from broker <host>:30001
2021/02/03 08:06:46 producer/broker/0 state change to [closing] because EOF
2021/02/03 08:06:46 Closed connection to broker <hostIP>:30001
2021/02/03 08:06:46 producer/leader/<topic>/0 state change to [retrying-1]
2021/02/03 08:06:46 producer/leader/<topic>/0 abandoning broker 0
2021/02/03 08:06:46 producer/broker/0 input chan closed
2021/02/03 08:06:46 producer/broker/0 shut down
2021/02/03 08:07:01 Producer shutting down.
2021/02/03 08:15:46 client/metadata fetching metadata for all topics from broker <host>:30001
2021/02/03 08:25:46 client/metadata fetching metadata for all topics from broker <host>:30001
...