Sarama ClusterAdmin connection issue - broken pipe

Question

I am using sarama(1.27) ClusterAdmin to manage topics in kafka1.1.0. My application that manages kafka topics, is running as a REST service. My application runs fine for a while and I can get/create/delete topic.

But after some time elapses without any activity, a new topic request gets error - write tcp xxxxx:37888->xxxxx:9092: write: broken pipe.

I came across this How to fix broker may not be available after broken pipe.

Since my application is running as a service, how do I prevent broken pipe issue ? I close ClusterAdmin only when application exits. Same ClusterAdmin connection is used to serve all requests. I reinitialize clusterAdmin for each request if for any reason it is nil(Usually it is not nil after first initialization, so same connection is reused).

Should I close clusteradmin after each request is served and open a NewClusterAdmin() for each topic request, or is there a keepalive option that I need to use?

Here is my existing code:

if admin == nil{
        admin, err := NewClusterAdmin([]string{"localhost:9092"}, s.config)
        ..
}
topicMetadata, err := admin.DescribeTopics([]string{topicName})

Answer 1

I also came cross this error. My way to fix this question is try again several times, e.g. 2 to 10 times.