chaincode event listener wont connect again to peers after peer nodes killed/restarted

Question

My fabric-node-sdk based application is listening for chaincode events using network.getContract(smartContractName).addContractListener. When I kill all the peer nodes, it prints the following logs, disconnects the event listener, and never tries

2021-08-24T07:22:26.056Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Eventer- name: peer1.org1.com, url:grpcs://peer1.org1.com:7051, connected:false, connectAttempted:true
2021-08-24T07:22:26.057Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer1.org1.com url:grpcs://peer1.org1.com:7051 timeout:3000
2021-08-24T07:22:26.057Z - error: [Eventer]: ServiceEndpoint grpcs://peer1.org1.com:7051 reset connection failed :: Error: Failed to connect before the deadline on Eventer- name: peer1.org1.com, url:grpcs://peer1.org1.com:7051, connected:false, connectAttempted:true
2021-08-24T07:22:29.057Z - error: [ServiceEndpoint]: Error: Failed to connect before the deadline on Eventer- name: peer0.org1.com, url:grpcs://peer0.org1.com:7051, connected:false, connectAttempted:true
2021-08-24T07:22:29.058Z - error: [ServiceEndpoint]: waitForReady - Failed to connect to remote gRPC server peer0.org1.com url:grpcs://peer0.org1.com:7051 timeout:3000
2021-08-24T07:22:29.058Z - error: [Eventer]: ServiceEndpoint grpcs://peer0.org1.com:7051 reset connection failed :: Error: Failed to connect before the deadline on Eventer- name: peer0.org1.com, url:grpcs://peer0.org1.com:7051, connected:false, connectAttempted:true
2021-08-24T07:22:29.059Z - error: [EventService]: send[peer1.org1.com,peer0.org1.com] - #3 - no targets started - Error: Event service peer0.org1.com is not connected
    at EventService.send (/home/ubuntu/rest-client/node_modules/fabric-common/lib/EventService.js:368:20)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async EventServiceManager.startEventService (/home/ubuntu/rest-client/node_modules/fabric-network/lib/impl/event/eventservicemanager.js:50:9)
    at async BlockEventSource.startEventService (/home/ubuntu/rest-client/node_modules/fabric-network/lib/impl/event/blockeventsource.js:100:9)
    at async BlockEventSource.start (/home/ubuntu/rest-client/node_modules/fabric-network/lib/impl/event/blockeventsource.js:64:13)
2021-08-24T07:22:29.059Z - error: [BlockEventSource]: Failed to start event service message=Event service peer0.org1.com is not connected, stack=Error: Event service peer0.org1.com is not connected
    at EventService.send (/home/ubuntu/rest-client/node_modules/fabric-common/lib/EventService.js:368:20)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async EventServiceManager.startEventService (/home/ubuntu/rest-client/node_modules/fabric-network/lib/impl/event/eventservicemanager.js:50:9)
    at async BlockEventSource.startEventService (/home/ubuntu/rest-client/node_modules/fabric-network/lib/impl/event/blockeventsource.js:100:9)
    at async BlockEventSource.start (/home/ubuntu/rest-client/node_modules/fabric-network/lib/impl/event/blockeventsource.js:64:13)

It does not throw any error that I can catch and handle it in a way that it can keep trying to connect to peer. Is there any way to handle it? Besides this, I can also see in the logs that it try to connect to other peer grpcs://peer1.org1.com:7051 timeout:3000 and timeout after 3000 ms. Is there anyway to increase this timeout? The application listens to a lot of events and keep the state in postgres database which is essential for the product.

Answer 1

This is the issue reported in this Jira: https://jira.hyperledger.org/browse/FABN-1657

It should be fixed in current npm packages tagged unstable-2.2, and in a forthcoming v2.2.9 release of the Node SDK.