Avoiding message timeout in pynsq

Question

For example, I send a message to NSQ by issuing:

curl -d "test2" http://127.0.0.1:4151/pub?topic=hello

I've found that if message handler execution takes longer than 100 second, it will throw and this message will be timed out.

ERROR:nsq.client:[127.0.0.1:4150:hello:channel]      
ERROR: ConnectionClosedError('Stream is closed',)
WARNING:nsq.reader:[127.0.0.1:4150:hello:channel] connection closed

What can I do to avoid this timeout?

Here is my code:

def process_message(message):
    print(message)
    time.sleep(100)
    message.touch()
    return True

r_check = nsq.Reader(
     message_handler=process_message,
    nsqd_tcp_addresses=['127.0.0.1:4150'],
    topic='hello', channel='channel',
    lookupd_poll_interval=15,
    lookupd_connect_timeout=100000,
    lookupd_request_timeout=100000,
    max_tries=10
)

nsq.run()

Thanks.

Answer 1

You should call message.touch() before processing the message in order to tell NSQD that you need more time to process the message.

There are two parameters in NSQD which control message timeouts: max-msg-timeout and msg-timeout.

Matt Reiferson explains how they work here.