I'm running python-rq tasks that can take anywhere from minutes to hours depending on the input data.
I can set a fixed timeout at the job scheduling with:
low = Queue('low', default_timeout=600) # 10 mins
low.enqueue_call(really_really_slow, timeout=3600) # one hour
Is it possible to prolong a python-rq task from within the running tasks when the timeout almost expires? For instance when the timeout is 3600 seconds and the task is running 3500 seconds?
The python function that is running on the worker forks a subprocess call to a c++ binary and reports its progress to a redis key, so it could in theory also prolong the job in that loop, but I don't know how to do that.