Celery workers stop execution unexpectedly

427 Views Asked by At

I am working on a Flask app and using APScheduler to schedule jobs and Celery to run that jobs as background processes. When time comes for a particular task, APScheduler assigns that task to Celery, but sometimes Celery worker does not execute that task until Celery app is restarted.

def run_job(data)
    logger.info('Queuing data send')
    # Call celery task to send post_data to URL
    send_request.apply_async(data)

where send_request() is a Celery task. So, I get logs for "Queuing data send" for multiple jobs but execution never reaches inside send_request method unless Celery app is restarted.

I am running Celery app as

celery_app.start(argv=['celery', 'worker', '-Ofair', '--without-gossip', '--without-mingle', '-l', 'info', '--autoscale', '12,3', '-Q', 'my_queue', '-n', 'my_queue'])
0

There are 0 best solutions below