For a very long time we'd been using Celery and flower to handle tasks and have a process that starts these tasks with a call to a flower endpoint. Like so
http://flower.com/api/task/send-task/task_name
This used to give back both a task-id and a state value (specifically Pending). After upgrading from Celery 4.3.0 --> 5.3.4 and Flower 0.9.2 --> 1.2.0 we no longer get that state value back. This is being done in conjunction with django btw on python 3.8.
Everything else in the process works fine, tasks are executed and run normally and the state is properly reported in the celery dashboard. It's just that this initial call to the API is missing this state value. Based on the code presented in the flower git repo it looked like the send-task would look for a backend initialized for the result, otherwise it won't attach the state. However our backend appears to be initialized without issue, as both the logs confirm the connection as well as the db having up to date records of task executions.
Here is the celery config from the settings.py
CELERY_RESULT_BACKEND = 'django-db'
CELERY_BROKER_URL = 'redis://localhost:6379/0'
CELERY_BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 57600}
CELERYD_PREFETCH_MULTIPLIER = 1
CELERYD_TASK_TIME_LIMIT = 57600
CELERY_ACKS_LATE = True
One potential point of failure is that in the yaml init file there's an exporting of the variable
CELERY_BROKER_URL=rediss://${RedisHost}:6379/0
Which is the redis hosted in AWS, this is the host that shows up as connected in the logs. However it was like this before and we still got the state so I don't know if it actually has anything to do with anything.
I've done a lot of trial and error, I've updated all the variables in the settings file to match those on the celery docs that say they all need to be lowercase, but without any success. I'm kind of running out of things to try for this so any help would be really appreciated.