Celery best way for callback after 1M tasks

78 Views Asked by At

I'm working with Celery 5.2.7 and Redis 4.4.0 in a single node docker-compose cluster and was wonder what's the best way to trigger a finalizer task after a bunch of 1M simple, but long lasting tasks have completed?
The tasks would have a single integer as input and no output at all. Can this be done with Chord?
Or are there better solutions, since I have read about memory issues that limit the amount of tasks in a chord...
How would I create such a task object, as examples are just dealing with range(100)?
BTW, I work on a 24 core machine with 128GB RAM.

0

There are 0 best solutions below