celery chord with a chain callback

989 Views Asked by At

I have a complex workflow involving different celery tasks. I have designed a workflow that involves calling a group in the middle of a chain. That means, it could be thought as a chain, a chord, and another chain as the chord callback. However, the chord callback would only execute the first task of the chain callback and not the next ones. How can this be achieved? It does work fine if the callback is a group, but this does not fit the logic needed here.

The following sample code never receives / executes task_h and task_i (ie. last executed task is the first of the chain callback, instead of the entire chain (tasks g, h and i). Hints?

ctask = chain(
    task_a.si(**kwargs),
    task_b.si(**kwargs),
    task_c.si(**kwargs),
    chord(
        [
            task_d.si(**kwargs),
            task_e.si(**kwargs),
            task_f.si(**kwargs),
        ],
        chain(
            task_g.si(**kwargs),
            task_h.si(**kwargs),
            task_i.si(**kwargs),
        ),
    ),
)

[task.set(queue='high_priority') for task in ctask.tasks]
task = ctask.apply_async()
celery:  {extras = ["redis"], version = "^5.0.5"}
broker:  rabbitmq
backend: redis
0

There are 0 best solutions below