Airflow dynamic serial tasks

36 Views Asked by At

What is the best way to execute dynamic airflow tasks (from params/REST API) with inter-dependency and piping the output of one onto the next task?

e.g.:

params=dict(
  tasks=dict(
    A=argsA,
    B=argsB,
    C=argsC,
    ...
))

should dynamically produce something like: A(**argsA) >> B(**argsB) >> C(**argsC) >> ...

Or what is the closest I can get from that?

Thank you

0

There are 0 best solutions below