Dataflow - process single input to multiple outputs using a ptransform

96 Views Asked by At

I read data from Pubsub and there are different types of data. I would provide runtime argument based on which it will have to create multiple outputs( using branching) but the idea is i would get multiple Pcollection after Ptransform but should i have separate pipeline for each Pcollection.

[Sample pipeline]

1

Number of pcollection produced from transform will depend based on runtime argument , so how many pcollection can be created and for each pcollection should i duplicate the send output , i put send output as different one because based on the tag each pcollection must be sent to different sources.

0

There are 0 best solutions below