"Flattening" a databag in Pig

309 Views Asked by At

Suppose I have a bunch of databags generated from a Pig UDF that holds several tuples of Strings. How can I pull all of them out of the databags and simple make each String its own "row" of data.

databags = FOREACH data GENERATE pigUdfThatMakesDataBags(data::someText); strings = FOREACH databags { ??? };

2

There are 2 best solutions below

0
On BEST ANSWER
databags = FOREACH data GENERATE pigUdfThatMakesDataBags(data::someText);
datatuples = FOREACH databags FLATTEN($0);      -- Bag to Tuples 
strings = FOREACH datatuples FLATTEN(TOBAG(*)); -- Tuples to Tokens'
DUMP strings;
0
On

Am I understand it right that you're looking for the FLATTEN?