How to access the results from ProcessFunction?

46 Views Asked by At

I am running a query against two topics and calculating the results. In the main class:

tableEnv.createTemporaryView("tbl1", stream1);
tableEnv.createTemporaryView("tbl2", stream2);
String data_query = "select ....";
Table raw_table = tableEnv.sqlQuery(data_query);
DataStream<Row> mystream = tableEnv.toDataStream(raw_table);
mystream.process(new DataProcessor()).setParallelism(4);

In ProcessFucntion class:

public class DataProcessor extends ProcessFunction<Row, List<byte[][]>> {
public void processElement(Row row, ProcessFunction<Row, List<byte[][]>>.Context context, Collector<List<byte[][]>> collector) throws Exception {
        int id = Integer.parseInt("" + row.getField(0));
        String data1 = ((String) (row.getField(1)));
        String data2 = ((String) (row.getField(2)));
byte[][] results = DataUtils.compute(id, data1, data2);
}

My questions:

  1. How to send the results to the output?
  2. How to access the output from the main file?
1

There are 1 best solutions below

0
kkrugler On
  1. Call collector.collect(). Though I'm not sure why your ProcessFunction is generating a list of byte[][] values, since it seems like you always have a single result.
  2. Not sure what you mean by "... access the output from the main file". What main file?