I want to merge two (multiple) streams using Flink. Both streams are themselves ordered, I want merged result to be ordered also. As an example
[1,2,4,5,7,8, ...] and [2,3,6,7, ..]
should produce following result.
[1,2,2,3,4,5,6,7,7,8,...]
Is there an operator for this use case?
I would probably implement this with a KeyedCoProcessFunction. This use case requires waiting for the next element from both inputs before knowing which to emit, which means you'll have to be prepared to buffer arbitrarily many elements from either input stream while waiting for the next element from the other stream. You can use ListState to store that buffer.
Update:
If the field is a timestamp, then you could union the streams and sort the result of the union by the event timestamps. This could be done rather straightforwardly with Flink SQL.
You would need to define watermarking on all of the sources. Since the sources are already ordered, you can use watermarking with no delay for out-of-orderness.
Docs: UNION ORDER BY