I have just started reading about Flink and wanted to know more about how Flink handles backpressure and how it handles failures when there is backpressure. I have read this blog post by data-artisans https://data-artisans.com/blog/how-flink-handles-backpressure and kind of understand the notion of buffer pools and how they deal with backpressure.
My Questions are
(1) how does watermark help in case of remote exchange (when the tasks are on different nodes and buffers are copied to the wire TCP) -- read this in the blog above.
(2) if there is a failure when the system is facing backpressure how is it dealt with?
(3) Is it just the async checkpoints that are used to recover? What does this checkpoint include?
(4) How much of the buffer pool's data state is captured in a checkpoint?
The best resource for understanding how Flink's network stack currently behaves is this talk by Nico Kruber at Flink Forward Berlin 2018: Improving throughput and latency with Flink’s network stack.
As for checkpointing, Data Streaming Fault Tolerance gives a pretty thorough explanation of how it works, and what data is captured in each snapshot.