MPI exceed buffer

Question

Suppose I have two processor:

The first one P0

• Call MPI_Send to send message A to p1;
• Call MPI_Recv to receive B from p1;

The second one P1

• Call MPI_Send to send message B to p0;
• Call MPI_Recv to receive A from p0;

What will happen if the sizes of both message A and B exceed the system buffer?

Answer 1

One should never ever assume that such a thing as buffering of the standard send exists. The MPI standard explicitly warns against it in Section 3.5 Semantics of Point-to-Point Communication:

A program is "safe" if no message buffering is required for the program to complete. One can replace all sends in such program with synchronous sends, and the program will still run correctly. This conservative programming style provides the best portability, since program completion does not depend on the amount of buffer space available or on the communication protocol used.

MPI specifically addresses the use case in your question and provides the two send-receive calls MPI_Sendrecv and MPI_Sendrecv_replace. The former uses separate send and receive buffers that must not overlap, while the latter uses a single buffer. Both guarantee that no deadlock will occur if the send and receive parts are matched with a corresponding receive/send operation.

Answer 2

The code is wrong in any case.

It may work by the mercy of the MPI implementation / configuration / state. But generally, this is a deadlock. You shouldn't ponder about the buffering of standard blocking send calls for correctness. They are allowed to buffer exclusively for performance reasons, which can be surprising for beginners. Code that seemed to be working for small messages sizes suddenly deadlocked for larger message sizes, but actually the code was wrong all along, it just din't show.