Imagine we have 2 nodes that perform money transfer. Node A initiate a money transfer from one account to another and node B is responsible for completing this transaction. So to complete this transaction node A has to send some TRANSFER request to node B, and on success node B has to respond with confirmation of TRANSFER request. The possible problem that I see is following: upon reception of TRANSFER request node B executed the transaction but fails to send a response. So node A thinks that request was failed and report a problem, but transaction actually was completed.
Even if considering 2-phase commit protocol(where node B upon reception of TRANSFER request doesn't commit transaction, but only execute it and wait for some commit confirmation from node A) there may be similar problem: when node A send TRANSFER commit request it can't be sure that node B received that request and transaction was actually completed(even if this request was delivered to destination host we can't be sure that it was commited by some process).
How to tackle this problem or is it really a problem?
I would say, yes, it's a problem that needs to be addressed. If the transfer request fails you can't be sure if it was already processed by the node B or failure occurs before processing happens (or before the time when the request was delivered to B).
In the case of 2PC is solved by introducing the
prepare phase. The data processing is done first but changes are not visible to the outer world until the transaction commits. If the failure happens before the prepare is processed then the whole transaction is aborted. If B crashes then on the restart there is no notion of the transaction thus it's as it was aborted if there is only a connection failure then B transaction timeouts eventually and it's aborted. If the failure happens after prepare was processed then all the work should be for commit. If communication to B fails it's up to transaction manager to (periodically) try to contact the B and finish the transaction at the end.