Strange UDP packet loss

Question

I try to read UDP data-gram sequentially. But got issue during second read operation. You can see in below snippest where i try to read first time and lost 12 byte of packet from UDP receive buffer.

Snippest Code:
sock = socket(AF_INET, SOCK_DGRAM | SOCK_CLOEXEC, IPPROTO_UDP);

bytesInBuffer = 0;
if (-1 == ioctl(sock, FIONREAD, &bytesInBuffer))
{
  printf("%s:%d, Fail to read bytes in buffer\n", __func__, __LINE__);
  //If failure on ioctl then continue with select and read functionality
  bytesInBuffer = 0;
}
printf("%s:%d, bytesInBuffer: %d\n", __func__, __LINE__, bytesInBuffer);

errno = 0;
/* Now recv as it will not block */
i32RetVal = recvfrom(sock, buffer, 40, MSG_NOSIGNAL | MSG_DONTWAIT, NULL, NULL);
printf("%s:%d, i32RetVal: %d\n", __func__, __LINE__, i32RetVal);

if (0 > i32RetVal)
{
    printf("%s:%d, Recv failed with status: %d, Err: %d, SErr: %s\n", __func__, __LINE__, status, errno, strerror(errno));
    status = -1;
    break;
}
else if (0 == i32RetVal)
{
    /* other side closed its send pipe */
    status = -1;
    printf("%s:%d, Recv failed as other side closed pipe\n", __func__, __LINE__);
    break;
}

bytesInBuffer = 0;
if (-1 == ioctl(sock, FIONREAD, &bytesInBuffer))
{
    printf("%s:%d, Fail to read bytes in buffer\n", __func__, __LINE__);
    //If failure on ioctl then continue with select and read functionality
    bytesInBuffer = 0;
}
printf("%s:%d, bytesInBuffer: %d\n", __func__, __LINE__, bytesInBuffer);

Output:
Recv_from:304, bytesInBuffer: 52
Recv_from:309, i32RetVal: 40
Recv_from:332, bytesInBuffer: 0

Question: Why 12 bytes from read buffer got discarded?

Any help will be appreciated. :)

Answer 1

If you read the UDP(7) manual page you will see this entry for FIONBIO:

... Returns the size of the next pending datagram in the integer in bytes, or 0 when no datagram is pending. ...

(Emphasis mine.)

So the FIONBIO doesn't give you the number of bytes in the buffer, but the size of the next packet. If no new packet has been received you get zero.

Answer 2

The buffer holds the datagram, not just the payload data.

Answer 3

What your code does is this:

• Query the size of the next datagram in the receive buffer, which is 52 bytes
• Read 40 bytes from the next datagram in the receive buffer and discard the rest
• Check whether recvfrom returned 0, which will never be the case since the socket is not set to non-blocking -- my bad, overlooked the MSG_DONTWAIT flag
• Assume wrongly that a zero-length receive means "other end has closed the connection" -- that's for TCP. UDP has no connections or "pipes", there is nothing to close: A zero length means a "no payload" datagram which consists only of the header (that's perfectly legal!)
• Query the size of the next datagram in the receive buffer (which is zero, since there is none)

Notably, UDP works on a strict per-datagram basis. You cannot read half a datagram (well, you can... but... read on).

No matter what happens, you will always receive a complete datagram or nothing. Datagrams may be lost (and will be lost), but not bytes. You will also always read a complete datagram ("read" as in "consume").

Reading a datagram removes it from the send buffer. Reading half a datagram (or any part of it) removes it from the send buffer, discarding the rest.