Getting number of bytes available to read in a socket

Question

Here's my scenario. I have a TCP client that is talking to the server. Both the server and the client are running on local machine (Windows).

The dialog goes something like:

1. Client sends data to the server (Request)
2. Client does shutdown for send on the socket
3. Client blocks via a read call for response
4. Server receives the data, processes, and sends back a response (one shot, not broken into chunks)
5. Server does shutdown for send on the socket
6. Client receives the response, and continues processing.

At step 3, I am using recv() call to block and read data from the socket. At this point, I would like to peek to see how many bytes of data is available, so that I can allocate so much memory. By design, it is known that the server has sent all the data, and there is no more data for this request to be sent. (See step 5 above).

I have tried recv() with MSG_PEEK option, but that does not seem to give the total number of bytes available.

Is there a way to retrieve it?

Thanks in advance.

Answer 1

On Windows at least, you can use ioctlsocket() with the FIONREAD command to determine the current number of bytes that are available for recv() to read without blocking. By the time you actually call recv(), more bytes may have arrived, though.

As @LokiAstari said, you should be calling recv() in a loop until it returns 0 bytes to indicate the socket has been closed. You do not need to know how many bytes are available, just pass a fixed-length buffer each time, and recv() will return how many bytes were actually read. Append each non-zero-length buffer read into another buffer that grows as needed until you have received all of the data, and then process that second buffer when ready.

Answer 2

Using TCP there is no way of knowing how much bytes there is. Remember that TCP is a "streaming" protocol, there is a start of the stream but no end until the connection has been closed.

What you can do is read small chunks into a buffer, and enlarge the buffer when needed.

Answer 3

It partly depends on the size of the response because even if the send it as one chunk, the underlying transport may break it into chunks. For example, Ethernet has a maximum packet size of 1500 bytes. Although you send it all once, while it is likely that you will receive it all at once, it is not guaranteed if you are using TCP because it is stream oriented. You are much more likely to receive the response as a single packet if you use UDP.

Answer 4

One way to do this, without pre-allocating or re-allocating buffers is to: (a) send the number of bytes the sender is sending (e.g., as a four byte int, in network byte order), (b) have the receiver receive those four bytes and allocate a receive buffer, and then (c) have the receiver send the actual data of the previously sent length. Note that (c) can occur by the sender without receiving feedback or a reply regarding (a) from the receiver.

I had been trying to use fstat to get the amount of data available on a socket, but it does not appear to be portable. On a Mac system, it works for my application. On a Linux system, it does not.