Python socket receives incomplete message from Java PrintWriter socket

Question

I made a python "queue" (similar to a JMS protocol) that will receive questions from two Java clients. The python-server will receive the message from one of the Java clients and the second one will read the question and post an answer. The connection and messaging works, the problem comes when a Java client answers with a String of great length.

The response received by python is incomplete! What is worse, the message is cut at a certain number of characters and always at the same length, but, that number is different if someone else hosts the server. (i.e.: friend1 hosts the server, friend2 sends response, length received: 1380chars. Friend2 hosts the server, friend1 posts the answer, length received: 1431chars) This is the server-side python code:

s = socket.socket()         
host = socket.gethostname() 
# host = "192.168.0.20"
port = 12345  
s.bind((host, port))

s.listen(5)                 
while True:
    c, addr = s.accept()     
    # print 'Got connection from', addr
    message = c.recv(8192) #Is this length a problem?

    # print message
    message = message.strip()
    ipAddress = addr[0]

I read questions here on StackOverflow, that c.recv() should have no problem with a big number of bytes and our response is somewhere close to 1500 characters. This is the java client:

private void openConnection(){
        try {

            socket = new Socket(HOST, PORT);

            out = new PrintWriter(socket.getOutputStream(), true);

            in = new BufferedReader(new InputStreamReader(
                    socketPregunta.getInputStream()));    

            stdIn = new BufferedReader(new InputStreamReader(System.in));


        } catch (Exception e) {}

}

public void sendAnswer(String answer) throws IOException{
        openConnection();

        out.write("PUBLISH-" + answer); //This answer is send incomplete!
        out.flush();

        closeConnection();
}

Thanks in advance!

Answer 1

From the documentation:

recv(buffersize[, flags]) -> data

Receive up to buffersize bytes from the socket. For the optional flags argument, see the Unix manual. When no data is available, block until at least one byte is available or until the remote end is closed. When the remote end is closed and all data is read, return the empty string.

So recv() can return fewer bytes than you ask for, which is what's happening in your case. There is discussion of this in the socket howto.

Basically you need to keep calling recv() until you have received a complete message, or the remote peer has closed the connection (signalled by recv() returning an empty string). How you do that depends on your protocol. The options are:

1. use fixed sized messages
2. have some kind of delimiter or sentinel to detect end of message
3. have the client provide the message length as part of the message
4. have the client close the connection when it has finished sending a message. Obviously it will not be able to receive a response in this case.

Looking at your Java code, option 4 might work for you because it is sending a message and then closing the connection. This code should work:

s = socket.socket()         
host = socket.gethostname() 
# host = "192.168.0.20"
port = 12345  
s.bind((host, port))

s.listen(5)                 
while True:
    c, addr = s.accept()     
    # print 'Got connection from', addr

    message = []
    chars_remaining = 8192
    recv_buf = c.recv(chars_remaining)
    while recv_buf:
        message.append(recv_buf)
        chars_remaining -= len(recv_buf)
        if chars_remaining = 0:
            print("Exhausted buffer")
            break
        recv_buf = c.recv(chars_remaining)

    # print message
    message = ''.join(message).strip()
    ipAddress = addr[0]