C# Why doesn't "Flush" force the bytes down the network stream?

Question

I have a project where I'm trying to send a serialized object to the server, then wait for an "OK" or "ERROR" message to come back.

I seem to be having a similar problem to th poster of : TcpClient send/close problem

The issue is that the only way I seem to be able to send the original object is to close the connection, but then (of course) I can't wait to see if the object was processed successfully by the server.

private void button4_Click(object sender, EventArgs e)
{
    RequestPacket req = new RequestPacket();

    /// ... Fill out request packet ...

    /// Connect to the SERVER to send the message...
    TcpClient Client = new TcpClient("localhost", 10287);
    using (NetworkStream ns = Client.GetStream())
    {
        XmlSerializer xml = new XmlSerializer(typeof(RequestPacket));
        xml.Serialize(ns, req);

        /// NOTE: This doesn't seem to do anything.... 
        ///       The server doesn't get the object I just serialized.
        ///       However, if I use ns.Close() it does... 
        ///          but then I can't get the response.
        ns.Flush();

        // Get the response. It should be "OK".
        ResponsePacket resp;

        XmlSerializer xml2 = new XmlSerializer(typeof(ResponsePacket));
        resp = (ResponsePacket)xml2.Deserialize(ns);


        /// ... EVALUATE RESPONSE ...
    }

    Client.Close()
}

UPDATE: In response to one commenter, I don't think the client can be at fault. It is simply waiting for the object, and the object never comes until I close the socket.... however, if I'm wrong, I'll GLADLY eat crow publicly. =) Here's the client:

    static void Main(string[] args)
    {
        // Read the port from the command line, use 10287 for default
        CMD cmd = new CMD(args);
        int port = 10287;

        if (cmd.ContainsKey("p")) port = Convert.ToInt32(cmd["p"]);

        TcpListener l = new TcpListener(port);
        l.Start();

        while (true)
        {
            // Wait for a socket connection.
            TcpClient c = l.AcceptTcpClient();
            
            Thread T = new Thread(ProcessSocket);

            T.Start(c);
        }
    }


    static void ProcessSocket(object c)
    {
        TcpClient C = (TcpClient)c;

        try
        {
            RequestPacket rp;
            //// Handle the request here.
            using (NetworkStream ns = C.GetStream())
            {
                XmlSerializer xml = new XmlSerializer(typeof(RequestPacket));
                rp = (RequestPacket)xml.Deserialize(ns);
            }

            ProcessPacket(rp);
        }
        catch
        {
            // not much to do except ignore it and go on.
        }
    }

Yeah.... it's that simple.

Answer 1

The short version is apparently, when using XmlSerializer (or any other big blob) to shove data down a NetworkStream, it will simply hold the line open indefinitely waiting for more information to be written. It only flushes the connection once you close it. This creates a situation where this method is great for sending, but not receiving. Or vice-versa. It becomes a one-way communication, and useless for continued back-and-forth communication over the same connection.

It's kind of crappy that I had to work around something that seemed so elegant on the surface, but dropping back to my old C days, I've resorted to sending a "number of bytes" packet first, then the actual packet. This enables me to READ at the other end the exact number of bytes so I never get caught in a blocking pattern.

To simplify my life, I created a class that holds some static methods for both sending and receiving. This class can send ANY XML-serializable class across the network, so it does what I need it to do.

If anyone has a more elegant solution, I'd be open to hearing it.

public class PacketTransit
{
    public static void SendPacket(TcpClient C, object Packet)
    {
        MemoryStream ms = new MemoryStream();
        XmlSerializer xml = new XmlSerializer(Packet.GetType());
        xml.Serialize(ms, Packet);
        ms.Position = 0;
        byte[] b = ms.GetBuffer();
        ms.Dispose();

        byte [] sizePacket = BitConverter.GetBytes(b.Length);
        // Send the 4-byte size packet first.
        C.Client.Send(sizePacket, sizePacket.Length, SocketFlags.None);
        C.Client.Send(b, b.Length, SocketFlags.None);
    }

    /// The string is the XML file that needs to be converted.
    public static string ReceivePacket(TcpClient C, Type PacketType)
    {
        byte [] FirstTen = new byte[1024];
        int size = 0;
        byte[] sizePacket = BitConverter.GetBytes(size);

        // Get the size packet
        int sp = C.Client.Receive(sizePacket, sizePacket.Length, SocketFlags.None);
        if (sp <= 0) return "";

        size = BitConverter.ToInt32(sizePacket, 0);

        // read until "size" is met
        StringBuilder sb = new StringBuilder();
        while (size > 0)
        {
            byte[] b = new byte[1024];
            int x = size;
            if (x > 1024) x = 1024;
            int r = C.Client.Receive(b, x, SocketFlags.None);
            size -= r;
            sb.Append(UTF8Encoding.UTF8.GetString(b));
        }

        return sb.ToString();
    }

    /// The XML data that needs to be converted back to the appropriate type.
    public static object Decode(string PacketData, Type PacketType)
    {
        MemoryStream ms = new MemoryStream(UTF8Encoding.UTF8.GetBytes(PacketData));
        XmlSerializer xml = new XmlSerializer(PacketType);
        object obj = xml.Deserialize(ms);
        ms.Dispose();

        return obj;
    }

    public static RequestPacket GetRequestPacket(TcpClient C)
    {
        string str = ReceivePacket(C, typeof(RequestPacket));

        if (str == "") return new RequestPacket();

        RequestPacket req = (RequestPacket) Decode(str, typeof(RequestPacket));

        return req;
    }

    public static ResponsePacket GetResponsePacket(TcpClient C)
    {
        string str = ReceivePacket(C, typeof(ResponsePacket));

        if (str == "") return new ResponsePacket();

        ResponsePacket res = (ResponsePacket)Decode(str, typeof(ResponsePacket));

        return res;
    }
}

To use this class, I simply need to call PacketTransit.SendPacket(myTcpClient, SomePacket) to send any given XML-Serializable object. I can then use PacketTransit.GetResponsePacket or PacketTransit.GetRequestPacket to receive it at the other end.

For me, this is working very well, but it was alot more of a workout than originally expected.

Answer 2

you should use a StreamWriter/Reader linked to your network stream, .Flush does nothing on a NetworkStream, see here:

http://www.c-sharpcorner.com/UploadFile/dottys/SocketProgDTRP11222005023030AM/SocketProgDTRP.aspx

Answer 3

Uh oh, you can blame Nagle's algorithm. It has nothing to do with C# though, it is a default behavior for TCP/IP stack. Enable NoDelay socket option using SetSocketOption method. But be careful, disabling Nagle's algorithm will downgrade the throughput.

I'm also not sure about that stream you are using on top of the socket, as I am not a C# developer at all, but try to drop its instance so it does write for sure :-)

Answer 4

I believe the real problem here may be that the XmlDeserializer may not return until it has read EOS from the stream. You may need to shutdown the sending stream for output to force this to happen.