Using a TCP connection to GET data from a HTTP server, on rare occasion there's garbage data

Question

I'm writing a networked game in C++ using Winsocks 2.2, using Visual Studio 2010, and decided it would be a good idea to use my web-server to store a list of active servers for the game. When a server starts up, it will register itself with my web server, on exit un-register; and the server itself will attempt to clean the list when someone accesses the server list depending (this behavior I'm still working on designing to not involve too much work on the server; but I figured when the game server tries to add itself, my php file will use fsockopen to detect if it can actually access the server from an outside network, if not, the server won't get added until it can properly setup port forwarding or somehow resolve the issue).

Okay, so after some research, I figured out how to get something from the server using a TCP connection from formatting a specialized message for the HTTP server. Here's what I have:

if(FAIL == Connection::Get_Connection(&m_Connection, networkSettings.ServerListAddress, 80))
{
    return FAIL;
}

m_Connection.SendMsg("GET /servers.php HTTP/1.1\r\nHost: cyclotron.leetnightshade.com\r\nUser-Agent: CycloTron\r\n\r\n");

I'm expecting back properly formatted data, which I'm not exactly getting. Here's what I'm getting:

2f
Server Count:1
129.21.138.1,40000,Depth of Hell
0

Here's another output of some garbage with all the header information:

HTTP/1.1 200 OK
Date: Tue, 12 Apr 2011 23:23:11 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=8254688ee345202bd177d57e4ba339b2; path=/
Set-Cookie: PHPSESSID=73eae89f61e7268f433af9bdfe299173; path=/
Set-Cookie: PHPSESSID=8fb5d6fd9f1023bb00290b4daa3c7952; path=/
Connection: close
Transfer-Encoding: chunked
Content-Type: text; charset=us-ascii

e
Server Count:1
21

129.21.138.1,40000,Depth of Hell
0

This is what my output is supposed to look like, and I do get this on occasion, but not all the time:

HTTP/1.1 200 OK
Date: Tue, 12 Apr 2011 23:32:13 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=a3c88c2d96d45c6f6d3b029e095c429a; path=/
Set-Cookie: PHPSESSID=bf19734ff60813d6d0a5ba944410356a; path=/
Set-Cookie: PHPSESSID=c36a2d9e12c81d4a19a7f41dc5522b4e; path=/
Content-Length: 47
Connection: close
Content-Type: text; charset=us-ascii

Server Count:1
129.21.138.1,40000,Depth of Hell

I don't think this matters too much, but this is my PHP code on the web server:

$num = mysql_num_rows($result);
echo 'Server Count:'.$num;

while ($row = mysql_fetch_assoc($result))
{
    // TODO: check date of entry, if it's really old, remove it.
    echo PHP_EOL.$row['address'].','.$row['port'].','.$row['displayName'];
}

And here's some of the code involving receiving the string (yes it's a little bare bones at the moment, and I realize I could use a cstring function to look for the two new lines so I don't have to do the string copy, I'm just trying to stick to using strings to make things easier):

memset(m_MsgBuffer, 0, sizeof (char) * M_BufferSize);

m_Received = recv(m_Connection.M_Socket, m_MsgBuffer, M_BufferSize, 0);

m_MsgBuffer[m_Received] = '\0';

string str = string(m_MsgBuffer);

size_t index = str.find("\r\n\r\n");
str.erase(0,index);

std::cout << "Received message: " << str << std::endl;

So, do any of you have an idea where this garbage data is coming from?

EDIT: After looking at the correct header information, the one with garbage has "Transfer-Encoding: chunked" and doens't have "content-length." ...what is going on?

Answer 1

The so-called "garbage" is in fact the chunked data from the server. An HTTP/1.1 server is free to send data back in chunked format, if it prefers, and the HTTP/1.1 spec is pretty clear: "All HTTP/1.1 applications MUST be able to receive and decode the "chunked" transfer-coding".

The details of chunked encoding are described in the HTTP/1.1 spec:

http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1

If you are writing an HTTP client, you need to read the HTTP spec...