Azure switching from little endian to big endian when deployed

Question

Okay, I have this weird issue. I've created a workerrole which connects to a chatserver using TcpClient and Reactive Extensions. The code works when I run in the emulator. In the beginning I thought there was some issues with Rx, but what it looks like now, is that the byte order changes between packages I receive.

A little background information. The first 4 bytes I receive is a header, the first 2 bytes is the packettype and the last 2 is the remaining length. To debug the issue I do the following statements:

var p1 = BitConverter.ToInt16(data, 0);
var p2 = IPAddress.NetworkToHostOrder(BitConverter.ToInt16(data, 0));
var i1 = BitConverter.ToInt16(data, 2);
var i2 = IPAddress.NetworkToHostOrder(BitConverter.ToInt16(data, 2));
Trace.TraceInformation("IsLittleEndian: {0}", BitConverter.IsLittleEndian);
Trace.TraceInformation("Header:({0},{1}),({2},{3})", p1, i1, p2, i2);

The first packet looks like this:

IsLittleEndian: True; TraceSource 'WaWorkerHost.exe' event
Header:(0,8704),(0,34); TraceSource 'WaWorkerHost.exe' event

The second pair is the correct one here ((0,34), where 0 = LoginSeed and 34 is the remainder).

The second packet looks like this:

IsLittleEndian: True; TraceSource 'WaWorkerHost.exe' event
Header:(7,152),(1792,-26624); TraceSource 'WaWorkerHost.exe' event

Here the first pair is suddenly the correct one and there's no change indicating this is big-endian suddenly. I can't help but think this is just some sort of bug in Azure, or is it something I've overlooked? Keep in mind this work in the emulator, so I would think it should be related to some infrastructure issues.