Marshalling IntPtrs within structs in C#

Question

I intend to model a C# class for parsing custom protocol data I receive as a raw buffer. The buffer is already at hand as a byte array.

This parser has to cope with varying field layouts, hence some of the protocol elements are IntPtrs.

When I try to access a pointer within a struct after mapping the raw buffer into it, I run into an access violation.

I made up a simple example which shows the crucial part:

namespace CS_StructMarshalling
{
  class Packet
  {  
    public PacketStruct Fields;

    [StructLayout(LayoutKind.Explicit)]
    public struct PacketStruct
    {
      [FieldOffset(0)] public UInt16 Flags;
      [FieldOffset(2)] public IntPtr Data;
    }

    public Packet()
    {
      Fields = new PacketStruct();
    }

    public void DecompileBinaryBuffer(ref Byte[] byteBuffer)
    {
      int size = Marshal.SizeOf(typeof(PacketStruct)); 
      IntPtr ptr = Marshal.AllocHGlobal(size);

      Marshal.Copy(byteBuffer, 0, ptr, size);

      this.Fields = (PacketStruct)Marshal.PtrToStructure(ptr, typeof(PacketStruct));
      Marshal.FreeHGlobal(ptr);
    }
  }

  class Program
  {
    static void Main(string[] args)
    {
      byte[] rawBuffer = new byte[] { 1, 2, 65, 66, 67, 68, 69, 70 };

      Packet testPkt = new Packet();
      testPkt.DecompileBinaryBuffer(ref rawBuffer);

      UInt16 testFlags = testPkt.Fields.Flags;
      String testData = Marshal.PtrToStringAnsi(testPkt.Fields.Data, 6);
    }
  }
}

I just try to wrap my head around this issue, but to no avail. To my understanding, the IntPtr must be marshalled seperately, but I don't find any hints how this could be done in a clean way.

Any pointers welcome. Thank you for your time.

Answer 1

There's a fundamental flaw in the approach, based on a possible misunderstanding what a pointer really means. A pointer, IntPtr in managed code, is simply the address of a location in memory. You read what is pointed-to with the kind of code you already use, like Marshal.PtrToStructure().

A problem with pointers is that they are only valid in one process. Every process has its own virtual memory address space. With an additional restriction in managed code, the garbage collector can randomly move an object from one location to another. There are workarounds for that, you could pinvoke ReadProcessMemory() to read data from another process. And you work around the garbage collector behavior by pinning an object, GCHandle.Alloc()

But these are workarounds that don't belong in a custom serialization scheme. An obvious failure mode for ReadProcessMemory() is just not knowing which process has the data. Or not having sufficient rights to use it. Or the process running on another machine. Pinning pointers is flawed because there is no good guarantee that you'll ever un-pin them.

So any serialization approach solves this problem by flattening the data, eliminating pointers by replacing them with the pointed-to data. And resurrect the object graph in the deserializer. The job done by classes like BinaryFormatter, XmlSerializer, DataContractSerializer and DataContractJsonSerializer. And 3rd party ones like Mark's favorite. Don't write your own, there are so many of them because it is hard to get right.

Answer 2

You have unpacked it fine; the problem is the data; you have hard-coded data of:

• Flags = 513
• Data = 1145258561 (the next 4 bytes if using x86)

the problem is simply: the pointer value "1145258561" is not valid, except perhaps in the long-since-dead AppDomain that the number came from.

You can't force your way into data that isn't yours.

Also, on x64 it will error much sooner, because the buffer is 8 bytes and the struct needs 10.

Additionally: your use of ref is incorrect, and you could probably avoid a HGlobal by using stackalloc.

Answer 3

You are decoding the character data as if it was a pointer to character data, thus it will point to an arbitrary position in memory that doesn't contain the character data, and most likely lies outside the part of the address space that your program is allowed to use. You simply can't decode the data as a pointer, as it's not a pointer.

Convert the data as the type they are. As the characters are 8-bit codes, you would need to decode it using an 8-bit character set:

ushort testFlags = BitConverter.ToUInt16(rawBuffer, 0);
string testData = Encoding.ASCII.GetString(rawBuffer, 2, rawBuffer.Length - 2);

Result:

513
ABCDEF