How to get protobuf-net and protostuff to mutually support inherited classes in .Net and Java?

Question

I'm doing communications between a .Net-based program on a Windows system and Android devices. On the .Net end I'm using Marc Gravell's great protobuf-net program, and on the Android end I'm using David Yu's great protostuff program.

My procedure (so far) is to use the .Net classes as the defining classes. I use the protobuf-net Serializer.GetProto() method to generate a .proto file, and protostuff's protostuff-compiler program to generate Java classes that correspond to the .Net classes, more-or-less.

This seems to work fairly well, except that I've run into a problem with inheritance. Yes, I know, inheritance isn't supposed to work with protocol buffers. But both protobuf-net and protostuff have implemented support for inherited classes, each in its own way.

So my question is, does anyone have any suggestion of a simple way to get inherited C# classes to map to inherited Java classes, and vice-versa?

Here's an example of what I'm working with. These are the C# classes:

   public /* abstract */ class ProgramInfoBase
   {
      private string _programName;

      private string _programVersion;

      [ProtoMember(1)]
      public string ProgramName
      {
         get { return _programName; }
         set { _programName = value; }
      }

      [ProtoMember(2)]
      public string ProgramVersion
      {
         get { return _programVersion; }
         set { _programVersion = value; }
      }
   }


   public class ProgramInfoAndroid : ProgramInfoBase
   {
      private string _androidDeviceName;


      [ProtoMember(1)]
      public string AndroidDeviceName
      {
         get { return _androidDeviceName; }
         set { _androidDeviceName = value; }
      }
   }


   public class ProgramInfoWindows : ProgramInfoBase
   {
      private string _windowsMachineName;

      [ProtoMember(1)]
      public string WindowsMachineName
      {
         get { return _windowsMachineName; }
         set { _windowsMachineName = value; }
      }
   }

And here's one of my .proto files:

package Merlinia.MessagingDefinitions;

option java_package = "com.Merlinia.MMessaging_Test.protostuff";

message ProgramInfoAndroid {
   optional string AndroidDeviceName = 1;
}
message ProgramInfoBase {
   optional string ProgramName = 1;
   optional string ProgramVersion = 2;
   // the following represent sub-types; at most 1 should have a value
   optional ProgramInfoAndroid ProgramInfoAndroid = 1001;
   optional ProgramInfoWindows ProgramInfoWindows = 1002;
}
message ProgramInfoWindows {
   optional string WindowsMachineName = 1;
}

Running that through protostuff's protostuff-compiler program produces three separate Java classes, which is to be expected. But what I'd like is to have it generate the corresponding C# class inheritance for the Java classes, and for the serialization and deserialization between protobuf-net and protostuff to support the inherited classes at both ends.

EDIT:

I've now changed my mind. Please see the following question: How to get protobuf-net to flatten and unflatten inherited classes in .Net?

Answer 1

Firstly, note that polymorphism is not defined in the protobuf specification; any implementations are bespoke. It would be nice if they were the same, though.

Basically, it looks like they adopt fundamentally different paradigms; protobuf-net treats sub-types as nested objects starting from the base-type downwards, as per the .proto that you have posted (which I assume came from GetProto, due to the familiar comment). This choice was made for many reasons, including:

• the data can be modelled and parsed by implementations that don't have any notion of inheritance
• no type/name/meta/etc dependencies in the data (which is something protobuf avoids everywhere else, so it is nice to keep this)
• which also helps provide full portability between platforms
• the data survives type refactorings
• if the reader / writer apps are at different levels, the client can still read the data it does known about, even if the writer adds a new sub-class that it doesn't know about
• works with arbitrary fields, merging, etc (as long as the two merges aren't in different inheritance branches)

protostuff, however, does things a different way; glancing at the repo, it writes the type name into field 127 (and restricts the data fields to 126), and uses that name to perform type resolution. I would guess (untested) this means that in .proto terms the schema is therefore:

message ProgramInfoBase {
   optional string ProgramName = 1;
   optional string ProgramVersion = 2;
   required string MagicTypeName = 127;
}
message ProgramInfoWindows {
   optional string ProgramName = 1;
   optional string ProgramVersion = 2;
   optional string WindowsMachineName = 3;
   required string MagicTypeName = 127;
}
message ProgramInfoAndroid {
   optional string ProgramName = 1;
   optional string ProgramVersion = 2;
   optional string AndroidDeviceName = 3;
   required string MagicTypeName = 127;
}

So, at this point you have a few choices:

• remodel the data in both implementations in a way that doesn't use inheritance at all - so you have a flat DTO model; you can of course elect to unflatten it back to a hierarchical domain-model after deserialization
• pick one model (presumably the one you already have data in), and continue to use inheritance in that implementation, and model the implementation as DTOs in the other side

For example, if you keep the java code as polymorphic, the .NET code would need something like the above with the magic type name, but: this would get really messy of there were conflicts - for example, if field 1 was int Foo in one sub-type, and string Bar in another sub-type: bad things; you'd also need to hard-code/recognize the pojo type names. Without trying to blow my own trumpet, these are exactly the type of issues, conflicts and name dependencies that I worked hard to circumvent in the protobuf-net implementation

If you keep the protobuf-net as polymorphic, then you could presumably just start from the .proto you posted, and just check (after deserialization on the java side) whether ProgramInfoAndroid or ProgramInfoWindows is non-null; then based on which is non-null, map it to one of 3 different domain types.