Protobuf use repeated oneof message vs many empty fields

1.8k Views Asked by At

I'm designing a protobuf to represent an event, where each event can hold extra fields.
There are a lot of possible extra fields (~100), but only a small portion of them will be used in each message (~3)

Each extra field will be used only once, but multiple of them can exist, therefore I would like to have a concept of an anyof message type, but unfortunately, there is no such thing in protobuf.

So to try and mock this behavior, and as mentioned in this discussion I thought I can put all my extra fields in a oneof, wrap it with a message, and use this message as repeated in my event:

message ExtraField {
  oneof extra_field_value {
    string extraData1 = 1;
    uint64 extraData2 = 2;
    ....
    SomeOtherMessage extraData100 = 100;
  }
}

message MyEvent {
  uint64 timestamp = 1;
  string event_name = 2;
  string some_other_data = 3;
  ...
  repeated ExtraField extra_fields = 8;
}

Even though this solution is more explicit for my understanding, it isn't the most memory effective, and the repeated message with oneof implementation allows to add the same extra field more than once (unwanted behavior)

I can also just write all the extra fields as-is in an inner message, but most of them will be empty all the time

message ExtraFields {
  string extraData1 = 1;
  uint64 extraData2 = 2;
  ....
  SomeOtherMessage extraData100 = 100;
}

message MyEvent {
  uint64 timestamp = 1;
  string event_name = 2;
  string some_other_data = 3;
  ...
  extraFields extra_fields = 8;
}

If I understand correctly, using empty fields in a message isn't going to make my serialized data larger, and therefore the second protobuf design is the preferred practice

Am I correct? Is there another protobuf design for my needs?

0

There are 0 best solutions below