Protostuff cannot be converted into Protobuf bytes?

362 Views Asked by At

Protostuff has a ProtobufIOUtil that seems to be able to convert Java objects into protobuf bytes. Currently, it has been found that there may be differences when dealing with map types.

For example:

@Getter
@Setter
public class User {
    @Tag(1)
    private long uid;

    @Tag(2)
    private String name;

    @Tag(3)
    private Map<Long, Integer> map = new HashMap<>(); // Map type field
}
syntax="proto3";

message User {
  int64 uid = 1;
  string name = 2;
  map<int64, int32> map = 3;
}

The test:

public class Test {
    public static void main(String[] args) throws InvalidProtocolBufferException {
        User user = new User();
        user.setUid(1);
        user.setName("s");
        Map<Long, Integer> map = new HashMap<Long, Integer>();
        map.put(1L, 1);
        user.setMap(map);
        Schema<User> schema = RuntimeSchema.getSchema(User.class);
        byte[] bytes = ProtobufIOUtil.toByteArray(user, schema, LinkedBuffer.allocate());
        System.out.println(Arrays.toString(bytes));

        UserOuterClass.User pbUser = UserOuterClass.User.newBuilder()
                .setUid(1)
                .setName("s")
                .putMap(1, 1)
                .build();
        System.out.println(Arrays.toString(pbUser.toByteArray()));
        UserOuterClass.User newPbUser = UserOuterClass.User.parseFrom(bytes);
        System.out.println(newPbUser.toString());
    }
}

The console output is:

[8, 1, 18, 1, 115, 26, 6, 10, 4, 8, 1, 16, 1]
[8, 1, 18, 1, 115, 26, 4, 8, 1, 16, 1]
uid: 1
name: "s"
map {
  key: 0
  value: 0
}

The dependencies:

    <dependency>
            <groupId>io.protostuff</groupId>
            <artifactId>protostuff-core</artifactId>
            <version>1.7.4</version>
        </dependency>
        <dependency>
            <groupId>io.protostuff</groupId>
            <artifactId>protostuff-runtime</artifactId>
            <version>1.7.4</version>
        </dependency>
        <dependency>
            <groupId>io.protostuff</groupId>
            <artifactId>protostuff-runtime-registry</artifactId>
            <version>1.7.4</version>
        </dependency>

        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>3.21.8</version>
        </dependency>

Bytes generated by ProtobufIOUtil.toByteArray() in ProtoStuff differs from bytes generated in ProtoBuf, once with Map type fields in Pojo class.

The map data structure encoded by protobuf is roughly fieldTag, eleLen, eleData, fieldTag, eleLen, eleData.

The byte data converted from protostuff to protobuf is roughly fieldTag, fieldLen, fieldTag, eleLen, eleData, fieldTag, eleLen, eleData.

Compared to protobuf, there are two more data points: fieldTag, fieldLen.

Anyway, I want the bytes generated by protostuff to be parsed in protobuf to generate the same object as the original object.

Is there any relevant configuration for protostuff that supports not writing fieldTag and fieldLen at the beginning?

If not, does protostuff support custom encoding and decoding (I know Delegate could be used to customize encoding and decoding, but it seems to require writing by myself. I want to combine it with the original map encoding and decoding, but I haven't found an implementation yet).

---UPDATE---

I now decide to use Delegate:

public class MapDelegate {
    public static final Delegate<Map<Long, Integer>> MAP_INT_LONG_DELEGATE = new Delegate<Map<Long, Integer>>() {
        @Override
        public WireFormat.FieldType getFieldType() {
            return WireFormat.FieldType.MESSAGE;
        }

        @Override
        public Map<Long, Integer> readFrom(Input input) throws IOException {
            // to be implemented
        }

        @Override
        public void writeTo(Output output, int number, Map<Long, Integer> value, boolean repeated) throws IOException {
            // to be implemented
        }

        @Override
        public void transfer(Pipe pipe, Input input, Output output, int number, boolean repeated) throws IOException {
            throw new UnsupportedOperationException("Transfer operation is not supported.");
        }

        @Override
        public Class<?> typeClass() {
            return Map.class;
        }
    };

}

The updated Test:

public class Test {
    public static void main(String[] args) throws InvalidProtocolBufferException {
        DefaultIdStrategy strategy = (DefaultIdStrategy) RuntimeEnv.ID_STRATEGY;
        strategy.registerDelegate(MAP_INT_LONG_DELEGATE);

        ...
    }
}

But no idea on how to implement readFrom and writeTo.

0

There are 0 best solutions below