How to store protobuf object in memcached?

Question

Following is the proto file I am using to create a GRPC system which gets data from the database/memcached.

message CMSContent
{
    repeated ArticleSummary Articles = 1;
    uint32 RecordCount = 2;
}

service Article
{
    rpc ListByCategory(ArticleByCatURI) returns (CMSContent){}
}

message ArticleByCatURI
{
    uint32 ApplicationId = 1;        
}

message ArticleSummary
{
    uint32 ArticleId = 1;
    string ArticleName   = 2;
}

I created the corresponding C# classes as detailed on GRPC Getting Started. I am using these classes on Server to store the result of a DB Query and trying to save these objects to memcached. Unfortunately the protobuf object is not getting added to memcached. Following is how I am adding it to memcached and the value of isKeyFirstTimeCreated is false when I try to add it.

 isKeyFirstTimeCreated = mc.Store(StoreMode.Add, key, t, DateTime.Now.Add(cacheDuration));

As per memcached requirement, all the objects which must be stored in memcached need to be Serializable. But the autogenerated protobuf classes are not serializable (at least to binary formatter which c# uses). Now since the classes are partial implementations, I made them serializable but that did not help as the RepeatedField is not Serializable.

IMO, there should be a better way or some mechanism which will allow me to save the protobuf objects directly into the memcached but unfortunately Googling didn't help me much in that regard.

I can achieve this in two ways

• by copying data from protobuf object to a serializable object and then save it into memcached for the future reference

• copy the data into a serilizable object and use that object to populate the protobuf object.

But these two approached need extra work. Is there a better way to achieve this?

[PS: I am using proto3 hence it supports c#]

Answer 1

Almost all cache stores support byte[]. Tools like protobuf lend themself directly to being stored in a byte[], so the most common approach here with binary serializers is essentially:

byte[] raw;
using(var ms = new MemoryStream(ms)) {
    YourChosenSerializer.Serialize(ms, obj);
    raw = ms.ToArray();
}
// now store raw

If your storage tool doesn't support byte[], then string is usually a good substitute, but binary serializers should be stores using something like base-64 encoding (via Convert.ToBase64String and Convert.FromBase64String).

As for the memcached library (or similar) supporting this directly: I would advise against this for separation of concerns reasons. However, extension methods usually make it very easy to provide this in a few lines in helper methods in your own code.