How to deserialize system.memory with JsonConvert.DeserializeObject in c#

Question

I have a class which uses a System.Memory<double> Property, let's call it PROP1 and CLASS1

When this class is serialized into a JSON file it's saved in a pretty common way :

(...) "PROP1":[7200.0,7200.0,7200.0] (...)

When I try to deserialize via JsonConvert.DeserializeObject<CLASS1>File.ReadAllText(fileName));

I get the following exception :

Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'System.Memory`1[System.Double]' because the type requires a JSON object (e.g. {"name":"value"}) to deserialize correctly. To fix this error either change the JSON to a JSON object (e.g. {"name":"value"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array.

I guess it cannot serialize it because Memory is working more like a pointer, so you should create an object first which the Memory refers to. But I could not find a fix to this (besides rewriting the class to another type)...and I couldn't find any threads with a similar problem. Any ideas how to deserialize it??

Answer 1

Since Json.NET doesn't seem to have built-in support for serializing and deserializing Memory<T> and ReadOnlyMemory<T>, you could create generic converters for them that will serialize and deserialize "snapshots" of the contents of Memory<T> and ReadOnlyMemory<T> slices as follows:

public class MemoryConverter<T> : JsonConverter<Memory<T>>
{
    public override void WriteJson(JsonWriter writer, Memory<T> value, JsonSerializer serializer) =>
        serializer.SerializeMemory((ReadOnlyMemory<T>)value, writer, false);
        
    public override Memory<T> ReadJson(JsonReader reader, Type objectType, Memory<T> existingValue, bool hasExistingValue, JsonSerializer serializer) =>
        reader.MoveToContentAndAssert().TokenType switch
        {
            JsonToken.String when typeof(T) == typeof(char) => 
                ((T [])(object)serializer.Deserialize<string>(reader).ToCharArray()).AsMemory(),
            JsonToken.StartArray when typeof(T) == typeof(byte) => 
                ((T [])(object)serializer.Deserialize<List<byte>>(reader).ToArray()).AsMemory(),
            _ => 
                serializer.Deserialize<T []>(reader).AsMemory()
        };
}

public class ReadOnlyMemoryConverter<T> : JsonConverter<ReadOnlyMemory<T>>
{
    public override void WriteJson(JsonWriter writer, ReadOnlyMemory<T> value, JsonSerializer serializer) =>
        serializer.SerializeMemory((ReadOnlyMemory<T>)value, writer, serializeAsString : true);
        
    public override ReadOnlyMemory<T> ReadJson(JsonReader reader, Type objectType, ReadOnlyMemory<T> existingValue, bool hasExistingValue, JsonSerializer serializer) =>
        reader.MoveToContentAndAssert().TokenType switch
        {
            JsonToken.String when typeof(T) == typeof(char) => 
                (ReadOnlyMemory<T>)(object)serializer.Deserialize<string>(reader).AsMemory(),
            JsonToken.StartArray when typeof(T) == typeof(byte) => 
                ((T [])(object)serializer.Deserialize<List<byte>>(reader).ToArray()).AsMemory(),
            _ => 
                serializer.Deserialize<T []>(reader).AsMemory()
        };
}

public static partial class JsonExtensions
{
    internal static void SerializeMemory<T>(this JsonSerializer serializer, ReadOnlyMemory<T> value, JsonWriter writer, bool serializeAsString) 
    {
        switch (value)
        {
            case ReadOnlyMemory<byte> m when MemoryMarshal.TryGetArray(m, out var seg) && seg.Offset == 0 && seg.Count == seg.Array.Length:
                writer.WriteValue(seg.Array); // Base64 encoded array.
            break;
            case ReadOnlyMemory<byte> m:
                writer.WriteValue(m.ToArray()); // Base64 encoded slice.
            break;
            case ReadOnlyMemory<char> m when serializeAsString:
                writer.WriteValue(m.ToString());
            break;
            default:
                serializer.Serialize(writer, MemoryMarshal.ToEnumerable(value));
            break;
        }
    }

    public static JsonReader MoveToContentAndAssert(this JsonReader reader)
    {
        ArgumentNullException.ThrowIfNull(reader);
        if (reader.TokenType == JsonToken.None)       // Skip past beginning of stream.
            reader.ReadAndAssert();
        while (reader.TokenType == JsonToken.Comment) // Skip past comments.
            reader.ReadAndAssert();
        return reader;
    }

    public static JsonReader ReadAndAssert(this JsonReader reader)
    {
        ArgumentNullException.ThrowIfNull(reader);
        if (!reader.Read())
            throw new JsonReaderException("Unexpected end of JSON stream.");
        return reader;
    }
}

Then you would serialize and deserialize your model using the following settings:

var settings = new JsonSerializerSettings
{
    Converters = { new MemoryConverter<double>(), new ReadOnlyMemoryConverter<double>() },
};

var json = JsonConvert.SerializeObject(class1, settings);
var model2 = JsonConvert.DeserializeObject<CLASS1>(json, settings);

And/or apply to your model as follows:

public class CLASS1
{
    [JsonConverter(typeof(MemoryConverter<double>))]
    public Memory<double> PROP1 { get; set; }
};

Warnings and notes:

• Warning: references of array slices are not preserved. If your Memory<double> is a slice of some array, and that array is also being serialized elsewhere in the model, then when deserialized the Memory<double> will not refer to the deserialized array by reference, it will refer to its own copy of the array.

  If you need to preserve the references of array slices, a different (and much more complicated) converter would be required.

• Since byte arrays are serialized as Base64 strings by Json.NET (and System.Text.Json), I did the same for Memory<byte> and ReadOnlyMemory<byte>. But if the Memory<byte> had been previously serialized as a JSON array, it will be read properly.

• Since ReadOnlyMemory<char> can sometimes wrap a string, I serialized it as such, but did not do the same for Memory<char> which can only wrap a char array.

  If you don't want that, pass serializeAsString : false inside ReadOnlyMemoryConverter<T>.Read().

• Absent the converters, I was unable to generate a reasonable serialization for Memory<T> out of the box with Json.NET 13.0.3. Instead of a JSON array, I got

  {"PROP1":{"Length":6,"IsEmpty":false}}
  

  Maybe you already have some converter installed that handles serialization but not deserialization?

Demo fiddle here.

Answer 2

The below answer is for JSON.Net.

---

You can use a custom JsonConverter.

public class MemoryConverter<T> : JsonConverter
{
    public override bool CanConvert(Type objectType) =>
        typeof(Memory<T>).IsAssignableFrom(objectType);

    public override bool CanRead => true;

    public override bool CanWrite => true;

    public override object ReadJson(JsonReader reader, 
                                    Type objectType, 
                                     object existingValue, 
                                     JsonSerializer serializer)
    {
        var mem = new Memory<T>(serializer.Deserialize<T[]>(reader));
        if (existingValue is Memory<T> existingMem && !existingMem.IsEmpty)
        {
            mem.CopyTo(existingMem);
            return existingMem;
        }
        return mem;
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        if (!(value is Memory<T> mem))
            throw new ArgumentException($"invalid type {value?.GetType().FullName}");
        writer.WriteStartArray();
        var span = mem.Span;
        for (var item in span)
        {
            serializer.Serialize(writer, item);
        }
        writer.WriteEndArray();
    }
}

Then you can apply it

[JsonConverter(typeof(MemoryConverter<int>))]
public Memory<int> YourValue { get; set; }

dotnetfiddle