I am consuming Json from a TcpClient, and to get a solution with low allocation and good performance I decided to use the new System.IO.Pipelines for handling IO and System.Text.Json for deserialization. The output from the pipe is a ReadOnlySequence<byte>. I am ok when there is only one segment in the ReadOnlySequence, so that I can pass this segment (which is a ReadOnlySpan<byte>) to the deserializer. But what should I do with multiple segments?
What I have so far is the code below. But in some cases, the length of the sequence is too large, so I get a stack overflow in stackalloc. Also, making a copy of the data seems to me as breaking the intention of System.IO.Pipelines. Shouldn't System.Text.Json.JsonSerializer.Deserialize have a ReadOnlySequence overload? Any suggestions for how this should be solved?
private void ProcessLine(ReadOnlySequence<byte> sequence)
{
if (sequence.IsSingleSegment)
{
_result = JsonSerializer.Deserialize<MyType>(sequence.FirstSpan, _jsonSerializerOptions);
}
else
{
Span<byte> stackSpan = stackalloc byte[(int)sequence.Length];
sequence.CopyTo(stackSpan);
_result = JsonSerializer.Deserialize<MyType>(stackSpan, _jsonSerializerOptions);
}
}
Use the
Utf8JsonReadertype, which wraps a sequence (note: it can also wrap a span etc), and let it deal with the single/multi/etc segment concerns: