C#: Wrapping SerialPort.BaseStream in StreamReader seemingly unable to deal with only CR as NewLine

Question

I ran into an issue with communication over serial ports using C#, .NET Framework 4.8 and com0com virtual bridge (I do not believe the last one actually matters). The implementation that I am using uses only CR without LF as a line separator and the receiving port is somehow unable to cope with it. I am using the following code as a test:

[Test]
public void NewLineTest()
{
    using var serverSerialPort = new SerialPort("COM4")
    {
        BaudRate = 115200,
        DataBits = 8,
        Parity = Parity.None,
        Handshake = Handshake.None,
        StopBits = StopBits.One,
        NewLine = "\r",
        Encoding = Encoding.ASCII
    };

    using var clientSerialPort = new SerialPort("COM5")
    {
        BaudRate = 115200,
        DataBits = 8,
        Parity = Parity.None,
        Handshake = Handshake.None,
        StopBits = StopBits.One,
        NewLine = "\r",
        Encoding = Encoding.ASCII
    };

    serverSerialPort.Open();
    clientSerialPort.Open();

    const string test = "test";
    serverSerialPort.WriteLine(test);
    string readLine = clientSerialPort.ReadLine();
    Assert.AreEqual(test, readLine);    // this is OK

    using var writer = new StreamWriter(serverSerialPort.BaseStream, Encoding.ASCII)
    {
        NewLine = "\r",
        AutoFlush = true
    };

    using var reader = new StreamReader(clientSerialPort.BaseStream, Encoding.ASCII);

    writer.WriteLine(test);
    readLine = reader.ReadLine();   // never returns unless "NewLine = \r" occurrences are commented out
    Assert.AreEqual(test, readLine);
}

As you can see, writing and reading directly to/from the SerialPort object works okay. The problem is when it is wrapped using StreamWriter/StreamReader, in which case the ReadLine() method freezes and only stops when using a timeout. Up to the WriteLine() it however seems to be working just fine, because if I listen to the output by using another application instead of a SerialPort client, the message comes as normal and the CR separator is there as before. The code above even works when using the full CR LF instead of just CR. I cannot make heads or tails out of this, considering that the implemetation of StreamWriter/StreamReader WriteLine/Readline seems to be in favor of supporting various line separators.

I also tried using a different SerialPort implementation such as SerialPortStream from a nuget, but I did not notice any difference.