RtspClientSharp - WriteableBitmap memory leak

48 Views Asked by At

Hello StackOverflow community,

I'm currently working on a C# application that connects to cameras and receives H.264 compressed video frames via RTSP. The RTSP service is defined by the following method:

private async Task ReceiveAsync(CancellationToken token)
{
    try
    {
        System.Net.NetworkCredential credential = new System.Net.NetworkCredential(userName, password);
        ConnectionParameters connectionParameter = new ConnectionParameters(new Uri(m_uri), credential);

        if (m_ssl) connectionParameter.RtpTransport = RtpTransportProtocol.TCP;
        else connectionParameter.RtpTransport = RtpTransportProtocol.UDP;

        using (var rtspClient = new RtspClient(connectionParameter))
        {
            rtspClient.FrameReceived += RtspClientOnFrameReceived;

            while (!token.IsCancellationRequested)
            {
                try
                {
                    await rtspClient.ConnectAsync(token);
                }
                catch (InvalidCredentialException)
                {
                    await Task.Delay(RetryDelay, token);
                    continue;
                }
                catch (RtspClientException)
                {
                    await Task.Delay(RetryDelay, token);
                    continue;
                }

                try
                {
                    await rtspClient.ReceiveAsync(token);
                }
                catch (RtspClientException)
                {
                    await Task.Delay(RetryDelay, token);
                }
            }
        }
    }
    catch (OperationCanceledException)
    {
    }
}

Every time a frame is received, the RtspClientOnFrameReceived method is executed. This method receives an H.264 encoded frame, decodes it, and the decoded image is then passed to an RtspImage with each frame. I present the class and now I will explain the issue I am facing:

private void RtspClientOnFrameReceived(object sender, RawFrame rawFrame)
{
    try
    {
        if (!(rawFrame is RawVideoFrame rawVideoFrame))
            return;

        DecodedVideoFrame decodedFrame = m_decoder.Decoder(rawVideoFrame) as DecodedVideoFrame;

        if (decodedFrame == null) return;

        lock (m_locker)
        {
            m_image = new RtspImage(m_height, m_width, decodedFrame);
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine("Error: " + ex.Message);
    }
}

The problem I am encountering is that RtspImage creates a WriteableBitmap, which is used to store the frame. However, this WriteableBitmap is never released, and it starts consuming memory. I have been conducting a dynamic memory analysis with ReSharper, and it indicates an object MediaContext from System.Windows.Media that increases its objects and memory but never releases that memory. I don't understand the reason for this, as I believe that with each new instance of RtspImage, the class's destructor should be called to release all its resources. I am sharing the RtspImage class:

internal class RtspImage : SntImage
{
    public RtspImage(int height, int width, DecodedVideoFrame decodedVideoFrame)
    {
        try
        {
            // Create WriteableBitmap to copy the pixel data to.
            WriteableBitmap target = new WriteableBitmap(
                width,
                height,
                96.0, 96.0,
                PixelFormats.Pbgra32,
                null);

            TransformParameters transformParameters = new TransformParameters(RectangleF.Empty,
                    new System.Drawing.Size(width, height),
                    ScalingPolicy.Stretch, PixelFormat.Bgra32, ScalingQuality.FastBilinear);

            decodedVideoFrame.TransformTo(target.BackBuffer, target.BackBufferStride, transformParameters);

            m_bitMap = TransformImage(target);
            m_bitMap.Freeze();
        }
        catch (Exception e)
        {
            throw e;
        }
    }

    public override bool CanDefineRegion => true;
    public override BitmapSource Image => m_bitMap;

    private BitmapSource m_bitMap;
}

I would appreciate any help in fixing this memory leak. I tried removing the WriteableBitmap from the RtspImage class, adding it to the class that creates the RtspClientSharp, and reusing this instance. To do this, I had to wrap the creation of RtspImage with Application.Current.Dispatcher.InvokeAsync since modifying these types of objects requires using the UI thread. This fixed the memory leak issue, but using the UI thread with multiple connected cameras introduced lag, making the application unusable. Therefore, I discarded this option.

I would like to know if anyone has encountered this problem or knows how to fix it.

0

There are 0 best solutions below