Screenshot video colours are brighter or darker than those on screen

Question

I'm trying to encode a video from a series of screenshots of a UWP control. I can export still images with no issue, but when I try to convert a series of screenshots to a video the colours in the resulting video are either:

• brighter than those on screen when I use DirectXPixelFormat.B8G8R8A8UIntNormalized
• dimmer than those on screen when I use DirectXPixelFormat.B8G8R8A8UIntNormalizedSrgb

Using the code below.

How to I generate a video with the same colours as those on screen?

Bonus question: how to I reduce the number of steps to go from the UIElement to the video?

public async Task RenderVideo(UIElement content, StorageFile file)
        {
            TimeSpan frameTime = TimeSpan.FromMilliseconds(1000 / fps);

            MediaComposition composition = new();           

            var width = (uint)content.ActualSize.X;
            var height = (uint)content.ActualSize.Y;
            for (some loop)
            {
                // Code here to modify the "content" control

                RenderTargetBitmap rendertargetBitmap = new();
                await rendertargetBitmap.RenderAsync(content);

                CanvasRenderTarget rendertarget = null;
                using (CanvasBitmap canvas = CanvasBitmap.CreateFromBytes(
                    CanvasDevice.GetSharedDevice(),
                    await rendertargetBitmap.GetPixelsAsync(),
                    rendertargetBitmap.PixelWidth,
                    rendertargetBitmap.PixelHeight,

// Pixel format specified here:
                    DirectXPixelFormat.B8G8R8A8UIntNormalized))
                {
                    rendertarget = new CanvasRenderTarget(CanvasDevice.GetSharedDevice(), width, height, 96);
                    using CanvasDrawingSession ds = rendertarget.CreateDrawingSession();
                    ds.Clear(Colors.White);
                    ds.DrawImage(canvas, 0, 0);
                }

                MediaClip clip = MediaClip.CreateFromSurface(rendertarget, frameTime);
                composition.Clips.Add(clip);
            }

            var profile = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.HD1080p);

            // Avoid odd video dimensions which some encoders don't like
            profile.Video.Width = (width % 2 == 0) ? width : width + 1;
            profile.Video.Height = (height % 2 == 0) ? height : height + 1;

            var saveOperation = composition.RenderToFileAsync(file, MediaTrimmingPreference.Fast, profile);

            saveOperation.Progress = new AsyncOperationProgressHandler<TranscodeFailureReason, double>(async (info, progress) =>
            {
                await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, new DispatchedHandler(() =>
                {
                    // Report progress
                }));
            });
            saveOperation.Completed = new AsyncOperationWithProgressCompletedHandler<TranscodeFailureReason, double>(async (info, status) =>
            {
                await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, new DispatchedHandler(() =>
                {
                    // Report success of failure
                }));
            });
        }

EDIT1:

I think I've tracked the issue down to a possible bug in how Win2D canvases with bitmap images are rendered with RenderTargetBitmap. In my main app the Win2D canvas is semi-transparent, so the brightness may have been from a white background that is handled differently on screen than by RenderTargetBitmap.

Here is the issue described as simply as I could manange:

Check out the UWP app here. This app displays three buttons for exporting and blocks of colours rendered three different ways:

• As UWP Rectangle controls
• As Win2D Filled rectangles
• As a Win2D bitmap

I refer to these two images in the text below:

A B

When no background is set on the parent control of the Win2D canvas (the StackPanel named Colours) it displays as image A on screen, but copies to the clipboard (using the button) as image B (where the white area is transparent).

If I set the background of the parent control to Black, it displays as image A on screen, and copies to the clipboard as image A.

If I set the background of the parent control to White, it displays as image B both on screen and also copies to the clipboard as image B.

I'm assuming this is a bug-or am I missing something?

Answer 1

The solution is to use 255 for the alpha as:

        var colours = new byte[]
        {
            255, 0, 0, 255,
            0, 255, 0, 255,
            0, 0, 255, 255
        };

It is because CanvasBitmap.CreateFromBytes uses premultiplied alpha as described at Premultiplied alpha.

Premultiplied Alpha Blending

Here's a different way to think about transparency:

RGB specifies how much color the object contributes to the scene Alpha specifies how much it obscures whatever is behind it In math:

blend(source, dest)  =  source.rgb + (dest.rgb * (1 - source.a))

In code:

RenderState.SourceBlend = Blend.One;
RenderState.DestinationBlend = Blend.InverseSourceAlpha;

In this world, RGB and alpha are linked. To make an object transparent.

The function definition from the metadata:

    //
    // Summary:
    //     Creates a CanvasBitmap from an array of bytes, using the specified pixel width/height,
    //     premultiplied alpha and default (96) DPI.
    [DefaultOverload]
    [Overload("CreateFromBytes")]
    public static CanvasBitmap CreateFromBytes(ICanvasResourceCreator resourceCreator, byte[] bytes, int widthInPixels, int heightInPixels, DirectXPixelFormat format);
    

The code:

        var colours = new byte[]
        {
            255, 0, 0, 0,
            0, 255, 0, 0,
            0, 0, 255, 0
        };

        CanvasBitmap bitmap = CanvasBitmap.CreateFromBytes(args.DrawingSession, colours, 1, 3, DirectXPixelFormat.R8G8B8A8UIntNormalized);
        args.DrawingSession.DrawImage(bitmap, new Rect(0, 150, 200, 150));

As the alpha value is 0 for each color, based on the formula from above, the result is source.rgb + dest.rgb. If the dest.rgb is white (255, 255, 255), it is (255, 255, 255). If the dest.rgb is black (0, 0, 0, 0), it is source.rgb. If the dest.rgb is Blue (0, 0, 255), it is (source.r, source.g, 255), and so forth.

When the alpha value is 255 for each color, based in the formula, the result is source.rgb regardless of dest.rgb.