How to modify an Avalonia WritableBitmap and update the changes as image control on UI?

Question

I have a WritableBitmap object _myBitmap in my MainWindowViewModel class which I want to use to show the live footage from a webcam. But essentially, what I have to do is to write some new values into the WritableBitmap's frame buffer periodically, as shown in the method _changeMyBitmapColor.

Expected app behavior: the gray level of the bitmap changes from 0 to 50, 100, 150... every 500ms.

Actual app behavior: The change of the bitmap color is only reflected on the UI when the app window is resized or when the dummy combo box is pressed. The color changed from 0 to 50 and seems to stay at 50 forever. Also, the displayed bitmap only changed its color when the window is resized or the dummy ComboBox in MainWindow.axaml is pressed.

I have no idea why resizing the window or poking the dummy ComboBox on the UI updates the displayed bitmap. Nor do I understand why the displayed bitmap only seem to change its color once throughout the lifetime of the app. From the console logs, it is confirmed that the framebuffer of the WritableBitmap _myBitmap is successfully modified every 500ms.

Here is a reproducible example. I implemented the ReactiveUI approach as described in this tutorial. Since there are unsafe methods in the project, there is a <AllowUnsafeBlocks>true</AllowUnsafeBlocks> in the TryUpdateBitmap.csproj. The project is created using the default Avalonia MVVM project template by running:

dotnet new avalonia.mvvm -o TryUpdateBitmap

File: MainWindowViewModel

namespace TryUpdateBitmap.ViewModels;

using Avalonia.Media.Imaging;
using Avalonia;
using Avalonia.Platform;
using System.Threading;
using System;
using System.Threading.Tasks;
using ReactiveUI;

public class MainWindowViewModel : ReactiveObject
{
    public MainWindowViewModel()
    {
        // Assume the bitmap is of shape 640x480 with 4 channels
        _myBitmap = new WriteableBitmap(new PixelSize(640, 480), new Vector(96, 96), PixelFormat.Bgra8888, AlphaFormat.Opaque);
        Console.WriteLine("Start app");
        Task.Run(_updateMyBitmapColorPeriodicallyAsync);
    }
    public WriteableBitmap MyBitmap 
    { 
        get => _myBitmap; 
        set => this.RaiseAndSetIfChanged(ref _myBitmap, value); 
    }

    private WriteableBitmap _myBitmap;
    private  async Task _updateMyBitmapColorPeriodicallyAsync()
    {
        PeriodicTimer timer = new(TimeSpan.FromMilliseconds(500)); // Color of my bitmap pixels changes every 500ms
        int colorOption = 0;
        while (await timer.WaitForNextTickAsync())
        {
            _changeMyBitmapColor(colorOption);
            colorOption = (colorOption + 1) % 5; // there will be 3 different color options
            Console.WriteLine("Bitmap modified!");
        }
    }
    private unsafe void _changeMyBitmapColor(int colorOption)
    {
        byte[] pixelValueOptions = [50, 100, 150, 200, 225];
        using ILockedFramebuffer frameBuffer = MyBitmap.Lock();
        void* dataStart = (void*)frameBuffer.Address;
        Span<byte> buffer = new Span<byte>(dataStart, 640 * 480 * 4); // assume each pixel is 1 byte in size and my image has 4 channels
        buffer.Fill(pixelValueOptions[colorOption]); // fill the bitmap memory buffer with some values
    }
}

File: MainWindow.axaml

<Window xmlns="https://github.com/avaloniaui"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:vm="using:TryUpdateBitmap.ViewModels"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        mc:Ignorable="d" d:DesignWidth="800" d:DesignHeight="450"
        x:Class="TryUpdateBitmap.Views.MainWindow"
        x:DataType="vm:MainWindowViewModel"
        Icon="/Assets/avalonia-logo.ico"
        Title="IHaveGivenUpIHaveLetMyselfDownIWillRunAroundAndDesertMyself">

    <Design.DataContext>
        <vm:MainWindowViewModel/>
    </Design.DataContext>
    <Grid ColumnDefinitions="640" RowDefinitions="500">
        <Image Source="{Binding MyBitmap, Mode=TwoWay}" Grid.Column="0" Grid.Row="0"/>
        <ComboBox Grid.Column="0" Grid.Row="1"/>
    </Grid>
</Window>

Any help is much appreciated! Happy Chinese New Year.

Answer 1

I have found two viable solutions, both implements the INotifyPropertyChange interface instead of using ReactiveUI from this guide.

So the key to this approach is to alter the writeablebitmap object referenced by the property _myBitmap. Destroying and reallocating a new bitmap instance and assigning to _myBitmap every frame might not be ideal for performance, so writing pixels to it as bytes is more desirable. But in that case, the instance referenced by _myBitmap always stays the same and things just won't work.

Solution1: Implement INotifyPropertyChange

File: MainWindowViewModel

namespace TryUpdateBitmap.ViewModels;

using Avalonia.Media.Imaging;
using Avalonia;
using Avalonia.Platform;
using System.Threading;
using System;
using System.Threading.Tasks;
using System.Collections.Generic;
using System.ComponentModel;

public class MainWindowViewModel : INotifyPropertyChanged
{
    public MainWindowViewModel()
    {
        // Assume the bitmap is of shape 640x480 with 4 channels
        _myBitmap = new WriteableBitmap(new PixelSize(640, 480), new Vector(96, 96), PixelFormat.Bgra8888, AlphaFormat.Opaque);
        _currentBitmapIndex = 0;
        _scratchBoards = [];
        _scratchBoards.Add(new WriteableBitmap(new PixelSize(640, 480), new Vector(96, 96), PixelFormat.Bgra8888, AlphaFormat.Opaque));
        _scratchBoards.Add(new WriteableBitmap(new PixelSize(640, 480), new Vector(96, 96), PixelFormat.Bgra8888, AlphaFormat.Opaque));
        Console.WriteLine("Start app");
        Task.Run(_updateMyBitmapColorPeriodicallyAsync);
    }
    public WriteableBitmap MyBitmap 
    { 
        get => _myBitmap; 
        set 
        {
            _myBitmap = value;
            OnPropertyChanged(nameof(MyBitmap));
        }
    }
    public event PropertyChangedEventHandler? PropertyChanged;
    protected virtual void OnPropertyChanged(string propertyName)
    {
        PropertyChanged?.Invoke(this, new PropertyChangedEventArgs(propertyName));
    }
    private WriteableBitmap _myBitmap;
    private int _currentBitmapIndex;
    private List<WriteableBitmap> _scratchBoards;
    private  async Task _updateMyBitmapColorPeriodicallyAsync()
    {
        PeriodicTimer timer = new(TimeSpan.FromMilliseconds(500)); // Color of my bitmap pixels changes every 500ms
        int colorOption = 0;
        while (await timer.WaitForNextTickAsync())
        {
            _changeMyBitmapColor(colorOption);
            colorOption = (colorOption + 1) % 5; // there will be 3 different color options
            Console.WriteLine("Bitmap modified!");
            _currentBitmapIndex = (_currentBitmapIndex + 1) % 2;
        }
    }
    private unsafe void _changeMyBitmapColor(int colorOption)
    {
        byte[] pixelValueOptions = [50, 100, 150, 200, 225];
        using ILockedFramebuffer frameBuffer = _scratchBoards[_currentBitmapIndex].Lock();
        void* dataStart = (void*)frameBuffer.Address;
        Span<byte> buffer = new Span<byte>(dataStart, 640 * 480 * 4); // assume each pixel is 1 byte in size and my image has 4 channels
        buffer.Fill(pixelValueOptions[colorOption]); // fill the bitmap memory buffer with some values
        MyBitmap = _scratchBoards[_currentBitmapIndex];
    }
}

By alternating the writablebitmap instance referenced by MyBitmap between the two elements in _scratchBoards(note do not modify the private member _myBitmap), the set accesser of MyBitmap is guaranteed to be triggered every frame, which raises a PropertyChanged event through OnPropertyChanged, which then notifies the view to re-render the displayed image.

Solution2: use CommunityToolkit

File: MainWindowViewModel

namespace TryUpdateBitmap.ViewModels;

using Avalonia.Media.Imaging;
using Avalonia;
using Avalonia.Platform;
using System.Threading;
using System;
using System.Threading.Tasks;
using System.Collections.Generic;
using CommunityToolkit.Mvvm.ComponentModel;

public partial class MainWindowViewModel : ObservableObject
{
    public MainWindowViewModel()
    {
        // Assume the bitmap is of shape 640x480 with 4 channels
        _myBitmap = new WriteableBitmap(new PixelSize(640, 480), new Vector(96, 96), PixelFormat.Bgra8888, AlphaFormat.Opaque);
        _currentBitmapIndex = 0;
        _scratchBoards = [];
        _scratchBoards.Add(new WriteableBitmap(new PixelSize(640, 480), new Vector(96, 96), PixelFormat.Bgra8888, AlphaFormat.Opaque));
        _scratchBoards.Add(new WriteableBitmap(new PixelSize(640, 480), new Vector(96, 96), PixelFormat.Bgra8888, AlphaFormat.Opaque));
        Console.WriteLine("Start app");
        Task.Run(_updateMyBitmapColorPeriodicallyAsync);
    }
    [ObservableProperty]
    private WriteableBitmap _myBitmap;
    private int _currentBitmapIndex;
    private List<WriteableBitmap> _scratchBoards;
    private  async Task _updateMyBitmapColorPeriodicallyAsync()
    {
        PeriodicTimer timer = new(TimeSpan.FromMilliseconds(500)); // Color of my bitmap pixels changes every 500ms
        int colorOption = 0;
        while (await timer.WaitForNextTickAsync())
        {
            _changeMyBitmapColor(colorOption);
            colorOption = (colorOption + 1) % 5; // there will be 3 different color options
            Console.WriteLine("Bitmap modified!");
            _currentBitmapIndex = (_currentBitmapIndex + 1) % 2;
        }
    }
    private unsafe void _changeMyBitmapColor(int colorOption)
    {
        byte[] pixelValueOptions = [50, 100, 150, 200, 225];
        using ILockedFramebuffer frameBuffer = _scratchBoards[_currentBitmapIndex].Lock();
        void* dataStart = (void*)frameBuffer.Address;
        Span<byte> buffer = new Span<byte>(dataStart, 640 * 480 * 4); // assume each pixel is 1 byte in size and my image has 4 channels
        buffer.Fill(pixelValueOptions[colorOption]); // fill the bitmap memory buffer with some values
        MyBitmap = _scratchBoards[_currentBitmapIndex]; // note that MyBitmap is implemented by community toolkit since _myBitmap is marked as ObservableProperty
    }
}

Huge thanks to the CommunityToolkit, the code needed is significantly reduced. To my knowledge, this approach does similar stuff as Solution1 under the hood, but it abstracts away a lot of boilerplate. What we have to do is to mark the MainWindowViewModel class as partial, inherit from ObservableObject and mark the private property _myBitmap as ObservableProperty. It will generate the public field MyBitmap and by changing the object referenced by MyBitmap(instead of _myBitmap), it notifies the property change and the displayed bitmap will be re-rendered.

Final note:

Both the two proposed solutions work but is likely not the optimal way to go. There are two additional writeablebitmap instances allocated, and alternating the object reference between the elements in the list seems a bit funny. I'm pretty new to Avalonia, MVVM, or to C# in general. Perhaps using the ReactiveUI approach or use Avalonia.Threading.Dispatch.UIThread is even simpler, but I haven't made it work. yet.
So for anyone with a better solution seeing this, please kindly share it. Much appreciated! Cheers!