How to read images from memory to ImageSource in C++/WinRT (WinUI3)?

Question

[1]

I want to display an image for the Image control in XAML, but this image comes from a bytes array.

What should I do? The image format may be one of JPG, BMP, or PNG.

<Image x:Name="img"/>

void MainWindow::myButton_Click(IInspectable const&, RoutedEventArgs const&) {
    unsigned long long size; // iamge data size
    const unsigned char* buf; // image data

    img().Source(???); // what should I do?
}

[2]

I tried InMemoryRandomAccessStream, but it didn't work.

I know it's convenient to use MemoryStream in C#, but how to implement it in C++?

Another question is whether this method works for JPG and PNG format images as well?

void MainWindow::myButton_Click(IInspectable const&, RoutedEventArgs const&) {
    unsigned long long size; // iamge data size
    const unsigned char* buf; // image data

    winrt::Microsoft::UI::Xaml::Media::Imaging::BitmapImage bmp;
    winrt::Windows::Storage::Streams::InMemoryRandomAccessStream stream;
    // How to read data from stream?

    bmp.SetSource(stream);
    img().Source(bmp);
}

[3]

Now I have implemented the function, but I still have two questions.

IAsyncAction MainWindow::myButton_Click(IInspectable const&, RoutedEventArgs const&) {
    unsigned long long size; // iamge data size
    const unsigned char* buf; // image data

    winrt::Microsoft::UI::Xaml::Media::Imaging::BitmapImage bmp;
    winrt::Windows::Storage::Streams::InMemoryRandomAccessStream stream;
    winrt::Windows::Storage::Streams::DataWriter dw(st.GetOutputStreamAt(0ULL));

    dw.WriteBytes({ buf, size }); // This is very inefficient!
    co_await dw.StoreAsync();

    dw.Close();
    bmp.SetSource(stream);
    img().Source(bmp);
    stream.Close();
}

Firstly, I found that DataWriter::WriteBytes() is indeed copying data. But I think ImageSource only needs to be read from my buf, and there is no need to completely copy the entire image into the stream before reading it. Perhaps it's something like MemoryView, Am I thinking this wrong?

Secondly, where should the Close() of DataWriter and InMemoryRandomAccessStream be called? Do BitmapImage need to free up memory? There is an asynchronous function here, and I can't figure out where it should be written.

[4]

According to IInspectable, I tried SHCreatMemStream and searched for a lot of information to come up with another solution. The current problem is that it is equally feasible as method [3], but memory allocation still occurs.

#include "Shlwapi.h"
#include "shcore.h"
#pragma comment(lib, "shlwapi.lib")

void MainWindow::myButton_Click(IInspectable const&, RoutedEventArgs const&) {
    unsigned long long size; // iamge data size
    const unsigned char* buf; // image data

    winrt::Microsoft::UI::Xaml::Media::Imaging::BitmapImage bmp;
    IStream* stream{ SHCreateMemStream(buf, size) }; // Copying takes a lot of time
    static const GUID guidIRandomAccessStream = { 0x905a0fe1, 0xbc53, 0x11df, { 0x8c, 0x49, 0x00, 0x1e, 0x4f, 0xc6, 0x86, 0xda } };

    winrt::Windows::Storage::Streams::IRandomAccessStream pRas{ };
    CreateRandomAccessStreamOverStream(stream, BSOS_OPTIONS::BSOS_DEFAULT, guidIRandomAccessStream, (void**)&pRas);

    bmp.SetSource(stream);
    img().Source(bmp);
}

Through performance testing, it was found that the time for CreateRandomAccessStreamOverStream is consistent, but the time consumed by SHCreateMemStream is proportional to the image size. It can be concluded that the process of creating a stream in SHCreateMemStream also involves copying memory.

I am not particularly familiar with Stream in Windows programming. Does the process of creating a stream necessarily involve memory copying? Can loading images from memory avoid additional overhead? And is there no structure similar to MemoryStreamView to replace it?

• Another question, I know that new images must require memory storage.
  
  Will BitmapImage setting Stream as Source take over the memory occupied by the Stream?
  
  and will the Image control setting BitmapImage as Source take over the memory occupied by BitmapImage?
  
  If that's the case, I can accept Stream allocating new memory, otherwise it's still an efficiency problem.

Thanks!!!!!!!

Answer 1

This problem has been completely solved by using Stream. As detailed in the first four issues.

As for the additional issue:

★1

use CreatStreamOnHGlobal instead of SHCreatMemStream.

Because SHCreatMemStream will perform another memory copy on the incoming memory block parameters during the call.

CreatStreamOnHGlobal does not perform memory copying after creating a stream. You can first allocate memory through GlobalAlloc, write the image data you need, and then call CreatStreamOnHGlobal without any time overhead. In addition, the second parameter of CreatStreamOnHGlobal can be set to true to automatically release the memory allocated by GlobalAlloc.

Through process memory monitoring, I found that for the same program, using CreatStreamOnHGlobal is twice as fast as SHCreatMemStream and does not result in memory leaks.

★2

The created Stream requires calling the Release method to reduce one reference.

IRandomAccessStream, as a local variable, will automatically destruct without the need for active release.

And BitmapImage also manages a portion of the references, which are released after leaving the scope.

Through testing the return value of the Release method on COM objects, it was found that the reference count returned to 0 after the entire program ended