IMFTransform covert camera stream color format, IMFTransform::ProcessOutput returns freeze image data

Question

I'm very new to Windows Media Foundation API. I try to create a program that can display and manipulate camera video input stream.
I'm using Media Foundation for video stream reading, the supportted color format is NV12, so I had to convert it to RGB or ARGB to create Opengl texture object.
I tried to do it using pure c++ code on CPU side, but the perfomance is very bad. Then I tried to use IMFTranform to do color convertion, the performance is very good, but I got a strange problem.
IMFTransform::Process returns S_OK, but I keep getting freezed image output. The result image pixel are not updated in realtime, keeps getting freezed for many frames, and the freezing time is getting longer and longger. If I use pure c++ code the do the color convertion, then everything works fine.

Here are my code, https://gist.github.com/zhiqiang-li/16d1a6a1b00e8fb39847c8ca323b5604. Please let me konw what do you think I'm doing wrong.

Answer 1

Did you try to let the SourceReader do the conversion for you :

hr = mAttributes->SetUINT32(MF_READWRITE_DISABLE_CONVERTERS, FALSE);

By default, the source reader and sink writer can perform some format conversions on uncompressed audio and video streams. To disable this behavior, set this attribute to TRUE when you create the source reader or sink writer.

By default it is FALSE, so you don't really need to explicitly set this attribute.

Then :

hr = mSourceReader->SetCurrentMediaType((DWORD)MF_SOURCE_READER_FIRST_VIDEO_STREAM, NULL, mediaType);

with :

mOutputMediaType->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_NV12);

Also calculate image size according to NV12 format (MF_MT_FRAME_SIZE). Don't set MF_MT_DEFAULT_STRIDE, the SourceReader will do it for you.

So the idea is to get NV12 format from the SourceReader, even if the capture source gives RGB32 format. The SourceReader is normally able to do this.