I have a simple C++ application that uses FFmpeg 3.2 to receive an H264 RTP stream. In order to save CPU, I'm doing the decoding part with the codec h264_cuvid. My FFmpeg 3.2 is compiled with hw acceleration enabled. In fact, if I do the command:
ffmpeg -hwaccels
I get
cuvid
This means that my FFmpeg setup has everything OK to "speak" with my NVIDIA card.
The frames that the function avcodec_decode_video2
provides me have the pixel format AV_PIX_FMT_CUDA
. I need to convert those frames to new ones with AV_PIX_FMT_RGB
. Unfortunately, I can't do the conversion using the well knwon functions sws_getContext
and sws_scale
because the pixel format AV_PIX_FMT_CUDA
is not supported. If I try with swscale I get the error:
"cuda is not supported as input pixel format"
Do you know how to convert an FFmpeg AVFrame
from AV_PIX_FMT_CUDA
to AV_PIX_FMT_RGB
?
(pieces of code would be very appreciated)
This my understanding of the hardware decoding on the latest FFMPeg 4.1 version. Below are my conclusion after studying the source code.
First I recommend to inspire yourself from the hw_decode example:
https://github.com/FFmpeg/FFmpeg/blob/release/4.1/doc/examples/hw_decode.c
With the new API, when you send a packet to the encoder using avcodec_send_packet(), then use avcodec_receive_frame() to retrieve the decoded frame.
There are two different kinds of
AVFrame
: software one, which is stored in the "CPU" memory (a.k.a RAM), and hardware one, which is stored in the graphic card memory.Getting AVFrame from the hardware
To retrieve the hardware frame and get it into a readable, convertible (with swscaler)
AVFrame
, av_hwframe_transfer_data() needs to be used to retrieve the data from the graphic card. Then look at the pixel format of the retrieved frame, it is usually NV12 format when using nVidia decoding.Listing supported "software" pixel formats
Side note here if you want to select the pixel format, not all AVPixelFormat are supported. AVHWFramesConstraints is your friend here:
Finally, a quicker way is probably the av_hwframe_transfer_get_formats() function, but you need to decode at least one frame.
Hope this will help!