Playing HW accelerated video in VLC using gstreamer

Question

I have a board with Freescale i.MX.6 processor. There is a hardware acceleration for H264 video. There is also gstreamer that is able to cooperate with the hardware accelerator.

I need to write an application that beside other things is able to play Full HD videos. As a player I have chosen VLC because of its options and reliable way to handle it programatically (libvlc and libvlc-qt libraries).

VLC is not able to cooperate with the hardware accelerator, therefore is there a way to play video using gstreamer (that use the accelerator) and send the output video into VLC?

Thanks for your time. Martin.

Answer 1

The Freescale VPU can certainly be used with GStreamer. Freescale developed GStreamer 0.10 plugins. I am writing some for 1.0.

You want to decode with GStreamer and transmit the decoded frames to VLC for displaying them. You could use RTP for that. But what you intend to do will not work well for FullHD videos, since it involves implicitely copying the frames at least once.

The VPU decodes to a DMA buffer (= a physically contiguous memory region, allocated with special allocation functions). This buffer could be passed on to libvlc without copying it, but even then, you'd need to add something inside VLC to display the frames without having to (implicitely) copy them. Without this VLC extension, you cannot watch FullHD.

Copy vs. no copy makes an immense difference. With copy, you'll see CPU usage of >80% with 720p, and 1080p will not be displayed smoothly. Without the copy, CPU usage is insignificant (< 10%). The i.MX6 has the ability to display frames directly from physically contiguous buffers, over DMA, without having to shove the pixels through the CPU. This is what you need. There are three ways to do this: the V4L2 output modified by Freescale; displaying to the Linux framebuffer directly using the i.MX IPU; and displaying using OpenGL ES 2 and the Vivante direct texture extension. The third is the best option.

But to be honest, I would either try to put both the VPU decoding and GLES direct texture output into VLC, or use GStreamer for both. Having VLC and GStreamer in the same application sounds like a recipe for frustration. (I assume you want to have one process with VLC and GStreamer inside; two processes would be considerably trickier to get working.)