[Libav-user] Hardware accelerated decoding

[wm4]

On Thu, 5 Sep 2013 18:06:00 +0200
Lucas Soltic <lucas.soltic at orange.fr> wrote:

> What is "h264 + hwaccel" ?

The "h264" decoder, using the hwaccel mechanism. The h264 decoder can
do both software and hardware decoding. It works by overriding the
get_format callback. When libavcodec calls your get_format callback,
you request a certain hwaccel pixel format (such as AV_PIX_FMT_VDA_VLD
in the case of VDA), which puts the decoder into hwaccel mode. Then,
the decoder will return AVFrames with that pixel format, and
AVFrame.data[3] will contain a reference to a surface, using the
hwaccel specific type (CVPixelBufferRef in case of VDA). With most
hwaccel types, you also need to override get_buffer2 to allocate a
surface using external API (VDA is an exception).

It's a quite messy API, with no examples, help, or anything around.
You'll have to look at the source code of video players which support
it. Especially the VDA part's memory management is extremely messy, and
there are differences between various ffmpeg and Libav versions.

> What is the difference between h264_vda and h264_vdpau that are listed among FFmpeg's decoders?

Both are outdated APIs. h264_vda was created only because a developer
misunderstood how VDA works. h264_vdpau is rather similar to
h264 + vdpau hwaccel, and in particular it isn't anything like h264_vda.
You should use the hwaccel API instead.

Don't let the similarity of the decoder names confuse you.

> And... is H264 VDA the only hwaccel that doesn't require many modifications to be used? (because of how it works => copying the data back to RAM)

h264_vda pretends to be a normal software decoder, and works just like
one. It doesn't require anything special; even ffplay could use it. The
only good thing about this decoder is that it's very easy to use. But
it appears it barely provides any advantages over software decoding due
to the high overhead of copying back the video data to RAM.

The actual hwaccel decoders are much harder to use; partially this is
because the interaction between the decoder and the platform specific
API (like VDA, vdpau, vaapi, DXVA) is complicated, in part because
ffmpeg's hardware decoding support is very low level, and in part
because of chaotic API design.