[Libav-user] VA-API decoding on Android [ffmpeg-2.1]

Mark Thompson sw at jkqxz.net
Sun Nov 20 17:43:13 EET 2016 

On Fri, Nov 18, 2016 at 9:50 PM, Michael Goffioul <michael.goffioul at gmail.com <mailto:michael.goffioul at gmail.com>> wrote:
> 
>     Hi,
> 
>     I'm investigating the possibility to use VA-API for H264 decoding on Android, using Android-x86. At the moment, I'm using Android-4.4-r5 (from android-x86), which is based on ffmpeg-2.1. I realize this is not the current ffmpeg version, nervertheless I understand that 2.1 should support VA-API decoding.
> 
>     I've recompiled libavcodec with support for libva (1.7.1) and I've added the intel driver (i965). Running vainfo on the device yields the following, so I assume everything is ok on that front:

Probably not:

> 
>     vainfo: VA-API version: 0.39 ...

This is saying that the libva you linked to has API version 0.39 (i.e. it is libva 1.7.1).

>                              ... (libva 1.1.0)

While this says that the headers you have built against are from libva 1.1.0.

>     vainfo: Driver version: Intel i965 driver for Intel(R) Bay Trail - 1.0.16.pre1 ()

And this says that the driver loaded at run time is similarly old.

>     vainfo: Supported profile and entrypoints
>           VAProfileMPEG2Simple            :VAEntrypointVLD
>           VAProfileMPEG2Simple            :VAEntrypointEncSlice
>           VAProfileMPEG2Main              :VAEntrypointVLD
>           VAProfileMPEG2Main              :VAEntrypointEncSlice
>           VAProfileH264ConstrainedBaseline:VAEntrypointVLD
>           VAProfileH264ConstrainedBaseline:VAEntrypointEncSlice
>           VAProfileH264Main               :VAEntrypointVLD
>           VAProfileH264Main               :VAEntrypointEncSlice
>           VAProfileH264High               :VAEntrypointVLD
>           VAProfileH264High               :VAEntrypointEncSlice

So old, in fact, that it shouldn't support encoding at all - so these entrypoints shouldn't be here, and the fact that they are suggests something very strange is going on.

I suggest that you fix your libva and i965 driver installation before proceeding with anything else.

>           VAProfileH264StereoHigh         :VAEntrypointVLD
>           VAProfileVC1Simple              :VAEntrypointVLD
>           VAProfileVC1Main                :VAEntrypointVLD
>           VAProfileVC1Advanced            :VAEntrypointVLD
>           VAProfileNone                   :VAEntrypointVideoProc
>           VAProfileJPEGBaseline           :VAEntrypointVLD
> 
>     My problem is that it seems ffmpeg's h264 decoder never uses VA-API. I've added log statements at various places to trace the code execution:
>     https://github.com/FFmpeg/FFmpeg/blob/release/2.1/libavcodec/h264.c#L3270 <https://github.com/FFmpeg/FFmpeg/blob/release/2.1/libavcodec/h264.c#L3270>
>     https://github.com/FFmpeg/FFmpeg/blob/release/2.1/libavcodec/h264.c#L3200 <https://github.com/FFmpeg/FFmpeg/blob/release/2.1/libavcodec/h264.c#L3200>
>     https://github.com/FFmpeg/FFmpeg/blob/release/2.1/libavcodec/utils.c#L3204 <https://github.com/FFmpeg/FFmpeg/blob/release/2.1/libavcodec/utils.c#L3204>
>     https://github.com/FFmpeg/FFmpeg/blob/release/2.1/libavcodec/utils.c#L3218 <https://github.com/FFmpeg/FFmpeg/blob/release/2.1/libavcodec/utils.c#L3218>
> 
>     The log I see is below. It appears ff_find_hwaccel returns NULL, as it tries to match the pix_fmt. The given pix_fmt is AV_PIX_FMT_YUV420P, and it should be AV_PIX_FMT_VAAPI_VLD for the hwaccel object to be detected. And I can't figure out how the pix_fmt could be set to AV_PIX_FMT_VAAPI_VLD.
> 
>     Any pointer or suggestion?
> 
>     I/FFMPEG  ( 2708): register hwaccel: h264_vaapi
>     I/FFMPEG  ( 2708): register hwaccel: mpeg2_vaapi
>     I/FFMPEG  ( 2708): register hwaccel: mpeg4_vaapi
>     D/FFMPEG  ( 2708): android source begin open
>     D/FFMPEG  ( 2708): android open, url: android-source:0x416b9df0
>     D/FFMPEG  ( 2708): ffmpeg open android data source success, source ptr: 0x416b9df0
>     D/FFMPEG  ( 2708): android source open success
>     I/FFMPEG  ( 2708): get_pixel_format: i=0, pix_fmt=-1, fmt=53, force_callback=0
>     I/FFMPEG  ( 2708): get_pixel_format: i=1, pix_fmt=-1, fmt=0, force_callback=0
>     I/FFMPEG  ( 2708): get_pixel_format: i=0, pix_fmt=-1, fmt=53, force_callback=1
>     I/FFMPEG  ( 2708): get_pixel_format: i=1, pix_fmt=-1, fmt=0, force_callback=1
>     I/FFMPEG  ( 2708): ff_find_hwaccel: codec=0000001c, pix_fmt=0
>     I/FFMPEG  ( 2708): ff_find_hwaccel: hwaccel=h264_vaapi
>     I/FFMPEG  ( 2708): ff_find_hwaccel: hwaccel=mpeg2_vaapi
>     I/FFMPEG  ( 2708): ff_find_hwaccel: hwaccel=mpeg4_vaapi
>     I/FFMPEG  ( 2708): [h264 @ 0x42dfdd00] hwaccel: (null)
>     I/FFMPEG  ( 2708): Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'android-source:0x416b9df0':
>     I/FFMPEG  ( 2708):   Metadata:
>     I/FFMPEG  ( 2708):     major_brand     : mp42
>     I/FFMPEG  ( 2708):     minor_version   : 0
>     I/FFMPEG  ( 2708):     compatible_brands: isomavc1mp42
>     I/FFMPEG  ( 2708):     creation_time   : 2010-06-07 20:19:05
>     I/FFMPEG  ( 2708):   Duration: 00:04:06.71, start: 0.000000, bitrate: 3606 kb/s
>     I/FFMPEG  ( 2708):     Stream #0:0(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
>     I/FFMPEG  ( 2708):     Metadata:
>     I/FFMPEG  ( 2708):       creation_time   : 2010-06-07 20:19:05
>     I/FFMPEG  ( 2708):       handler_name    : (C) 2007 Google Inc. v08.13.2007.
>     I/FFMPEG  ( 2708):     Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 3478 kb/s, 24 fps, 24 tbr, 24k tbn, 48 tbc (default)
>     I/FFMPEG  ( 2708):     Metadata:
>     I/FFMPEG  ( 2708):       creation_time   : 2010-06-07 20:19:05
>     I/FFMPEG  ( 2708):       handler_name    : (C) 2007 Google Inc. v08.13.2007.
>     D/FFMPEG  ( 2708): android source close
>     I/FFMPEG  ( 2708): get_pixel_format: i=0, pix_fmt=-1, fmt=53, force_callback=0
>     I/FFMPEG  ( 2708): get_pixel_format: i=1, pix_fmt=-1, fmt=0, force_callback=0
>     I/FFMPEG  ( 2708): get_pixel_format: i=0, pix_fmt=-1, fmt=53, force_callback=1
>     I/FFMPEG  ( 2708): get_pixel_format: i=1, pix_fmt=-1, fmt=0, force_callback=1
>     I/FFMPEG  ( 2708): ff_find_hwaccel: codec=0000001c, pix_fmt=0
>     I/FFMPEG  ( 2708): ff_find_hwaccel: hwaccel=h264_vaapi
>     I/FFMPEG  ( 2708): ff_find_hwaccel: hwaccel=mpeg2_vaapi
>     I/FFMPEG  ( 2708): ff_find_hwaccel: hwaccel=mpeg4_vaapi
>     I/FFMPEG  ( 2708): [h264 @ 0x42dfe320] hwaccel: (null)
> 

On 20/11/16 01:20, Michael Goffioul wrote:
> Basically, a little HOWTO use VA-API in ffmpeg 2.1 would be quite handy. Does that exist somewhere?

Not really.  The usual example for this sort of thing is the support in the ffmpeg utility itself, but the version you have predates that (it existed only in libavcodec for quite a while).

You could look at the current support in ffmpeg <http://git.videolan.org/?p=ffmpeg.git;a=blob;f=ffmpeg_vaapi.c;hb=HEAD>, but that uses newer API (hwcontext) around all of the frame setup, so you would need to write those parts as well.

The media players also make use of it, so there may be some hints there: mpv <https://github.com/mpv-player/mpv/blob/master/video/decode/vaapi.c> or vlc <http://git.videolan.org/?p=vlc.git;a=blob;f=modules/codec/avcodec/vaapi.c;hb=HEAD>.

If it is possible, I think upgrading to a newer version of ffmpeg would be a good idea.

- Mark

More information about the Libav-user mailing list