[FFmpeg-user] H264 VAAPI Encoder

[Andy Furniss]

Mark Thompson wrote:
> On 18/06/16 16:37, Mark Thompson wrote:
>> On 17/06/16 23:33, Andy Furniss wrote:
>>>
>>> AMD are working on vaapi encode for mesa, only a few patches about so far and they got rejected - though not for functionality.
>>>
>>> They do work with gstreamer, but trying above with ffmpeg fails as below (render node or X) seems the surface is seen as rgb - but the h/w takes nv12.
>>>
>>> Maybe mesa/the driver is giving false info?
>>>
>>> vainfo
>>> libva info: VA-API version 0.38.1
>>> libva info: va_getDriverName() returns 0
>>> libva info: User requested driver 'radeonsi'
>>> libva info: Trying to open /usr/lib/dri/radeonsi_drv_video.so
>>> libva info: Found init function __vaDriverInit_0_38
>>> libva info: va_openDriver() returns 0
>>> vainfo: VA-API version: 0.38 (libva 1.6.3.pre1)
>>> vainfo: Driver version: mesa gallium vaapi
>>> vainfo: Supported profile and entrypoints
>>>        VAProfileMPEG2Simple            : VAEntrypointVLD
>>>        VAProfileMPEG2Main              : VAEntrypointVLD
>>>        VAProfileVC1Simple              : VAEntrypointVLD
>>>        VAProfileVC1Main                : VAEntrypointVLD
>>>        VAProfileVC1Advanced            : VAEntrypointVLD
>>>        VAProfileH264Baseline           : VAEntrypointVLD
>>>        VAProfileH264Baseline           : VAEntrypointEncSlice
>>>        VAProfileH264Baseline           : VAEntrypointEncPicture
>>>        VAProfileH264Main               : VAEntrypointVLD
>>>        VAProfileH264High               : VAEntrypointVLD
>>>        VAProfileNone                   : VAEntrypointVideoProc
>>>
>>> ffmpeg doesn't seem to care about VAProfileH264Baseline
>>>
>>> I tried avconv and it does care = bail, but I can use -profile 66 and then get the same error as ffmpeg (though it converts to rgba rather than rgb0)
>>
>> Default for H264 is High, so yeah you need -profile 66 to get it to work.
>>
>>> [AVHWDeviceContext @ 0x3ca4c40] Format 0x3231564e -> unknown.
>>> [AVHWDeviceContext @ 0x3ca4c40] Format 0x32315659 -> unknown.
>>> [AVHWDeviceContext @ 0x3ca4c40] Format 0x56595559 -> unknown.
>>> [AVHWDeviceContext @ 0x3ca4c40] Format 0x59565955 -> unknown.
>>> [AVHWDeviceContext @ 0x3ca4c40] Format 0x41524742 -> bgra.
>>> [AVHWDeviceContext @ 0x3ca4c40] Format 0x41424752 -> rgba.
>>> [AVHWDeviceContext @ 0x3ca4c40] Format 0x58524742 -> bgr0.
>>> [AVHWDeviceContext @ 0x3ca4c40] Format 0x58424752 -> rgb0.
>>
>> The detection queries the capabilities of a video processor instance (VAProfileNone / VAEntrypointVideoProc) to try to determine which formats are interesting, then ignores the ones which aren't.  It should probably have said "not supported" here rather than "unknown" for nv12 (and yv12 immediately below it).
>>
>> I've sent a patch to change the behaviour slightly which will improve this case, see <https://lists.libav.org/pipermail/libav-devel/2016-June/077669.html>.
>>
>>> [auto-inserted scaler 0 @ 0x3cf2a60] w:720 h:576 fmt:yuv420p sar:1/1 -> w:720 h:576 fmt:nv12 sar:1/1 flags:0x4
>>> [auto-inserted scaler 1 @ 0x3cf4ec0] w:720 h:576 fmt:nv12 sar:1/1 -> w:720 h:576 fmt:rgb0 sar:1/1 flags:0x4
>>> [hwupload @ 0x3cd61c0] Surface format is rgb0.
>>
>> hwupload reports its input capabilities based on what is returned above, so it will be saying that it can only take the RGB formats as input.  Hence ffmpeg auto-inserts a swscale instance to change it back from nv12 to rgba.  (The filter chain becomes 'format=nv12,format=rgba,hwupload'.)
>>
>>> [AVHWFramesContext @ 0x3cf1700] Created surface 0x1.
>>> [AVHWFramesContext @ 0x3cf1700] Direct mapping possible.
>>> [h264_vaapi @ 0x3ce7dc0] Using fixed QP = 20 / 20 / 24 for IDR / P / B frames.
>>> [h264_vaapi @ 0x3ce7dc0] Using (null) as format of reconstructed frames.
>>> [AVHWFramesContext @ 0x3d37fa0] Unsupported format: (null).
>>
>> The input to the encode is now in VAAPI rgba surfaces, but that doesn't match any of the capabilities so it doesn't know what to make the reconstructed frames.  It should have fallen back to the one supported format (i.e. nv12), but a typo stoped it from doing that: fixed in <https://lists.libav.org/pipermail/libav-devel/2016-June/077668.html>.
>>
>>> [h264_vaapi @ 0x3ce7dc0] Failed to initialise reconstructed frame context: -22.
>>> Output #0, matroska, to 'out.mkv':
>>>      Stream #0:0, 0, 0/0: Unknown: none, SAR 1:1 DAR 0:0
>>>      Metadata:
>>>        encoder         : Lavc57.46.100 h264_vaapi
>>> Stream mapping:
>>>    Stream #0:0 -> #0:0 (mpeg2video (native) -> h264 (h264_vaapi))
>>> Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
>>> [AVIOContext @ 0x3ce8760] Statistics: 0 seeks, 0 writeouts
>>> [AVIOContext @ 0x3bfa540] Statistics: 2019472 bytes read, 2 seeks
>>
>> So, I also went and built mesa with the encode patches and had a go.  With the changes above, it does get to actually trying to encode, but that nuked my GPU to the point of requiring reboot.
>>
>> With the linked patch you should be able to have a go too: if it works for you then I'm interested in what your setup is.  If it nukes your GPU then, well, you had some warning at least.  I think I'll wait until it's a bit more stable before pursuing this further.
>
> Ok, I tried a bit more (including a few power cycles), and it does work.  The critical extra step is to disable B-frames (that should probably happen automatically for baseline profile).  It then works with input uploaded from the CPU or from a decode on the GPU.
>
> For example: "./avconv -v 55 -y -vaapi_device /dev/dri/renderD129 -hwaccel vaapi -hwaccel_output_format vaapi -i in.mp4 -an -vf 'format=nv12|vaapi,hwupload' -c:v h264_vaapi -profile 66 -bf 0 -b 10M out.mp4".
>
> The bitstream it outputs does work in ffmpeg, but it's very weird.  It has CABAC enabled (not allowed in baseline), and the POC seems to only advance on every second frame (as if they were actually fields of an interlaced stream, except it isn't).
>
> So, yeah.  I think there is a lot more work to do before this is generally usable, but it is certainly working and I will keep testing it as more versions turn up.
>
> Thanks,

Ahh, good - I hadn't seen this mail before preparing/sending the other 
one :-(

The cabac thing is interesting - I have previously noticed that mediainfo
showed gstreamer vaapi as constrained but yes for cabac - but it also 
said 2 for ref frames when ffprobe called one.

gstreamer omx files said no to cabac - I failed to find a way with 
ffmpeg/ffprobe to see if cabac was off/on - is there one??

Also windows files get called as high, but I don't see any b frames
though the only windows files I have were made with a game recording app 
that comes with the driver (raptr).