[Libav-user] Reduce latency in av_hwframe_transfer_data

[Michael Goffioul]

HI,

I'm using FFMPEG (5.1.2) on an Android (x86) platform running on an Intel
Baytrail chipset. Video decoding uses VA-API for mpeg2 and h264. The
implementation follows the same pattern as the hw_decode.c example.

Everything works fine, but when timing the various parts of the decoding
loop (send packet, receive frame, hw-transfer, scale/pixfmt conversion), it
appears that most of the time is spent in av_hwframe_transfer_data. E.g.
these are typical numbers for various resolution:
- 1280x720: ~10ms
- 1920x1080: ~20ms
- 3840x1714: ~40ms

My question is then whether there's anything I do or look into in order to
reduce that latency? Is it at all possible, or is it just the way it is and
I can't squeeze more performance out of that platform?

Not sure whether it's helpful, but here's some VA-API related log output
from FFMPEG:
10-06 22:18:00.162  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Opened VA display via Android device android.
10-06 22:18:00.162  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] libva: VA-API version 1.4.0
10-06 22:18:00.162  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] libva: va_getDriverName() returns 0
10-06 22:18:00.162  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] libva: Trying to open /vendor/lib64/dri/i965_drv_video.so
10-06 22:18:00.163  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] libva: Found init function __vaDriverInit_1_4
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] libva: va_openDriver() returns 0
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Initialised VAAPI connection: version 1.4
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Format 0x32315659 -> yuv420p.
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Format 0x30323449 -> yuv420p.
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Format 0x3231564e -> nv12.
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Format 0x32595559 -> yuyv422.
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Format 0x59565955 -> uyvy422.
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Format 0x48323234 -> yuv422p.
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Format 0x58424752 -> rgb0.
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Format 0x58524742 -> bgr0.
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Format 0x30313050 -> p010le.
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] VAAPI driver: Intel i965 driver for Intel(R) Bay Trail -
2.4.0.pre1 (a9d2c1f).
10-06 22:18:00.165  3805  3853 I FFMPEG  : [AVHWDeviceContext @
0x760040312900] Driver not found in known nonstandard list, using standard
behaviour.
10-06 22:18:00.165  3805  3853 I HWACCEL : hw codec h264 enabled:
s=0x7600e0316700 pix_fmts=44
10-06 22:18:00.165  3805  3853 I FFMPEG  : [h264 @ 0x7600e0316700] Format
vaapi chosen by get_format().
10-06 22:18:00.165  3805  3853 I FFMPEG  : [h264 @ 0x7600e0316700] Format
vaapi requires hwaccel initialisation.
10-06 22:18:00.165  3805  3853 I FFMPEG  : [h264 @ 0x7600e0316700]
Considering format 0x3231564e -> nv12.
10-06 22:18:00.166  3805  3853 I FFMPEG  : [h264 @ 0x7600e0316700] Picked
nv12 (0x3231564e) as best match for yuv420p.
10-06 22:18:00.166  3805  3853 I FFMPEG  : [AVHWFramesContext @
0x760060312bc0] Created surface 0x4000000.
10-06 22:18:00.166  3805  3853 I FFMPEG  : [AVHWFramesContext @
0x760060312bc0] Direct mapping disabled: derived image format 3231564e does
not match expected format 32315659.
10-06 22:18:00.166  3805  3853 I FFMPEG  : [AVHWFramesContext @
0x760060312bc0] Created surface 0x4000001.
10-06 22:18:00.166  3805  3853 I FFMPEG  : [AVHWFramesContext @
0x760060312bc0] Created surface 0x4000002.
10-06 22:18:00.166  3805  3853 I FFMPEG  : [AVHWFramesContext @
0x760060312bc0] Created surface 0x4000003.
10-06 22:18:00.166  3805  3853 I FFMPEG  : [AVHWFramesContext @
0x760060312bc0] Created surface 0x4000004.
10-06 22:18:00.166  3805  3853 I FFMPEG  : [AVHWFramesContext @
0x760060312bc0] Created surface 0x4000005.
10-06 22:18:00.166  3805  3853 I FFMPEG  : [AVHWFramesContext @
0x760060312bc0] Created surface 0x4000006.
10-06 22:18:00.166  3805  3853 I FFMPEG  : [AVHWFramesContext @
0x760060312bc0] Created surface 0x4000007.

Thanks,
Michael.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20221006/26301a5a/attachment.htm>