[FFmpeg-devel] [PATCH] lavc/vaapi_h26[45]: add crop info support in vaapi_h26[4, 5]

[Mark Thompson]

On 30/11/16 02:20, Jun Zhao wrote:
> From 20bedd18213420c77d5e8a26fbe741d8d204ac10 Mon Sep 17 00:00:00 2001
> From: Jun Zhao <jun.zhao at intel.com>
> Date: Tue, 29 Nov 2016 14:14:25 +0800
> Subject: [PATCH] lavc/vaapi_h26[45]: add crop info support in vaapi_h26[4,5]
>
> add crop information support in vaapi_h26[4,5] hwaccel decode,
> and align h264/hevc software decoder. After this fix, FATE test
> h264-conformance-cvfc1_sony_c/hevc-conformance-CONFWIN_A_Sony_1
> will pass in i965/SKL.
>
> Signed-off-by: Wang, Yi A <yi.a.wang at intel.com>
> Signed-off-by: Jun Zhao <jun.zhao at intel.com>
> ---
>  libavcodec/h264dec.c        | 13 +++++++++++++
>  libavcodec/hevc_refs.c      |  7 +++++++
>  libavutil/hwcontext.h       |  6 ++++++
>  libavutil/hwcontext_vaapi.c |  1 +
>  4 files changed, 27 insertions(+)

No.  Top/left cropping needs a more general approach, not hacks in specific decoders/hwcontexts.

Among other things:
- The hardware frame context is owned by the user, you can't edit it like this from inside the decoder.
- It can't be per-frame-context anyway, because multiple streams can be decoded into the same frame context.
- More components need to be aware of it: the scale filters definitely should be (at <https://git.libav.org/?p=libav.git;a=blob;f=libavfilter/vf_scale_vaapi.c;h=50be68eee07374a78bba1eed8850c3ab9e019fe6;hb=HEAD#l291>, in VAAPI, similarly for the other hardware APIs).  Other components can probably ignore it (with docs saying you should put it through the relevant scale filter if you want correctly-cropped images), but we should make that decision explicitly.
- Messing with pointers/sizes on frame upload/download has issues with APIs which can only transfer whole frames - see the comment at <http://git.videolan.org/?p=ffmpeg.git;a=blob;f=libavutil/hwcontext.h;h=785da090b3fc4b4e91f989370ea4606c79bae823;hb=HEAD#l321>.  (That was written with VDPAU in mind, there are likely others.)

As wm4 notes in another reply, this ends up pointing to some sort of cropping information associated with the AVFrame which all interested components can then read.

I know several people have expressed interest in this problem: given that, it's probably a good idea to have some discussion first and decide roughly what the result should look like before writing a lot of code for it.

Thanks,

- Mark