[FFmpeg-devel] [PATCH] lavc/vaapi_encode_h265: fix max_transform_hierarchy_depth_inter/intra

[Fu, Linjie]

> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Mark Thompson
> Sent: Monday, April 13, 2020 20:20
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] [PATCH] lavc/vaapi_encode_h265: fix
> max_transform_hierarchy_depth_inter/intra
> 
> On 13/04/2020 05:32, Linjie Fu wrote:
> > Set the max_transform_hierarchy_depth_inter/intra to 2 by default
> > based on the Programmer's Reference Manuals (PRM) in [1].
> >
> > Intel Encoder only supports 2 levels of quad-tree. That is:
> >     - max_transform_hierarchy_depth_inter/intra <= 2.
> >
> > [1] <https://01.org/sites/default/files/documentation/intel-gfx-prm-osrc-
> kbl-vol10-hevc.pdf>
> >
> > Signed-off-by: Linjie Fu <linjie.fu at intel.com>
> > ---
> > Fixed value for intel platform, makes more sense on TGL+ platform.
> > (If conflict with other driver capability, we may add query support
> >  later)
> >  libavcodec/vaapi_encode_h265.c | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> >
> > diff --git a/libavcodec/vaapi_encode_h265.c
> b/libavcodec/vaapi_encode_h265.c
> > index cd48545..d6cb82a 100644
> > --- a/libavcodec/vaapi_encode_h265.c
> > +++ b/libavcodec/vaapi_encode_h265.c
> > @@ -445,8 +445,9 @@ static int
> vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx)
> >      sps->log2_min_luma_transform_block_size_minus2   = 0;
> >      sps->log2_diff_max_min_luma_transform_block_size = 3;
> >      // Full transform hierarchy allowed (2-5).
> > -    sps->max_transform_hierarchy_depth_inter = 3;
> > -    sps->max_transform_hierarchy_depth_intra = 3;
> > +    // Default to 2 based on Programmer's Reference Manuals of Intel
> graphics
> > +    sps->max_transform_hierarchy_depth_inter = 2;
> > +    sps->max_transform_hierarchy_depth_intra = 2;
> >      // AMP works.
> >      sps->amp_enabled_flag = 1;
> >      // SAO and temporal MVP do not work.
> >
> 
> I don't much like the idea of changing this based on a value in a Kaby Lake
> document given that the current value hasn't had any problems on Kaby Lake.
> Can you explain the benefits of changing this?  

It fixes the encoding issue for HEVC on gen12+ platform.
We didn't notice this either, until it triggers gpu hang for encoding on Tiger Lake (gen12).

> Can you confirm that it continues to work on all the other currently-working platforms?
> 
Yes, we set up CI and runs full-round tests for conformance and catch regression on different
platforms(CFL/CML/ICL/KBL/SKL).

Identical fix has been merged in gstreamer-vaapi, no regression is detected on other platforms.
https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/282/diffs?commit_id=17d82e14e78af901f1cd7f2344e173ad6ae6a8a6

- Linjie