[FFmpeg-devel] [PATCH] lavc/vaapi_encode_h265: fix max_transform_hierarchy_depth_inter/intra

[Mark Thompson]

On 13/04/2020 05:32, Linjie Fu wrote:
> Set the max_transform_hierarchy_depth_inter/intra to 2 by default
> based on the Programmer's Reference Manuals (PRM) in [1].
> 
> Intel Encoder only supports 2 levels of quad-tree. That is:
>     - max_transform_hierarchy_depth_inter/intra <= 2.
> 
> [1] <https://01.org/sites/default/files/documentation/intel-gfx-prm-osrc-kbl-vol10-hevc.pdf>
> 
> Signed-off-by: Linjie Fu <linjie.fu at intel.com>
> ---
> Fixed value for intel platform, makes more sense on TGL+ platform.
> (If conflict with other driver capability, we may add query support
>  later)
>  libavcodec/vaapi_encode_h265.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c
> index cd48545..d6cb82a 100644
> --- a/libavcodec/vaapi_encode_h265.c
> +++ b/libavcodec/vaapi_encode_h265.c
> @@ -445,8 +445,9 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx)
>      sps->log2_min_luma_transform_block_size_minus2   = 0;
>      sps->log2_diff_max_min_luma_transform_block_size = 3;
>      // Full transform hierarchy allowed (2-5).
> -    sps->max_transform_hierarchy_depth_inter = 3;
> -    sps->max_transform_hierarchy_depth_intra = 3;
> +    // Default to 2 based on Programmer's Reference Manuals of Intel graphics
> +    sps->max_transform_hierarchy_depth_inter = 2;
> +    sps->max_transform_hierarchy_depth_intra = 2;
>      // AMP works.
>      sps->amp_enabled_flag = 1;
>      // SAO and temporal MVP do not work.
> 

I don't much like the idea of changing this based on a value in a Kaby Lake document given that the current value hasn't had any problems on Kaby Lake.  Can you explain the benefits of changing this?  Can you confirm that it continues to work on all the other currently-working platforms?

- Mark