[FFmpeg-devel] [PATCH] lavc/vaapi_encode_h265: fix conf_win_xxx_offset for 4:2:2/4:4:4 encoding

[Linjie Fu]

Use desc->log2_chroma_w/h to calculate the sps->conf_win_right/bottom_offset.

Based on Table 6-1, SubWidthC and SubHeightC depend on chroma format(log2_chroma_w/h).

Based on D-28 and D-29, set the correct cropped width/height.

croppedWidth  = pic_width_in_luma_samples −
                SubWidthC * ( conf_win_right_offset + conf_win_left_offset );

croppedHeight = pic_height_in_luma_samples −
                SubHeightC * ( conf_win_bottom_offset + conf_win_top_offset );

Signed-off-by: Linjie Fu <linjie.fu at intel.com>
---
[v2]: use desc->log2_chroma_w/h.

 libavcodec/vaapi_encode_h265.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c
index ace7696..bebb0f9 100644
--- a/libavcodec/vaapi_encode_h265.c
+++ b/libavcodec/vaapi_encode_h265.c
@@ -410,10 +410,10 @@ static int vaapi_encode_h265_init_sequence_params(AVCodecContext *avctx)
         sps->conformance_window_flag = 1;
         sps->conf_win_left_offset   = 0;
         sps->conf_win_right_offset  =
-            (ctx->surface_width - avctx->width) / 2;
+            (ctx->surface_width - avctx->width) >> desc->log2_chroma_w;
         sps->conf_win_top_offset    = 0;
         sps->conf_win_bottom_offset =
-            (ctx->surface_height - avctx->height) / 2;
+            (ctx->surface_height - avctx->height) >> desc->log2_chroma_h;
     } else {
         sps->conformance_window_flag = 0;
     }
-- 
2.7.4