[FFmpeg-devel] [PATCH v2 2/2] vaapi_encode: fix VBR mode generating low bitrates

[Aman Gupta]

From: Aman Gupta <aman at tmm1.net>

On some Intel chips, sending VAEncMiscParameterTypeRateControl multiple
times causes the encoder to produce very low bitrates that do not match
the requested rate controls.

This is a regression as of af532c921575eb8ee805cc2c64a914f6302442e1. Prior
to that global params were sent only once at the start of the
encoding session.

The broken VBR behavior was observed on:

  - i5-7260U Kaby Lake
  - N3060 CherryView
  - N3710 CherryView
  - J3455 Broxton

/cc intel/intel-vaapi-driver#480

Signed-off-by: Aman Gupta <aman at tmm1.net>
---
 libavcodec/vaapi_encode.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index 231efba6cc..960f8230de 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -236,6 +236,12 @@ static int vaapi_encode_issue(AVCodecContext *avctx,
 
     if (pic->type == PICTURE_TYPE_IDR) {
         for (i = 0; i < ctx->nb_global_params; i++) {
+            if (pic->encode_order != 0 &&
+                ctx->global_params_type[i] == VAEncMiscParameterTypeRateControl &&
+                ctx->va_rc_mode == VA_RC_VBR) {
+                // send VAEncMiscParameterTypeRateControl only once in VBR mode
+                continue;
+            }
             err = vaapi_encode_make_misc_param_buffer(avctx, pic,
                                                       ctx->global_params_type[i],
                                                       ctx->global_params[i],
-- 
2.20.1