[FFmpeg-devel] [PATCH] avcodec/encode: restructure the core encoding code

[Fu, Linjie]

> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> James Almer
> Sent: Tuesday, May 26, 2020 22:36
> To: ffmpeg-devel at ffmpeg.org
> Subject: [FFmpeg-devel] [PATCH] avcodec/encode: restructure the core
> encoding code
> 
> This commit follows the same logic as 061a0c14bb, but for the encode API:
> The
> new public encoding API will no longer be a wrapper around the old
> deprecated
> one, and the internal API used by the encoders now consists of a single
> receive_packet() callback that pulls frames as required.
> 
> amf encoders adapted by James Almer
> librav1e encoder adapted by James Almer
> nvidia encoders adapted by James Almer
> MediaFoundation encoders adapted by James Almer
> vaapi encoders adapted by Linjie Fu
> v4l2_m2m encoders adapted by Andriy Gelman
> 
> Signed-off-by: James Almer <jamrial at gmail.com>
> ---
> 
> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> index cb05ebd774..98026f451a 100644
> --- a/libavcodec/vaapi_encode.c
> +++ b/libavcodec/vaapi_encode.c
> @@ -25,6 +25,7 @@
>  #include "libavutil/pixdesc.h"
> 
>  #include "vaapi_encode.h"
> +#include "encode.h"
>  #include "avcodec.h"
> 
>  const AVCodecHWConfigInternal *ff_vaapi_encode_hw_configs[] = {
> @@ -1043,7 +1044,7 @@ static int
> vaapi_encode_check_frame(AVCodecContext *avctx,
>      return 0;
>  }
> 
> -int ff_vaapi_encode_send_frame(AVCodecContext *avctx, const AVFrame
> *frame)
> +static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame
> *frame)
>  {
>      VAAPIEncodeContext *ctx = avctx->priv_data;
>      VAAPIEncodePicture *pic;
> @@ -1066,9 +1067,7 @@ int
> ff_vaapi_encode_send_frame(AVCodecContext *avctx, const AVFrame
> *frame)
>              err = AVERROR(ENOMEM);
>              goto fail;
>          }
> -        err = av_frame_ref(pic->input_image, frame);
> -        if (err < 0)
> -            goto fail;
> +        av_frame_move_ref(pic->input_image, frame);

Slight change for the vaapi part is needed here:

av_frame_move_ref() would reset src(frame), hence we need to get
all attributes passed from frame to pic before we call av_frame_move_ref().

Didn't notice this until encoding quality drop is observed in pre-check[1] for
this patch set while verifying. (and sorry for didn't catch this earlier)

After adding this fix, all pre-check is good. 

- Linjie

[1] https://github.com/intel-media-ci/ffmpeg/pull/206

Details: 
diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
index 98026f451a..e39db20200 100644
--- a/libavcodec/vaapi_encode.c
+++ b/libavcodec/vaapi_encode.c
@@ -1067,7 +1067,6 @@ static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame)
             err = AVERROR(ENOMEM);
             goto fail;
         }
-        av_frame_move_ref(pic->input_image, frame);

         if (ctx->input_order == 0 || frame->pict_type == AV_PICTURE_TYPE_I)
             pic->force_idr = 1;
@@ -1075,6 +1074,8 @@ static int vaapi_encode_send_frame(AVCodecContext *avctx, AVFrame *frame)
         pic->input_surface = (VASurfaceID)(uintptr_t)frame->data[3];
         pic->pts = frame->pts;

+        av_frame_move_ref(pic->input_image, frame);
+
         if (ctx->input_order == 0)
             ctx->first_pts = pic->pts;
         if (ctx->input_order == ctx->decode_delay)
--