[FFmpeg-devel] [PATCH] libavcodec/qsvdec: use the param from decodeHeader to configure surface

[Xiang, Haihao]

On Mon, 2021-03-22 at 14:31 +0800, wenbin.chen at intel.com wrote:
> From: "Chen,Wenbin" <wenbin.chen at intel.com>
> 
> MSDK recognizes both yuv420p10 and yuv420p9 as MFX_FOURCC_P010, but param
> are different. When decode yuv420p9 video, ffmpeg-qsv will use
> yuv420p10le to configure surface which is different with param from
> DecoderHeader and this will lead to error. Now change it use
> param from decoderHeader to configure surface.


Both yuv420p10 and yuv420p9 have 3 planes (
https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/pixdesc.c#L1359-L1406) ,
but MFX_FOURCC_P010 has 2 planes only. MSDK doesn't recognizes yuv420p10 and
yuv420p9 as MFX_FOURCC_P010. FFmpeg-qsv uses p010le instead of yuv420p10le to
configure surface for 10bit video.


> 
> Signed-off-by Wenbin Chen <wenbin.chen at intel.com>
> ---
>  libavcodec/qsvdec.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/qsvdec.c b/libavcodec/qsvdec.c
> index 569ccd4fba..3ab48ea7a2 100644
> --- a/libavcodec/qsvdec.c
> +++ b/libavcodec/qsvdec.c
> @@ -309,13 +309,13 @@ static int alloc_frame(AVCodecContext *avctx, QSVContext
> *q, QSVFrame *frame)
>      if (frame->frame->format == AV_PIX_FMT_QSV) {
>          frame->surface = *(mfxFrameSurface1*)frame->frame->data[3];
>      } else {
> -        frame->surface.Info = q->frame_info;
> -
>          frame->surface.Data.PitchLow = frame->frame->linesize[0];
>          frame->surface.Data.Y        = frame->frame->data[0];
>          frame->surface.Data.UV       = frame->frame->data[1];
>      }
>  
> +    frame->surface.Info = q->frame_info;
> +
>      if (q->frames_ctx.mids) {
>          ret = ff_qsv_find_surface_idx(&q->frames_ctx, frame);
>          if (ret < 0)