[FFmpeg-devel] [PATCH] avcodec/cuviddec: fix AV1 decoding error

[James Almer]

On 6/24/2022 1:04 AM, Zhao Zhili wrote:
> From: Zhao Zhili <zhilizhao at tencent.com>
> 
> cuvidParseVideoData only supports pure OBUs, it report unknown
> error with AV1CodecConfigurationRecord. Check whether extradata
> is AV1CodecConfigurationRecord and skip the first 4 bytes to fix
> the issue.
> 
> The bug is revealed in ffmpeg cmd since 45e3b6a68 and ffd1316e.
> ---
>   libavcodec/cuviddec.c | 10 ++++++++++
>   1 file changed, 10 insertions(+)
> 
> diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c
> index cb3cda7e24..fc9c0515fe 100644
> --- a/libavcodec/cuviddec.c
> +++ b/libavcodec/cuviddec.c
> @@ -953,6 +953,16 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx)
>           extradata_size = avctx->extradata_size;
>       }
>   
> +    // Check first bit to determine whether it's AV1CodecConfigurationRecord.
> +    // Skip first 4 bytes of AV1CodecConfigurationRecord to keep configOBUs
> +    // only, otherwise cuvidParseVideoData report unknown error.
> +    if (avctx->codec->id == AV_CODEC_ID_AV1 &&
> +            extradata_size > 4 &&
> +            extradata && extradata[0] & 0x80) {

IMO the check for extradata here is unnecessary as the memcpy 
dereferencing it below will crash anyway if it were NULL.

If anything, a check like (avctx->extradata_size || !avctx->extradata) 
should be added to avcodec_open2() if there isn't one already to ensure 
the user set up valid extradata when opening a decoder.

> +        extradata += 4;
> +        extradata_size -= 4;
> +    }
> +
>       ctx->cuparse_ext = av_mallocz(sizeof(*ctx->cuparse_ext)
>               + FFMAX(extradata_size - (int)sizeof(ctx->cuparse_ext->raw_seqhdr_data), 0));
>       if (!ctx->cuparse_ext) {

LGTM either way.