[FFmpeg-cvslog] avcodec/cuviddec: fix AV1 decoding error

[Zhao Zhili]

ffmpeg | branch: master | Zhao Zhili <zhilizhao at tencent.com> | Fri Jun 24 12:04:04 2022 +0800| [ffc91940a4262f605cb650c93e52a657090ae148] | committer: Timo Rothenpieler

avcodec/cuviddec: fix AV1 decoding error

cuvidParseVideoData only supports pure OBUs, it reports an unknown
error with AV1CodecConfigurationRecord. Check whether extradata
is AV1CodecConfigurationRecord and skip the first 4 bytes to fix
the issue.

The bug is revealed in ffmpeg cmd since 45e3b6a68 and ffd1316e.

Signed-off-by: Timo Rothenpieler <timo at rothenpieler.org>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=ffc91940a4262f605cb650c93e52a657090ae148
---

 libavcodec/cuviddec.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/libavcodec/cuviddec.c b/libavcodec/cuviddec.c
index cb3cda7e24..5f265b1c70 100644
--- a/libavcodec/cuviddec.c
+++ b/libavcodec/cuviddec.c
@@ -953,6 +953,16 @@ static av_cold int cuvid_decode_init(AVCodecContext *avctx)
         extradata_size = avctx->extradata_size;
     }
 
+    // Check first bit to determine whether it's AV1CodecConfigurationRecord.
+    // Skip first 4 bytes of AV1CodecConfigurationRecord to keep configOBUs
+    // only, otherwise cuvidParseVideoData report unknown error.
+    if (avctx->codec->id == AV_CODEC_ID_AV1 &&
+            extradata_size > 4 &&
+            extradata[0] & 0x80) {
+        extradata += 4;
+        extradata_size -= 4;
+    }
+
     ctx->cuparse_ext = av_mallocz(sizeof(*ctx->cuparse_ext)
             + FFMAX(extradata_size - (int)sizeof(ctx->cuparse_ext->raw_seqhdr_data), 0));
     if (!ctx->cuparse_ext) {