[FFmpeg-cvslog] avcodec/vmixdec: Check shift before use

[Michael Niedermayer]

ffmpeg | branch: release/6.1 | Michael Niedermayer <michael at niedermayer.cc> | Wed Mar 20 03:30:56 2024 +0100| [e2a58916b1bde8de18c8216d4d7f9d49e01ae612] | committer: Michael Niedermayer

avcodec/vmixdec: Check shift before use

Fixes: shift exponent 32 is too large for 32-bit type 'unsigned int'
Fixes: 65909/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VMIX_fuzzer-519459745831321

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 70b26b693e9e06bcd9fe83ee5063ee40e32ce02f)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=e2a58916b1bde8de18c8216d4d7f9d49e01ae612
---

 libavcodec/vmixdec.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libavcodec/vmixdec.c b/libavcodec/vmixdec.c
index d6b6e3557f..ab283d13db 100644
--- a/libavcodec/vmixdec.c
+++ b/libavcodec/vmixdec.c
@@ -235,6 +235,9 @@ static int decode_frame(AVCodecContext *avctx,
     else if (offset != 3)
         return AVERROR_INVALIDDATA;
 
+    if (s->lshift > 31)
+        return AVERROR_INVALIDDATA;
+
     q = quality[FFMIN(avpkt->data[offset - 2], FF_ARRAY_ELEMS(quality)-1)];
     for (int n = 0; n < 64; n++)
         s->factors[n] = quant[n] * q;