[FFmpeg-cvslog] avformat/westwood_vqa: Check chunk size

[Michael Niedermayer]

ffmpeg | branch: release/5.1 | Michael Niedermayer <michael at niedermayer.cc> | Thu Sep 22 20:08:52 2022 +0200| [87f556a10c1c031c0837b7ad740e2ddc18595282] | committer: Michael Niedermayer

avformat/westwood_vqa: Check chunk size

the type is also changed to int as it is interpreted as int in av_get_packet()

Fixes: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 50993/clusterfuzz-testcase-minimized-ffmpeg_dem_WSVQA_fuzzer-6593408795279360
Fixes: 51896/clusterfuzz-testcase-minimized-ffmpeg_dem_WSVQA_fuzzer-4613908817903616

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Anton Khirnov <anton at khirnov.net>
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 5c0df3da0b7288a43a3b783117064cfcbc8037a5)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=87f556a10c1c031c0837b7ad740e2ddc18595282
---

 libavformat/westwood_vqa.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/libavformat/westwood_vqa.c b/libavformat/westwood_vqa.c
index e3d2e2668c..03b2d9e03c 100644
--- a/libavformat/westwood_vqa.c
+++ b/libavformat/westwood_vqa.c
@@ -178,13 +178,15 @@ static int wsvqa_read_packet(AVFormatContext *s,
     int ret = -1;
     uint8_t preamble[VQA_PREAMBLE_SIZE];
     uint32_t chunk_type;
-    uint32_t chunk_size;
-    int skip_byte;
+    int chunk_size;
+    unsigned skip_byte;
 
     while (avio_read(pb, preamble, VQA_PREAMBLE_SIZE) == VQA_PREAMBLE_SIZE) {
         chunk_type = AV_RB32(&preamble[0]);
         chunk_size = AV_RB32(&preamble[4]);
 
+        if (chunk_size < 0)
+            return AVERROR_INVALIDDATA;
         skip_byte = chunk_size & 0x01;
 
         if (chunk_type == VQFL_TAG) {
@@ -193,9 +195,9 @@ static int wsvqa_read_packet(AVFormatContext *s,
              * so it can be combined with the next VQFR packet. This way each packet
              * includes a whole frame as expected. */
             wsvqa->vqfl_chunk_pos = avio_tell(pb);
-            wsvqa->vqfl_chunk_size = (int)(chunk_size);
-            if (wsvqa->vqfl_chunk_size < 0 || wsvqa->vqfl_chunk_size > 3 * (1 << 20))
+            if (chunk_size > 3 * (1 << 20))
                 return AVERROR_INVALIDDATA;
+            wsvqa->vqfl_chunk_size = chunk_size;
             /* We need a big seekback buffer because there can be SNxx, VIEW and ZBUF
              * chunks (<512 KiB total) in the stream before we read VQFR (<256 KiB) and
              * seek back here. */