[FFmpeg-cvslog] avformat/ape: Check frames size

[Michael Niedermayer]

ffmpeg | branch: release/3.2 | Michael Niedermayer <michael at niedermayer.cc> | Sat Sep 17 21:19:53 2022 +0200| [89914fa82b70eec8e34e6f1400230fb4cc5a2f12] | committer: Michael Niedermayer

avformat/ape: Check frames size

Fixes: signed integer overflow: 9223372036854775806 + 3 cannot be represented in type 'long'
Fixes: 50993/clusterfuzz-testcase-minimized-ffmpeg_dem_APE_fuzzer-6389264140599296

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit d0349c9929e2891c90011a83152624d5cf18e628)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=89914fa82b70eec8e34e6f1400230fb4cc5a2f12
---

 libavformat/ape.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/libavformat/ape.c b/libavformat/ape.c
index 99d6b666a5..6d62ee0ab9 100644
--- a/libavformat/ape.c
+++ b/libavformat/ape.c
@@ -325,6 +325,8 @@ static int ape_read_header(AVFormatContext * s)
             ape->frames[i].pos  -= ape->frames[i].skip;
             ape->frames[i].size += ape->frames[i].skip;
         }
+        if (ape->frames[i].size > INT_MAX - 3)
+            return AVERROR_INVALIDDATA;
         ape->frames[i].size = (ape->frames[i].size + 3) & ~3;
     }
     if (ape->fileversion < 3810) {