[FFmpeg-cvslog] avcodec/xpmdec: Check size before allocation to avoid truncation

[Michael Niedermayer]

ffmpeg | branch: release/3.4 | Michael Niedermayer <michael at niedermayer.cc> | Thu Jan 12 22:05:07 2023 +0100| [2ca69aa42da16887488e6f903f3af2e041ecfa5b] | committer: Michael Niedermayer

avcodec/xpmdec: Check size before allocation to avoid truncation

Fixes:OOM
Fixes:out of array access (no testcase)
Fixes: 48567/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_XPM_fuzzer-6573323838685184

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 95f0f84dae4f040d91f1e60dc5438612c58e8906)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=2ca69aa42da16887488e6f903f3af2e041ecfa5b
---

 libavcodec/xpmdec.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libavcodec/xpmdec.c b/libavcodec/xpmdec.c
index 8aee89dd30..64f13e83f0 100644
--- a/libavcodec/xpmdec.c
+++ b/libavcodec/xpmdec.c
@@ -351,6 +351,9 @@ static int xpm_decode_frame(AVCodecContext *avctx, void *data,
         return AVERROR_INVALIDDATA;
     }
 
+    if (size > SIZE_MAX / 4)
+        return AVERROR(ENOMEM);
+
     size *= 4;
 
     ptr += mod_strcspn(ptr, ",") + 1;