[FFmpeg-cvslog] avcodec/zmbv: Check decomp_size

[Michael Niedermayer]

ffmpeg | branch: release/3.3 | Michael Niedermayer <michael at niedermayer.cc> | Wed Aug 16 16:03:23 2017 +0200| [99491bd260d3847decb417ede86eb872586f2a6f] | committer: Michael Niedermayer

avcodec/zmbv: Check decomp_size

Fixes: OOM
Fixes: 2710/clusterfuzz-testcase-minimized-4750001420894208

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Tomas Härdin <tjoppen at acc.umu.se>
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 931c0ac95cebe62f2bdd53a81bf40e3916be6476)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=99491bd260d3847decb417ede86eb872586f2a6f
---

 libavcodec/zmbv.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libavcodec/zmbv.c b/libavcodec/zmbv.c
index f126515bd1..b09dc41ebd 100644
--- a/libavcodec/zmbv.c
+++ b/libavcodec/zmbv.c
@@ -589,6 +589,11 @@ static av_cold int decode_init(AVCodecContext *avctx)
     // Needed if zlib unused or init aborted before inflateInit
     memset(&c->zstream, 0, sizeof(z_stream));
 
+    if ((avctx->width + 255ULL) * (avctx->height + 64ULL) > FFMIN(avctx->max_pixels, INT_MAX / 4) ) {
+        av_log(avctx, AV_LOG_ERROR, "Internal buffer (decomp_size) larger than max_pixels or too large\n");
+        return AVERROR_INVALIDDATA;
+    }
+
     c->decomp_size = (avctx->width + 255) * 4 * (avctx->height + 64);
 
     /* Allocate decompression buffer */