[FFmpeg-cvslog] avcodec/jpeg2000: Check sum of sizes of band->prec before allocating

[Michael Niedermayer]

ffmpeg | branch: release/3.3 | Michael Niedermayer <michael at niedermayer.cc> | Sun Jan 14 00:39:40 2018 +0100| [488a6ae7e5e8ffbcddcdda62174b898bbe6a8b4d] | committer: Michael Niedermayer

avcodec/jpeg2000: Check sum of sizes of band->prec before allocating

Fixes: OOM
Fixes: 4810/clusterfuzz-testcase-minimized-6034253235093504

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 6887e412434776eb260ad3904f565be491dd5726)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=488a6ae7e5e8ffbcddcdda62174b898bbe6a8b4d
---

 libavcodec/jpeg2000.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libavcodec/jpeg2000.c b/libavcodec/jpeg2000.c
index afeb9df27c..026b2db56f 100644
--- a/libavcodec/jpeg2000.c
+++ b/libavcodec/jpeg2000.c
@@ -543,6 +543,9 @@ int ff_jpeg2000_init_component(Jpeg2000Component *comp,
         if (!reslevel->band)
             return AVERROR(ENOMEM);
 
+        if (reslevel->num_precincts_x * (uint64_t)reslevel->num_precincts_y * reslevel->nbands > avctx->max_pixels / sizeof(*reslevel->band->prec))
+            return AVERROR(ENOMEM);
+
         for (bandno = 0; bandno < reslevel->nbands; bandno++, gbandno++) {
             ret = init_band(avctx, reslevel,
                             comp, codsty, qntsty,