[FFmpeg-cvslog] iffdec: Fix integer overflow.

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michaelni at gmx.at> | Tue Jul 24 05:17:36 2012 +0200| [0e1925ddc4bb1499fcfc6a1a3990115f8d30c243] | committer: Michael Niedermayer

iffdec: Fix integer overflow.

Found-by: durandal_1707
Signed-off-by: Michael Niedermayer <michaelni at gmx.at>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=0e1925ddc4bb1499fcfc6a1a3990115f8d30c243
---

 libavcodec/iff.c |    5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libavcodec/iff.c b/libavcodec/iff.c
index fe95357..a3241f6 100644
--- a/libavcodec/iff.c
+++ b/libavcodec/iff.c
@@ -238,6 +238,11 @@ static int extract_header(AVCodecContext *const avctx,
                 s->mask_buf = av_malloc((s->planesize * 32) + FF_INPUT_BUFFER_PADDING_SIZE);
                 if (!s->mask_buf)
                     return AVERROR(ENOMEM);
+                if (s->bpp > 16) {
+                    av_log(avctx, AV_LOG_ERROR, "bpp %d too large for palette\n", s->bpp);
+                    av_freep(&s->mask_buf);
+                    return AVERROR(ENOMEM);
+                }
                 s->mask_palbuf = av_malloc((2 << s->bpp) * sizeof(uint32_t) + FF_INPUT_BUFFER_PADDING_SIZE);
                 if (!s->mask_palbuf) {
                     av_freep(&s->mask_buf);