[FFmpeg-cvslog] avcodec/tiff: more completely check bpp/bppcount

[Michael Niedermayer]

ffmpeg | branch: release/0.10 | Michael Niedermayer <michaelni at gmx.at> | Fri Oct  3 16:08:32 2014 +0200| [7c1150bf05cd4f3c216fbe3441894a567a18f2ee] | committer: Michael Niedermayer

avcodec/tiff: more completely check bpp/bppcount

Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
(cherry picked from commit e1c0cfaa419aa5d320540d5a1b3f8fd9b82ab7e5)

Conflicts:

	libavcodec/tiff.c
(cherry picked from commit e9125e74897135d690cf44f6e6d39e80dcd07803)

Conflicts:

	libavcodec/tiff.c

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=7c1150bf05cd4f3c216fbe3441894a567a18f2ee
---

 libavcodec/tiff.c |   13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/libavcodec/tiff.c b/libavcodec/tiff.c
index 00ff8d3..930da9f 100644
--- a/libavcodec/tiff.c
+++ b/libavcodec/tiff.c
@@ -386,11 +386,11 @@ static int tiff_decode_tag(TiffContext *s)
         s->height = value;
         break;
     case TIFF_BPP:
-        s->bppcount = count;
-        if(count > 4){
-            av_log(s->avctx, AV_LOG_ERROR, "This format is not supported (bpp=%d, %d components)\n", s->bpp, count);
+        if(count > 4U){
+            av_log(s->avctx, AV_LOG_ERROR, "This format is not supported (bpp=%d, %d components)\n", value, count);
             return -1;
         }
+        s->bppcount = count;
         if(count == 1) s->bpp = value;
         else{
             switch(type){
@@ -407,6 +407,13 @@ static int tiff_decode_tag(TiffContext *s)
                 s->bpp = -1;
             }
         }
+        if (s->bpp > 64U) {
+            av_log(s->avctx, AV_LOG_ERROR,
+                   "This format is not supported (bpp=%d, %d components)\n",
+                   s->bpp, count);
+            s->bpp = 0;
+            return AVERROR_INVALIDDATA;
+        }
         break;
     case TIFF_SAMPLES_PER_PIXEL:
         if (count != 1) {