[FFmpeg-cvslog] avcodec/tiff: Check subsample & rps values more completely

[Michael Niedermayer]

ffmpeg | branch: release/2.6 | Michael Niedermayer <michael at niedermayer.cc> | Mon Jan 18 03:31:25 2016 +0100| [524ee420502cdc387af000a67ab6f7d67eed8497] | committer: Michael Niedermayer

avcodec/tiff: Check subsample & rps values more completely

Fixes out of array access
Fixes: 83aedfb29af669c4d6e10f1bfad974d2/asan_heap-oob_1ab42fe_4984_9f6ec14462f8d8a00ea24b320572a963.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 89f464e9c229006e16f6bb5403c5529fdd0a9edd)

Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=524ee420502cdc387af000a67ab6f7d67eed8497
---

 libavcodec/tiff.c |    9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/libavcodec/tiff.c b/libavcodec/tiff.c
index 151e501..6ee57b1 100644
--- a/libavcodec/tiff.c
+++ b/libavcodec/tiff.c
@@ -1000,8 +1000,13 @@ static int tiff_decode_tag(TiffContext *s, AVFrame *frame)
             av_log(s->avctx, AV_LOG_ERROR, "subsample count invalid\n");
             return AVERROR_INVALIDDATA;
         }
-        for (i = 0; i < count; i++)
+        for (i = 0; i < count; i++) {
             s->subsampling[i] = ff_tget(&s->gb, type, s->le);
+            if (s->subsampling[i] <= 0) {
+                av_log(s->avctx, AV_LOG_ERROR, "subsampling %d is invalid\n", s->subsampling[i]);
+                return AVERROR_INVALIDDATA;
+            }
+        }
         break;
     case TIFF_T4OPTIONS:
         if (s->compr == TIFF_G3)
@@ -1249,7 +1254,7 @@ static int decode_frame(AVCodecContext *avctx,
                          avpkt->size - s->strippos);
     }
 
-    if (s->rps <= 0) {
+    if (s->rps <= 0 || s->rps % s->subsampling[1]) {
         av_log(avctx, AV_LOG_ERROR, "rps %d invalid\n", s->rps);
         return AVERROR_INVALIDDATA;
     }