[FFmpeg-cvslog] 4xm: Check available space in read_huffman_tables()

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michaelni at gmx.at> | Sat Jan 26 02:03:05 2013 +0100| [53a3fdbfc56da54b2c0a44eb1f956ec9d67d1425] | committer: Michael Niedermayer

4xm: Check available space in read_huffman_tables()

Fixes integer overflow and out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni at gmx.at>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=53a3fdbfc56da54b2c0a44eb1f956ec9d67d1425
---

 libavcodec/4xm.c |    9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/libavcodec/4xm.c b/libavcodec/4xm.c
index 2db3026..39254f7 100644
--- a/libavcodec/4xm.c
+++ b/libavcodec/4xm.c
@@ -599,8 +599,10 @@ static const uint8_t *read_huffman_tables(FourXContext *f,
     for (;;) {
         int i;
 
-        if (start <= end && ptr_end - ptr < end - start + 1 + 1)
+        if (ptr_end - ptr < FFMAX(end - start + 1, 0) + 1) {
+            av_log(f->avctx, AV_LOG_ERROR, "invalid data in read_huffman_tables\n");
             return NULL;
+        }
         for (i = start; i <= end; i++)
             frequency[i] = *ptr++;
         start = *ptr++;
@@ -614,6 +616,11 @@ static const uint8_t *read_huffman_tables(FourXContext *f,
     while ((ptr - buf) & 3)
         ptr++; // 4byte align
 
+    if (ptr > ptr_end) {
+        av_log(f->avctx, AV_LOG_ERROR, "ptr overflow in read_huffman_tables\n");
+        return NULL;
+    }
+
     for (j = 257; j < 512; j++) {
         int min_freq[2] = { 256 * 256, 256 * 256 };
         int smallest[2] = { 0, 0 };