[FFmpeg-cvslog] smacker: check the return value of smacker_decode_tree

[Kostya Shishkov]

ffmpeg | branch: master | Kostya Shishkov <kostya.shishkov at gmail.com> | Wed Jun 12 14:27:00 2013 +0200| [bbb2945f2dfbc2306c1655bf7eb6e220912a9a7c] | committer: Luca Barbato

smacker: check the return value of smacker_decode_tree

Also prevent a memory leak.

Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable at libav.org

Signed-off-by: Luca Barbato <lu_zero at gentoo.org>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=bbb2945f2dfbc2306c1655bf7eb6e220912a9a7c
---

 libavcodec/smacker.c |   11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/libavcodec/smacker.c b/libavcodec/smacker.c
index ca9194f..5f84c50 100644
--- a/libavcodec/smacker.c
+++ b/libavcodec/smacker.c
@@ -629,7 +629,16 @@ static int smka_decode_frame(AVCodecContext *avctx, void *data,
         h[i].lengths = av_mallocz(256 * sizeof(int));
         h[i].values = av_mallocz(256 * sizeof(int));
         skip_bits1(&gb);
-        smacker_decode_tree(&gb, &h[i], 0, 0);
+        if (smacker_decode_tree(&gb, &h[i], 0, 0) < 0) {
+            for (; i >= 0; i--) {
+                if (vlc[i].table)
+                    ff_free_vlc(&vlc[i]);
+                av_free(h[i].bits);
+                av_free(h[i].lengths);
+                av_free(h[i].values);
+            }
+            return AVERROR_INVALIDDATA;
+        }
         skip_bits1(&gb);
         if(h[i].current > 1) {
             res = init_vlc(&vlc[i], SMKTREE_BITS, h[i].length,