[FFmpeg-cvslog] avcodec/gdv: Check input palette size before rescale()

[Michael Niedermayer]

ffmpeg | branch: release/4.1 | Michael Niedermayer <michael at niedermayer.cc> | Thu Apr 25 19:18:08 2019 +0200| [08d736d5363e1ad9210f2067de3bedc9a9229b45] | committer: Michael Niedermayer

avcodec/gdv: Check input palette size before rescale()

Fixes: Timeout (22sec -> 11sec)
Fixes: 13576/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_GDV_fuzzer-5681024577568768

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda at gmail.com>
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit f857753f56f86046d454969e33ba85b3bac99be2)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=08d736d5363e1ad9210f2067de3bedc9a9229b45
---

 libavcodec/gdv.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/gdv.c b/libavcodec/gdv.c
index e946bd61df..f46bc28210 100644
--- a/libavcodec/gdv.c
+++ b/libavcodec/gdv.c
@@ -475,6 +475,8 @@ static int gdv_decode_frame(AVCodecContext *avctx, void *data,
     if (pal && pal_size == AVPALETTE_SIZE)
         memcpy(gdv->pal, pal, AVPALETTE_SIZE);
 
+    if (compression < 2 && bytestream2_get_bytes_left(gb) < 256*3)
+        return AVERROR_INVALIDDATA;
     rescale(gdv, gdv->frame, avctx->width, avctx->height,
             !!(flags & 0x10), !!(flags & 0x20));
 
@@ -482,8 +484,6 @@ static int gdv_decode_frame(AVCodecContext *avctx, void *data,
     case 1:
         memset(gdv->frame + PREAMBLE_SIZE, 0, gdv->frame_size - PREAMBLE_SIZE);
     case 0:
-        if (bytestream2_get_bytes_left(gb) < 256*3)
-            return AVERROR_INVALIDDATA;
         for (i = 0; i < 256; i++) {
             unsigned r = bytestream2_get_byte(gb);
             unsigned g = bytestream2_get_byte(gb);