[FFmpeg-cvslog] avcodec/lagarith: Check that the range coded data stream is consistent when the probabilities indicate no data could have been coded.

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Thu Jun 14 22:16:52 2018 +0200| [8d21ab4d128ddae03fe6b21542c29dee240151db] | committer: Michael Niedermayer

avcodec/lagarith: Check that the range coded data stream is consistent when the probabilities indicate no data could have been coded.

Fixes: Timeout
Fixes: 8638/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LAGARITH_fuzzer-5132046098759680
Fixes: 8943/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LAGARITH_fuzzer-4883030219948032

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=8d21ab4d128ddae03fe6b21542c29dee240151db
---

 libavcodec/lagarith.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/libavcodec/lagarith.c b/libavcodec/lagarith.c
index 0f4aa89486..ba2da2eeb2 100644
--- a/libavcodec/lagarith.c
+++ b/libavcodec/lagarith.c
@@ -141,6 +141,7 @@ static int lag_read_prob_header(lag_rac *rac, GetBitContext *gb)
     unsigned prob, cumulative_target;
     unsigned cumul_prob = 0;
     unsigned scaled_cumul_prob = 0;
+    int nnz = 0;
 
     rac->prob[0] = 0;
     rac->prob[257] = UINT_MAX;
@@ -164,6 +165,8 @@ static int lag_read_prob_header(lag_rac *rac, GetBitContext *gb)
                 prob = 256 - i;
             for (j = 0; j < prob; j++)
                 rac->prob[++i] = 0;
+        }else {
+            nnz++;
         }
     }
 
@@ -172,6 +175,10 @@ static int lag_read_prob_header(lag_rac *rac, GetBitContext *gb)
         return -1;
     }
 
+    if (nnz == 1 && (show_bits_long(gb, 32) & 0xFFFFFF)) {
+        return AVERROR_INVALIDDATA;
+    }
+
     /* Scale probabilities so cumulative probability is an even power of 2. */
     scale_factor = av_log2(cumul_prob);