[FFmpeg-cvslog] avcodec/mss3: Change types in rac_get_model_sym() to match the types they are initialized from

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Sat May  6 19:07:59 2017 +0200| [2ef0f392711445e173a56b2c073dedb021ae3783] | committer: Michael Niedermayer

avcodec/mss3: Change types in rac_get_model_sym() to match the types they are initialized from

Fixes integer overflow
Fixes: 1372/clusterfuzz-testcase-minimized-5712192982745088

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=2ef0f392711445e173a56b2c073dedb021ae3783
---

 libavcodec/mss3.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libavcodec/mss3.c b/libavcodec/mss3.c
index 7f3801774a..81b7e2017c 100644
--- a/libavcodec/mss3.c
+++ b/libavcodec/mss3.c
@@ -356,8 +356,9 @@ static int rac_get_model2_sym(RangeCoder *c, Model2 *m)
 
 static int rac_get_model_sym(RangeCoder *c, Model *m)
 {
-    int prob, prob2, helper, val;
+    int val;
     int end, end2;
+    unsigned prob, prob2, helper;
 
     prob       = 0;
     prob2      = c->range;