[FFmpeg-cvslog] avcodec/aacdec_fixed: Fix integer overflow in predict()

[Michael Niedermayer]

ffmpeg | branch: release/3.2 | Michael Niedermayer <michael at niedermayer.cc> | Fri Oct 27 02:23:20 2017 +0200| [fcfa854abdfbda3fa64d93d8f14f10a7babe0c07] | committer: Michael Niedermayer

avcodec/aacdec_fixed: Fix integer overflow in predict()

Fixes: runtime error: signed integer overflow: -2110708110 + -82837504 cannot be represented in type 'int'
Fixes: 3547/clusterfuzz-testcase-minimized-6009386439802880

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 0976752420706c0a8b3cb8fd61497a47c7d7270f)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=fcfa854abdfbda3fa64d93d8f14f10a7babe0c07
---

 libavcodec/aacdec_fixed.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/aacdec_fixed.c b/libavcodec/aacdec_fixed.c
index e7c2d2d299..06bfa87e28 100644
--- a/libavcodec/aacdec_fixed.c
+++ b/libavcodec/aacdec_fixed.c
@@ -307,9 +307,9 @@ static av_always_inline void predict(PredictorState *ps, int *coef,
 
         if (shift < 31) {
             if (shift > 0) {
-                *coef += (pv.mant + (1 << (shift - 1))) >> shift;
+                *coef += (unsigned)((pv.mant + (1 << (shift - 1))) >> shift);
             } else
-                *coef += pv.mant << -shift;
+                *coef += (unsigned)(pv.mant << -shift);
         }
     }