[FFmpeg-cvslog] avcodec/sbrdsp_fixed: Fix integer overflow in shift in sbr_hf_g_filt_c()

[Michael Niedermayer]

ffmpeg | branch: release/2.8 | Michael Niedermayer <michael at niedermayer.cc> | Wed Nov  1 14:00:20 2017 +0100| [efe9439caa250a2729cbaf968acda558fe503f4f] | committer: Michael Niedermayer

avcodec/sbrdsp_fixed: Fix integer overflow in shift in sbr_hf_g_filt_c()

Fixes: runtime error: shift exponent 66 is too large for 64-bit type 'long long'
Fixes: 3642/clusterfuzz-testcase-minimized-5443853801750528

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 981e99ab99986935affad7c164ebdfe28e8ea7f8)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=efe9439caa250a2729cbaf968acda558fe503f4f
---

 libavcodec/sbrdsp_fixed.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/libavcodec/sbrdsp_fixed.c b/libavcodec/sbrdsp_fixed.c
index f45bb847a8..07ef12117c 100644
--- a/libavcodec/sbrdsp_fixed.c
+++ b/libavcodec/sbrdsp_fixed.c
@@ -233,12 +233,14 @@ static void sbr_hf_g_filt_c(int (*Y)[2], const int (*X_high)[40][2],
     int64_t accu;
 
     for (m = 0; m < m_max; m++) {
-        int64_t r = 1LL << (22-g_filt[m].exp);
-        accu = (int64_t)X_high[m][ixh][0] * ((g_filt[m].mant + 0x40)>>7);
-        Y[m][0] = (int)((accu + r) >> (23-g_filt[m].exp));
+        if (22 - g_filt[m].exp < 61) {
+            int64_t r = 1LL << (22-g_filt[m].exp);
+            accu = (int64_t)X_high[m][ixh][0] * ((g_filt[m].mant + 0x40)>>7);
+            Y[m][0] = (int)((accu + r) >> (23-g_filt[m].exp));
 
-        accu = (int64_t)X_high[m][ixh][1] * ((g_filt[m].mant + 0x40)>>7);
-        Y[m][1] = (int)((accu + r) >> (23-g_filt[m].exp));
+            accu = (int64_t)X_high[m][ixh][1] * ((g_filt[m].mant + 0x40)>>7);
+            Y[m][1] = (int)((accu + r) >> (23-g_filt[m].exp));
+        }
     }
 }