[FFmpeg-cvslog] avcodec/h264idct_template: Fix several runtime error: signed integer overflow

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Tue Feb 28 22:07:36 2017 +0100| [04c99c8042c8bfae817c722d90aa0f1a40db861e] | committer: Michael Niedermayer

avcodec/h264idct_template: Fix several runtime error: signed integer overflow

Fixes: 689/clusterfuzz-testcase-6029352737177600

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=04c99c8042c8bfae817c722d90aa0f1a40db861e
---

 libavcodec/h264idct_template.c | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/libavcodec/h264idct_template.c b/libavcodec/h264idct_template.c
index 9c5a43c..c627160 100644
--- a/libavcodec/h264idct_template.c
+++ b/libavcodec/h264idct_template.c
@@ -261,15 +261,15 @@ void FUNCC(ff_h264_luma_dc_dequant_idct)(int16_t *_output, int16_t *_input, int
 
     for(i=0; i<4; i++){
         const int offset= x_offset[i];
-        const int z0= temp[4*0+i] + temp[4*2+i];
-        const int z1= temp[4*0+i] - temp[4*2+i];
-        const int z2= temp[4*1+i] - temp[4*3+i];
-        const int z3= temp[4*1+i] + temp[4*3+i];
-
-        output[stride* 0+offset]= ((((z0 + z3)*qmul + 128 ) >> 8));
-        output[stride* 1+offset]= ((((z1 + z2)*qmul + 128 ) >> 8));
-        output[stride* 4+offset]= ((((z1 - z2)*qmul + 128 ) >> 8));
-        output[stride* 5+offset]= ((((z0 - z3)*qmul + 128 ) >> 8));
+        const SUINT z0= temp[4*0+i] + temp[4*2+i];
+        const SUINT z1= temp[4*0+i] - temp[4*2+i];
+        const SUINT z2= temp[4*1+i] - temp[4*3+i];
+        const SUINT z3= temp[4*1+i] + temp[4*3+i];
+
+        output[stride* 0+offset]= (int)((z0 + z3)*qmul + 128 ) >> 8;
+        output[stride* 1+offset]= (int)((z1 + z2)*qmul + 128 ) >> 8;
+        output[stride* 4+offset]= (int)((z1 - z2)*qmul + 128 ) >> 8;
+        output[stride* 5+offset]= (int)((z0 - z3)*qmul + 128 ) >> 8;
     }
 #undef stride
 }