[FFmpeg-cvslog] avcodec/g729dec: Use 64bit and clip in scalar product

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Tue Nov  5 23:28:35 2019 +0100| [bf9c4a12750e593d753011166b066efce208d9e0] | committer: Michael Niedermayer

avcodec/g729dec: Use 64bit and clip in scalar product

The G729 reference decoder clips after each individual operation and keeps track if overflow
occurred (in the fixed point implementation), this here is
simpler and faster but not 1:1 the same what the reference does.

Non fuzzed samples which trigger any such overflow are welcome, so
the need and impact of different clipping solutions can be evaluated.

Fixes: signed integer overflow: 1271483721 + 1073676289 cannot be represented in type 'int'
Fixes: 18617/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ACELP_KELVIN_fuzzer-5137705679978496

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=bf9c4a12750e593d753011166b066efce208d9e0
---

 libavcodec/g729dec.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/libavcodec/g729dec.c b/libavcodec/g729dec.c
index 67054b71df..16869e0df8 100644
--- a/libavcodec/g729dec.c
+++ b/libavcodec/g729dec.c
@@ -332,11 +332,14 @@ static int16_t g729d_voice_decision(int onset, int prev_voice_decision, const in
 
 static int32_t scalarproduct_int16_c(const int16_t * v1, const int16_t * v2, int order)
 {
-    int res = 0;
+    int64_t res = 0;
 
     while (order--)
         res += *v1++ * *v2++;
 
+    if      (res > INT32_MAX) return INT32_MAX;
+    else if (res < INT32_MIN) return INT32_MIN;
+
     return res;
 }