[FFmpeg-cvslog] avcodec/wavarc: avoid signed integer overflow in AC code

[Michael Niedermayer]

ffmpeg | branch: release/6.1 | Michael Niedermayer <michael at niedermayer.cc> | Tue Mar 26 03:14:08 2024 +0100| [e7093154883e7cfc017b4ab71625ed0ca4aab3be] | committer: Michael Niedermayer

avcodec/wavarc: avoid signed integer overflow in AC code

Fixes: 62285/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_WAVARC_fuzzer-659847401740697
Fixes: signed integer overflow: 65312 * 34078 cannot be represented in type 'int'

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 1eb8cbd09c5f22d7ba9e0d443712a6ab80648637)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=e7093154883e7cfc017b4ab71625ed0ca4aab3be
---

 libavcodec/wavarc.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavcodec/wavarc.c b/libavcodec/wavarc.c
index 99cbaf0109..aa1af6330b 100644
--- a/libavcodec/wavarc.c
+++ b/libavcodec/wavarc.c
@@ -414,7 +414,7 @@ static int ac_init(AVCodecContext *avctx,
 
 static uint16_t ac_get_prob(WavArcContext *s)
 {
-    return ((s->freq_range - 1) + (s->ac_value - s->ac_low) * s->freq_range) /
+    return ((s->freq_range - 1) + (s->ac_value - s->ac_low) * (unsigned)s->freq_range) /
            ((s->ac_high - s->ac_low) + 1U);
 }
 
@@ -439,8 +439,8 @@ static int ac_normalize(AVCodecContext *avctx, WavArcContext *s, GetBitContext *
         goto fail;
 
     range = (s->ac_high - s->ac_low) + 1;
-    s->ac_high = (range * s->range_high) / s->freq_range + s->ac_low - 1;
-    s->ac_low += (range * s->range_low)  / s->freq_range;
+    s->ac_high = (range * (unsigned)s->range_high) / s->freq_range + s->ac_low - 1;
+    s->ac_low += (range * (unsigned)s->range_low)  / s->freq_range;
 
     if (s->ac_high < s->ac_low)
         goto fail;