[FFmpeg-devel] [PATCH 2/4] avcodec/apedec: Fix 2 integer overflows in filter_3800()

[Michael Niedermayer]

Fixes: signed integer overflow: 1683879955 - -466265224 cannot be represented in type 'int'
Fixes: 37419/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_APE_fuzzer-6074294407921664

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
---
 libavcodec/apedec.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
index 022a6f9398..bf481ba354 100644
--- a/libavcodec/apedec.c
+++ b/libavcodec/apedec.c
@@ -909,8 +909,8 @@ static av_always_inline int filter_3800(APEPredictor *p,
         return predictionA;
     }
     d2 =  p->buf[delayA];
-    d1 = (p->buf[delayA] - p->buf[delayA - 1]) * 2U;
-    d0 =  p->buf[delayA] + ((p->buf[delayA - 2] - p->buf[delayA - 1]) * 8U);
+    d1 = (p->buf[delayA] - (unsigned)p->buf[delayA - 1]) * 2;
+    d0 =  p->buf[delayA] + ((p->buf[delayA - 2] - (unsigned)p->buf[delayA - 1]) * 8);
     d3 =  p->buf[delayB] * 2U - p->buf[delayB - 1];
     d4 =  p->buf[delayB];
 
-- 
2.17.1