[FFmpeg-cvslog] avcodec/exr: Fix undefined integer multiplication

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Tue Sep 14 00:14:56 2021 +0200| [e67deaf86cb1a79054c4f6dcfcaab9b2c60eb8a4] | committer: Michael Niedermayer

avcodec/exr: Fix undefined integer multiplication

Fixes: signed integer overflow: 7020950083487072256 * 2 cannot be represented in type 'long long'
Fixes: 37523/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EXR_fuzzer-5133634955771904

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=e67deaf86cb1a79054c4f6dcfcaab9b2c60eb8a4
---

 libavcodec/exr.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/libavcodec/exr.c b/libavcodec/exr.c
index 67340c892d..66b7e258ee 100644
--- a/libavcodec/exr.c
+++ b/libavcodec/exr.c
@@ -1033,12 +1033,14 @@ static int dwa_uncompress(EXRContext *s, const uint8_t *src, int compressed_size
     }
 
     if (ac_size > 0) {
-        unsigned long dest_len = ac_count * 2LL;
+        unsigned long dest_len;
         GetByteContext agb = gb;
 
         if (ac_count > 3LL * td->xsize * s->scan_lines_per_block)
             return AVERROR_INVALIDDATA;
 
+        dest_len = ac_count * 2LL;
+
         av_fast_padded_malloc(&td->ac_data, &td->ac_size, dest_len);
         if (!td->ac_data)
             return AVERROR(ENOMEM);
@@ -1062,12 +1064,14 @@ static int dwa_uncompress(EXRContext *s, const uint8_t *src, int compressed_size
     }
 
     {
-        unsigned long dest_len = dc_count * 2LL;
+        unsigned long dest_len;
         GetByteContext agb = gb;
 
         if (dc_count != dc_w * dc_h * 3)
             return AVERROR_INVALIDDATA;
 
+        dest_len = dc_count * 2LL;
+
         av_fast_padded_malloc(&td->dc_data, &td->dc_size, FFALIGN(dest_len, 64) * 2);
         if (!td->dc_data)
             return AVERROR(ENOMEM);