[FFmpeg-cvslog] avcodec/mpegvideo_enc: Fix potential overflow in RD
Michael Niedermayer
git at videolan.org
Thu Jun 13 23:41:12 EEST 2024
ffmpeg | branch: release/5.1 | Michael Niedermayer <michael at niedermayer.cc> | Sun May 12 00:13:58 2024 +0200| [cb4e45231cdb3447c9690906b4edcd0b88f68a56] | committer: Michael Niedermayer
avcodec/mpegvideo_enc: Fix potential overflow in RD
Fixes: CID1500285 Unintentional integer overflow
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit b6b2b01025e016ce29e5add57305384a663edcfc)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=cb4e45231cdb3447c9690906b4edcd0b88f68a56
---
libavcodec/mpegvideo_enc.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/libavcodec/mpegvideo_enc.c b/libavcodec/mpegvideo_enc.c
index c528ac188e..d1c64a921d 100644
--- a/libavcodec/mpegvideo_enc.c
+++ b/libavcodec/mpegvideo_enc.c
@@ -1327,7 +1327,7 @@ static int estimate_best_b_count(MpegEncContext *s)
goto fail;
}
- rd += (out_size * lambda2) >> (FF_LAMBDA_SHIFT - 3);
+ rd += (out_size * (uint64_t)lambda2) >> (FF_LAMBDA_SHIFT - 3);
}
/* get the delayed frames */
@@ -1336,7 +1336,7 @@ static int estimate_best_b_count(MpegEncContext *s)
ret = out_size;
goto fail;
}
- rd += (out_size * lambda2) >> (FF_LAMBDA_SHIFT - 3);
+ rd += (out_size * (uint64_t)lambda2) >> (FF_LAMBDA_SHIFT - 3);
rd += c->error[0] + c->error[1] + c->error[2];
More information about the ffmpeg-cvslog
mailing list