[FFmpeg-cvslog] checkasm/h264dsp: use int64_t scale values

James Almer git at videolan.org
Fri May 17 23:04:34 EEST 2024


ffmpeg | branch: master | James Almer <jamrial at gmail.com> | Tue May 14 23:28:50 2024 -0300| [727a603158b466a3f11f419012856e845a0721b9] | committer: James Almer

checkasm/h264dsp: use int64_t scale values

Fixes "signed integer overflow: [varies] * 104858 cannot be represented in type 'int'" errors
under ubsan.

Signed-off-by: James Almer <jamrial at gmail.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=727a603158b466a3f11f419012856e845a0721b9
---

 tests/checkasm/h264dsp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/checkasm/h264dsp.c b/tests/checkasm/h264dsp.c
index cb180cc44f..0cc1f32740 100644
--- a/tests/checkasm/h264dsp.c
+++ b/tests/checkasm/h264dsp.c
@@ -83,7 +83,7 @@ static void dct4x4_##size(dctcoef *coef)                                     \
     }                                                                        \
     for (y = 0; y < 4; y++) {                                                \
         for (x = 0; x < 4; x++) {                                            \
-            static const int scale[] = { 13107 * 10, 8066 * 13, 5243 * 16 }; \
+            const int64_t scale[] = { 13107 * 10, 8066 * 13, 5243 * 16 };    \
             const int idx = (y & 1) + (x & 1);                               \
             coef[y*4 + x] = (coef[y*4 + x] * scale[idx] + (1 << 14)) >> 15;  \
         }                                                                    \



More information about the ffmpeg-cvslog mailing list