[FFmpeg-devel] [PATCH] checkasm/h264dsp: use int64_t scale values

James Almer jamrial at gmail.com
Wed May 15 05:29:07 EEST 2024


Fixes "signed integer overflow: [varies] * 104858 cannot be represented in type 'int'" errors
under ubsan.

Signed-off-by: James Almer <jamrial at gmail.com>
---
 tests/checkasm/h264dsp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/checkasm/h264dsp.c b/tests/checkasm/h264dsp.c
index cb180cc44f..0cc1f32740 100644
--- a/tests/checkasm/h264dsp.c
+++ b/tests/checkasm/h264dsp.c
@@ -83,7 +83,7 @@ static void dct4x4_##size(dctcoef *coef)                                     \
     }                                                                        \
     for (y = 0; y < 4; y++) {                                                \
         for (x = 0; x < 4; x++) {                                            \
-            static const int scale[] = { 13107 * 10, 8066 * 13, 5243 * 16 }; \
+            const int64_t scale[] = { 13107 * 10, 8066 * 13, 5243 * 16 };    \
             const int idx = (y & 1) + (x & 1);                               \
             coef[y*4 + x] = (coef[y*4 + x] * scale[idx] + (1 << 14)) >> 15;  \
         }                                                                    \
-- 
2.45.0



More information about the ffmpeg-devel mailing list