[FFmpeg-devel] [PATCH 2/2] checkasm: sw_scale: Reduce range of test data in the yuv2yuvX test to get closer to real data

[Martin Storsjö]

This avoids overflows on some inputs in the x86 case, where the
assembly version would clip/overflow differently from the
C reference function.

This doesn't seem to be a real issue with actual input data, but
only with the previous fully random input data.

This also makes the test produce a bit more realistic output pixel
values, instead of having essentially all pixels clipped to either
0 or 255.

Signed-off-by: Martin Storsjö <martin at martin.st>
---
 tests/checkasm/sw_scale.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/tests/checkasm/sw_scale.c b/tests/checkasm/sw_scale.c
index d72506ed86..89403da317 100644
--- a/tests/checkasm/sw_scale.c
+++ b/tests/checkasm/sw_scale.c
@@ -187,8 +187,16 @@ static void check_yuv2yuvX(int accurate)
     } *vFilterData;
     uint8_t d_val = rnd();
     memset(dither, d_val, LARGEST_INPUT_SIZE);
+
     randomize_buffers((uint8_t*)src_pixels, LARGEST_FILTER * LARGEST_INPUT_SIZE * sizeof(int16_t));
     randomize_buffers((uint8_t*)filter_coeff, LARGEST_FILTER * sizeof(int16_t));
+    // Limit the range of the filter coefficients and intermediate
+    // pixel values, to avoid risk of clipping filter intermediates on x86.
+    for (i = 0; i < LARGEST_FILTER; i++)
+        filter_coeff[i] >>= 2;
+    for (i = 0; i < LARGEST_FILTER * LARGEST_INPUT_SIZE; i++)
+        src_pixels[i] >>= 2;
+
     ctx = sws_alloc_context();
     if (accurate)
         ctx->flags |= SWS_ACCURATE_RND;
-- 
2.25.1