[FFmpeg-devel] [PATCH] tests/checkasm/float_dsp: Increase allowed difference for float_dsp.vector_dmul

[James Almer]

On 10/29/2017 11:57 AM, Michael Niedermayer wrote:
> The choosen value is the lowest power of 2 that allows 1000 iterations of fate-checkasm-float_dsp
> to pass on x86-32

Ticket #6848 reports this value is still not enough. Maybe something
like 1.0e-12 or 1.0e-13 instead?

> 
> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> ---
>  tests/checkasm/float_dsp.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tests/checkasm/float_dsp.c b/tests/checkasm/float_dsp.c
> index 9b0a221c25..676c341f45 100644
> --- a/tests/checkasm/float_dsp.c
> +++ b/tests/checkasm/float_dsp.c
> @@ -165,7 +165,7 @@ static void test_vector_dmul_scalar(const double *src0, const double *src1)
>      call_ref(cdst, src0, src1[0], LEN);
>      call_new(odst, src0, src1[0], LEN);
>      for (i = 0; i < LEN; i++) {
> -        if (!double_near_abs_eps(cdst[i], odst[i], DBL_EPSILON)) {
> +        if (!double_near_abs_eps(cdst[i], odst[i], 256*DBL_EPSILON)) {
>              fprintf(stderr, "%d: %- .12f - %- .12f = % .12g\n", i,
>                      cdst[i], odst[i], cdst[i] - odst[i]);
>              fail();
>