[FFmpeg-devel] [PATCH] checkasm: add sample argument to adjust during bench

[Rémi Denis-Courmont]

Le torstaina 16. toukokuuta 2024, 16.09.49 EEST J. Dekker a écrit :
> Some timers on certain device and test combinations can produce noisy
> results, affecting the reliability of performance measurements. One
> notable example of this is the Canaan K230 RISC-V development board.
> 
> An option to adjust the number of samples (--samples) has been added,
> allowing developers to increase or adjust the sample count for more
> reliable results.

IMO, this should pick a reasonable estimate rather than rely on the tester to 
guess.

> 
> Signed-off-by: J. Dekker <jdek at itanimul.li>
> ---
> 
>  This could also be implemented as a compile time define or a configure
>  argument.
> 
>  tests/checkasm/checkasm.c | 12 +++++++++++-
>  tests/checkasm/checkasm.h |  4 ++--
>  2 files changed, 13 insertions(+), 3 deletions(-)
> 
> diff --git a/tests/checkasm/checkasm.c b/tests/checkasm/checkasm.c
> index ffc89882b1..c31adc3690 100644
> --- a/tests/checkasm/checkasm.c
> +++ b/tests/checkasm/checkasm.c
> @@ -72,6 +72,9 @@
>  void (*checkasm_checked_call)(void *func, int dummy, ...) =
> checkasm_checked_call_novfp; #endif
> 
> +/* Trade-off between speed and accuracy */
> +uint64_t bench_runs = 1000;
> +
>  /* List of tests to invoke */
>  static const struct {
>      const char *name;
> @@ -819,7 +822,7 @@ static void bench_uninit(void)
>  static int usage(const char *path)
>  {
>      fprintf(stderr,
> -            "Usage: %s [--bench] [--test=<pattern>] [--verbose] [seed]\n",
> +            "Usage: %s [--bench] [--samples=<count>] [--test=<pattern>]
> [--verbose] [seed]\n", path);
>      return 1;
>  }
> @@ -866,6 +869,13 @@ int main(int argc, char *argv[])
>              state.test_name = arg + 7;
>          } else if (!strcmp(arg, "--verbose") || !strcmp(arg, "-v")) {
>              state.verbose = 1;
> +        } else if (!strncmp(arg, "--samples=", 10)) {
> +            l = strtoul(arg + 10, &end, 10);
> +            if (*end == '\0') {
> +                bench_runs = l;
> +            } else {
> +                return usage(argv[0]);
> +            }
>          } else if ((l = strtoul(arg, &end, 10)) <= UINT_MAX &&
>                     *end == '\0') {
>              seed = l;
> diff --git a/tests/checkasm/checkasm.h b/tests/checkasm/checkasm.h
> index 07fcc751ff..11f07a919f 100644
> --- a/tests/checkasm/checkasm.h
> +++ b/tests/checkasm/checkasm.h
> @@ -167,7 +167,7 @@ extern AVLFG checkasm_lfg;
> 
>  static av_unused void *func_ref, *func_new;
> 
> -#define BENCH_RUNS 1000 /* Trade-off between accuracy and speed */
> +extern uint64_t bench_runs;
> 
>  /* Decide whether or not the specified function needs to be tested */
>  #define check_func(func, ...) (checkasm_save_context(), func_ref =
> checkasm_check_func((func_new = func), __VA_ARGS__)) @@ -339,7 +339,7 @@
> typedef struct CheckasmPerf {
>              int ti, tcount = 0;\
>              uint64_t t = 0; \
>              checkasm_set_signal_handler_state(1);\
> -            for (ti = 0; ti < BENCH_RUNS; ti++) {\
> +            for (ti = 0; ti < bench_runs; ti++) {\

The C compiler cannot prove that the global bench_runs is not modified by the 
tested code. This needs to be cached in an automatic variable.

>                  PERF_START(t);\
>                  tfunc(__VA_ARGS__);\
>                  tfunc(__VA_ARGS__);\


-- 
レミ・デニ-クールモン
http://www.remlab.net/