[FFmpeg-devel] [PATCH v2] fate: Add a target for listing failed tests

[Alexander Strasser]

Hi Martin,

still looks good to me.

On 2024-12-04 16:08 +0200, Martin Storsjö wrote:
[...]
> --- a/tests/Makefile
> +++ b/tests/Makefile
> @@ -313,6 +313,12 @@ $(FATE): $(FATE_UTILS:%=tests/%$(HOSTEXESUF)) | $(FATE_OUTDIRS)
>  fate-list:
>  	@printf '%s\n' $(sort $(FATE))
>
> +fate-list-failing:
> +	@awk -F: '$$2 != 0 { print "fate-" $$1 }' tests/data/fate/*.rep

Unfortunately there is a small problem when calling
fate-list-failing target when no .rep files are present.

The target fails because we cannot open `tests/data/fate/*.rep`.
No output on stdout, which is correct because no test failed,
but we get error messages on stderr and a non-zero exit status.

This was also present in v1, but I just spotted it only today.

For now I found this way to solve it:

    @! ls tests/data/fate/*.rep >/dev/null 2>&1 || awk -F: '$$2 != 0 { print "fate-" $$1 }' tests/data/fate/*.rep

It's a bit expensive, but I couldn't think of something better so far.
Maybe it's good enough for now if there is no better idea?
Could be improved later on if needed.


Best regards,
  Alexander


> +
> +fate-clear-results:
> +	@rm -f tests/data/fate/*.rep
> +
>  coverage.info: TAG = LCOV
>  coverage.info:
>  	$(M)lcov -q -d $(CURDIR) -b $(patsubst src%,./,$(SRC_LINK)) --capture | \
> --