[FFmpeg-devel] [PATCH] fate.sh: Allow overriding what targets to make for running the tests

[Martin Storsjö]

On Fri, 1 Dec 2023, Rémi Denis-Courmont wrote:

> Le 30 novembre 2023 23:13:59 GMT+02:00, "Martin Storsjö" <martin at martin.st> a écrit :
>> On Thu, 30 Nov 2023, Rémi Denis-Courmont wrote:
>
>>> In other words, is publishing on the FATE website worth making the 
>>> tests coverage and/or the build time worse?
>>
>> By making the test coverage worse, you mean if I'd be doing the full 
>> testing of many combinations already, and I'd stop doing that in order 
>> to do this lesser testing instead? If I'd be doing it (I currently 
>> don't) I guess that would be my concern, not others?
>
> No. The point is that this is adding a small hack that works for one 
> specific case for a short while (testing Armv8 IMM8 and DP), but is 
> known not to be sufficient anyway (for SVE, PAuth, RVV, etc).

I'll reiterate the question from the bottom of the mail, that you didn't 
respond to.

Would you be ok with a setup, where a FATE instance optionally can run a 
subset of tests instead of the full suite, but run them multiple times 
with e.g. different QEMU settings? That would allow repeating checkasm for 
all the interesting cases - and if one really wanted to spend a lot of CPU 
time on it, also could run the full FATE suite in all those 
configurations.

// Martin