[FFmpeg-devel] [PATCH] Add "unofficial" FATE tests

[Vitor Sessak]

On 07/09/2010 03:10 AM, Mike Melanson wrote:
> On 07/08/2010 03:58 PM, Vitor Sessak wrote:
>> On 07/09/2010 12:51 AM, M?ns Rullg?rd wrote:
>>> Vitor Sessak<vitor1001 at gmail.com> writes:
>>>
>>>> The attached patch add support for adding to "make fate" new tests
>>>> that are not yet integrated in Mike's database. It add two new targets:
>>>>
>>>> - "make fate1" does all the tests in Mike's database
>>>> - "make fate2" does all the tests described in tests/fate2.mak
>>>> - "make fate" is an alias for "make fate1 fate2"
>>>
>>> Do we really need to split it like that?
>>
>> This way Mike can add a FATE test that runs "make fate2" in the same way
>> he has one that runs "make test".
>
> As discussed, that's not going to happen. I already have a method for
> breaking down 'make test' into a smaller tests and inserting them into
> FATE.

Mike, did you read the whole thread? Tests are not supposed to stay long 
in "make fate2". The whole idea is to unload you of a part of the work 
of adding new tests. Today, to add a new FATE test one have to:

1) bug you
2) wait for you to have time (which looks a scarce resource)
3) you test the command line with your script
3a) if it fails in one of them, try to find out why
4) you add it to the database

Now, one would just need:

1) Discuss it in the ML
2) Commit it as make fate2
3) Revert if fate goes yellow and find you why
4) Wait (no hurry) you to move it out of "make fate2" to the main fate 
database

What you don't like about all this?

-Vitor