[FFmpeg-devel] [PATCH 2/2] fate: add decoder test for ATRAC9

[Carl Eugen Hoyos]

2018-07-01 14:51 GMT+02:00, James Almer <jamrial at gmail.com>:
> On 7/1/2018 8:46 AM, Rostislav Pehlivanov wrote:
>> Signed-off-by: Rostislav Pehlivanov <atomnuker at gmail.com>
>> ---
>>  tests/fate/atrac.mak | 13 ++++++++++++-
>>  1 file changed, 12 insertions(+), 1 deletion(-)
>>
>> diff --git a/tests/fate/atrac.mak b/tests/fate/atrac.mak
>> index acf79a539c..38a5f33150 100644
>> --- a/tests/fate/atrac.mak
>> +++ b/tests/fate/atrac.mak
>> @@ -33,7 +33,17 @@ fate-atrac3p-2: REF =
>> $(SAMPLES)/atrac3p/sonateno14op27-2-cut.pcm
>>
>>  FATE_ATRAC3P-$(call DEMDEC, OMA, ATRAC3P) += $(FATE_ATRAC3P)
>>
>> -FATE_ATRAC_ALL = $(FATE_ATRAC1-yes) $(FATE_ATRAC3-yes)
>> $(FATE_ATRAC3P-yes)
>> +FATE_ATRAC9 += fate-atrac9-1
>> +fate-atrac9-1: CMD = pcm -i $(TARGET_SAMPLES)/atrac9/sy_title01.at9
>> +fate-atrac9-1: REF = $(SAMPLES)/atrac9/sy_title01.at9.pcm
>> +
>> +FATE_ATRAC9 += fate-atrac9-2
>> +fate-atrac9-2: CMD = pcm -i
>> $(TARGET_SAMPLES)/atrac9/3_br144_nb10_ib10_g04_bex1_sf1_d1.at9
>> +fate-atrac9-2: REF =
>> $(SAMPLES)/atrac9/3_br144_nb10_ib10_g04_bex1_sf1_d1.at9.pcm
>
> Are these pcm files from a test suit (if any), or at least created
> by an official binary decoder?

I believe we added support for codecs when we were not able to
produce "official" binary output.

> I'd rather avoid adding tests using reference files created by
> this same decoder

Why not?
I agree that in an ideal world, we would only have known correct
reference files but this is not the case now (we have always been
knowingly testing "wrong" output") and, more important, I don't
think fate is a test for correctness but much more a regression
test to find unintended changes in FFmpeg's behaviour, don't you
agree? Finding such unintended changes even makes sense for
decoders that are known not to produce correct output.

Good to know that this decoder is bit-exact at least on some
platform=-)

Carl Eugen