[FFmpeg-devel] [RFC] swscale regression tests

[Ramiro Polla]

2010/7/20 M?ns Rullg?rd <mans at mansr.com>:
> Ramiro Polla <ramiro.polla at gmail.com> writes:
>> I have attached an attempt at making swscale regression tests. This
>> method creates one file for each input pixel format (currently 42) and
>> compresses the output with bzip2. I have not tested with different
>> platforms (x86_64 linux only yet), and IIRC it doesn't give the same
>> result on all platforms. This is just a request for comments whether
>> this approach is good. I have also removed the fast bilinear test
>> since it works differently on many platforms and would make the
>> regression tests more complex (like keeping one reference file for
>> each platform).
>>
>> The output bzip2ed files add 280Kb of binary data to repository (5.6Mb
>> uncompressed). As for disk usage (with 'du'), it takes 340Kb (5.8Mb
>> uncompressed).
>
> Why can't you store a checksum instead?

Storing a checksum makes it harder to find out which test failed. And
the compressed files don't seem to take as much space as I had
originally thought they would take.

>> Index: Makefile
>> ===================================================================
>> --- Makefile ?(revision 24341)
>> +++ Makefile ?(working copy)
>> @@ -180,12 +180,14 @@
>> ?VCODEC_TESTS := $(addprefix regtest-, $(VCODEC_TESTS) $(VCODEC_TESTS-yes))
>> ?LAVF_TESTS ?:= $(addprefix regtest-, $(LAVF_TESTS) ?$(LAVF_TESTS-yes))
>> ?LAVFI_TESTS := $(addprefix regtest-, $(LAVFI_TESTS) $(LAVFI_TESTS-yes))
>> +SWSCALE_TESTS := $(addprefix regtest-, $(SWSCALE_TESTS) $(SWSCALE_TESTS-yes))
>>
>> ?CODEC_TESTS = $(VCODEC_TESTS) $(ACODEC_TESTS)
>>
>> ?codectest: $(CODEC_TESTS)
>> ?lavftest: ?$(LAVF_TESTS)
>> ?lavfitest: $(LAVFI_TESTS)
>> +swscaletest: $(SWSCALE_TESTS)
>>
>> ?AREF = tests/data/acodec.ref.wav
>> ?VREF = tests/data/vsynth1.ref.yuv
>> @@ -234,6 +236,11 @@
>> ? ? ? @$(SRC_PATH)/tests/lavfi-regression.sh $@ lavfi tests/vsynth1 "$(TARGET_EXEC)" "$(TARGET_PATH)"
>> ? ? ? @diff -u -w $(call REFFILE,lavfi,$@) $(call RESFILE,lavfi,$@)
>>
>> +$(SWSCALE_TESTS): libswscale/swscale-test$(EXESUF)
>> + ? ? @echo "TEST: SWSCALE $(@:regtest-%=%)"
>> + ? ? @$(SRC_PATH)/tests/swscale-regression.sh $@ swscale tests/swscale "$(TARGET_EXEC)" "$(TARGET_PATH)"
>> + ? ? @bzdiff -u -w $(call REFFILE,swscale,$@).bz2 $(call RESFILE,swscale,$@).bz2
>> +
>> ?seektest: codectest lavftest tests/seek_test$(EXESUF)
>> ? ? ? $(SRC_PATH)/tests/seek-regression.sh $(SRC_PATH) "$(TARGET_EXEC)" "$(TARGET_PATH)"
>
> This no longer applies. ?I have changed the way regtests are run.

Thanks for the updates, I'll work on it later.

>> --- /dev/null 2009-04-20 11:01:36.000000000 -0300
>> +++ tests/swscale-regression.sh ? ? ? 2010-07-20 00:38:29.000000000 -0300
>> @@ -0,0 +1,12 @@
>> +#!/bin/sh
>> +#
>> +# automatic regression test for libswscale
>> +#
>> +#
>> +#set -x
>
> Do we really need to cargo-cult that around _all_ the scripts?

If we don't put it, someone will yell that it's not consistent, and if
we want it removed, that takes another patch and potentially more
discussion. Anyways, patch attached to remove those comments (the file
name should be descriptive enough already).

Also why is there a space after the shebang in some scripts? (like "#!
/bin/sh"). It's not wrong, but just weird, and wastes one byte =)

>> +set -e
>> +
>> +. $(dirname $0)/regression-funcs.sh
>> +
>> +eval "$target_exec ${target_path}/libswscale/swscale-test" $test 2> /dev/null | bzip2 > $logfile.bz2
>
> Why the eval and the quotes? ?Just drop both.

Before I found out regression-funcs.sh would set up all the necessary
variables I had put the command in the Makefile, and for some reason
'eval' was needed. I'll drop them on the next patch.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: remove_comments.diff
Type: application/octet-stream
Size: 1184 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100720/12c043c2/attachment.obj>