[FFmpeg-devel] [RFC] Lavfi test system

[Stefano Sabatini]

On date Sunday 2010-03-14 23:53:52 +0000, M?ns Rullg?rd encoded:
> Stefano Sabatini <stefano.sabatini-lala at poste.it> writes:
> 
> > On date Wednesday 2010-03-10 01:59:10 +0100, Stefano Sabatini encoded:
> >> On date Monday 2010-03-08 23:02:10 +0100, Michael Niedermayer encoded:
> >> > On Mon, Mar 08, 2010 at 10:40:02PM +0100, Stefano Sabatini wrote:
> >> > > On date Monday 2010-03-08 03:17:11 +0100, Michael Niedermayer encoded:
> >> > > > we dont want spurious unreproduceable failures
> >> > > > or other way around spurious unreproduceable successes in teh test wen
> >> > > > they should fail
> >> > > > it makes debuging hard ...
> >> > > 
> >> > > I agree but this way we are using only a limited subset of slice
> >> > > values.
> >> > 
> >> > true, could we postpone resolving this though.
> >> > having some regression tests is very important
> >> > testing all slice sizes is not that important
> >> > 
> >> > 
> >> > > 
> >> > > Another solution would be to iterate a few times each test with
> >> > > different slice height values taken from a pseudo random
> >> > > sequence. Would this be OK?
> >> > 
> >> > that slows it down proportionally
> >> > maybe we could add a "r" parameter to slicify to make it produce
> >> > deterministic but random slice sizes that would test more than a
> >> > single size for a video
> >> 
> >> Thinking more about it, that looked like the best solution, see the
> >> attached patch (which depends on that one extending slicify which I just
> >> posted in a new thread).
> >
> > Patches updated.
> >
> > Regards.
> > -- 
> > FFmpeg = Faithful & Fanciful Multipurpose Proud Ecumenical Gorilla
> >
> > Index: libavfilter-soc/ffmpeg/Makefile
> > ===================================================================
> > --- libavfilter-soc.orig/ffmpeg/Makefile	2010-03-13 11:45:37.000000000 +0100
> > +++ libavfilter-soc/ffmpeg/Makefile	2010-03-13 12:05:18.000000000 +0100
> > @@ -250,7 +250,18 @@
> >  LAVF_TESTS-$(call MUXDEM,WAV)                += wav
> >  LAVF_TESTS-$(call MUXDEM,YUV4MPEGPIPE)       += yuv4mpeg
> >  
> > -LAVFI_TEST=
> > +LAVFI_TESTS =       \
> > +crop                \
> > +crop_scale          \
> > +crop_scale_vflip    \
> > +crop_vflip          \
> > +null                \
> > +scale200            \
> > +scale500            \
> > +vflip               \
> > +vflip_crop          \
> > +vflip_vflip         \
> > +
> 
> Please indent the continuation lines a bit.

Now that I committed I'm not sure it's what you intended, you're free
to perform cosmetics changes of course.

> >  ACODEC_TESTS := $(addprefix regtest-, $(ACODEC_TESTS) $(ACODEC_TESTS-yes))
> >  VCODEC_TESTS := $(addprefix regtest-, $(VCODEC_TESTS) $(VCODEC_TESTS-yes))
> > @@ -261,8 +272,7 @@
> >  
> >  codectest: $(CODEC_TESTS)
> >  lavftest:  $(LAVF_TESTS)
> > -
> > -# lavfitest: $(LAVFI_TESTS)
> > +lavfitest: $(LAVFI_TESTS)
> >  
> >  $(ACODEC_TESTS): regtest-aref
> >  $(VCODEC_TESTS): regtest-vref
> > Index: libavfilter-soc/ffmpeg/tests/lavfi-regression.sh
> > ===================================================================
> > --- libavfilter-soc.orig/ffmpeg/tests/lavfi-regression.sh	2010-03-13 11:42:05.000000000 +0100
> > +++ libavfilter-soc/ffmpeg/tests/lavfi-regression.sh	2010-03-13 12:05:18.000000000 +0100
> > @@ -20,14 +20,20 @@
> >      vfilters=$2
> >  
> >      if [ -n "$test" ] ; then
> > -        do_video_encoding ${test_name}.avi "" "-vcodec rawvideo -vfilters $vfilters"
> > +        do_video_encoding ${test_name}.nut "" "-vcodec rawvideo -vfilters $vfilters"
> >      fi
> >  }
> >  
> > -# example tests:
> > -# do_lavfi "crop" "crop=100:100:-1:-1"
> > -# do_lavfi "crop_scale" "crop=100:100,scale=200:-1"
> > -# do_lavfi "scale" "scale=200:200"
> > +do_lavfi "crop"               "crop=100:100"
> > +do_lavfi "crop_scale"         "crop=100:100,scale=400:-1"
> > +do_lavfi "crop_scale_vflip"   "null,null,crop=200:200,crop=20:20,scale=200:200,scale=250:250,vflip,vflip,null,scale=200:200,crop=100:100,vflip,scale=200:200,null,vflip,crop=100:100,null"
> > +do_lavfi "crop_vflip"         "crop=100:100,vflip"
> > +do_lavfi "null"               "null"
> > +do_lavfi "scale200"           "scale=200:200"
> > +do_lavfi "scale500"           "scale=500:500"
> > +do_lavfi "vflip"              "vflip"
> > +do_lavfi "vflip_crop"         "vflip,crop=100:100"
> > +do_lavfi "vflip_vflip"        "vflip,vflip"
> 
> Looks good enough.  Maybe we want to parallelise these tests later,
> but that can wait.

Excellent, committed!

The test system is still incomplete, I'll post the continuation as a
separate thread (also I still have to fix somehow the fourCC trouble).

Regards.
-- 
FFmpeg = Fundamental Fabulous Marvellous Programmable Everlasting Gargoyle