[Ffmpeg-devel] regression tests broken

[Diego Biurrun]

On Sat, Feb 25, 2006 at 01:16:17AM +0100, Michael Niedermayer wrote:
> 
> On Fri, Feb 24, 2006 at 07:38:11PM +0100, Diego Biurrun wrote:
> > The regression tests currently fail for me on both x86 and ppc, in
> > different places:
> > 
> > x86:
> > ../ffmpeg_g -y -flags +bitexact -dct fastint -idct simple -y -strict -2 -f pgmyuv -i vsynth1/%02d.pgm -an -vcodec snow -qscale 2 -flags +qpel -me iter -dia_size 2 -cmp 12 -subcmp 12 -s 128x64 ./data/a-snow.avi
> > /usr/src/mplayer/ffmpeg/tests/regression.sh: line 112:  7038 Illegal instruction     $ffmpeg -y -flags +bitexact -dct fastint -idct simple -benchmark $* >$datadir/bench.tmp 2>/tmp/ffmpeg$$
> > make[1]: *** [codectest] Error 132
> > make[1]: Leaving directory `/usr/src/mplayer/ffmpeg/tests'
> > make: *** [test] Error 2
> 
> gdb backtrace & disasm, gcc version, config.h, cputype please
> here the regression tests pass, even the ffserver ones

K6-III running Debian unstable (somewhat outdated), tried with both gcc
2.95.4 and 3.3.  I'm away from that machine right now, details later.

> > ppc:
> > ../ffmpeg_g -y -flags +bitexact -dct fastint -idct simple -y -i
> > ./data/a-adpcm_yam.wav -f wav ./data/out.wav
> > FFmpeg version CVS, Copyright (c) 2000-2004 Fabrice Bellard
> >   configuration:  --enable-gpl --enable-pp 
> >   libavutil version: 49.0.0
> >   libavcodec version: 51.7.0
> >   libavformat version: 50.3.0
> >   built on Feb 24 2006 17:01:00, gcc: 4.0.3 20060128 (prerelease)
> > (Debian 4.0.2-8)
> > size=    1032kB time=6.0 bitrate=1411.3kbits/s    
> > --- ./data/ffmpeg.regression    2006-02-24 19:33:07.000000000 +0100
> > +++ /home/diego/src/ffmpeg/tests/ffmpeg.regression.ref  2006-02-24
> > 16:57:59.000000000 +0100
> > @@ -133,10 +133,10 @@
> >  3533710 ./data/a-snow53.avi
> >  799d3db687f6cdd7a837ec156efc171f *./data/out.yuv
> >  stddev:  0.00 PSNR:99.99 bytes:7602176
> > -82f2344784a352477a9567a05dac1a83 *./data/a-dv.dv
> > +229e5f8d1f386d0cf84f629be7d74955 *./data/a-dv.dv
> >  7200000 ./data/a-dv.dv
> > -dba82b4c8fb136b721ae0cef3b6579c1 *./data/out.yuv
> > -stddev:  9.59 PSNR:28.48 bytes:7602176
> > +0ac51c62d3c2a4bacf216ea73050485e *./data/out.yuv
> > +stddev:  9.10 PSNR:28.94 bytes:7602176
> 
> is this a new issue? (=did my recent commit to dv.c broke it) or was it
> always broken?

I could swear it's a recent issue, but if I revert libavcodec/dv.c to
revision 1.71 it still fails..

The snapshot from 20060220 produces a different result:

../ffmpeg_g -y -flags +bitexact -dct fastint -idct simple -y -i
./data/a-adpcm_yam.wav -f wav ./data/out.wav
FFmpeg version CVS, Copyright (c) 2000-2004 Fabrice Bellard
  configuration:  --enable-gpl --enable-pp 
  libavutil version: 49.0.0
  libavcodec version: 51.7.0
  libavformat version: 50.3.0
  built on Feb 25 2006 12:08:21, gcc: 4.0.3 20060212 (prerelease)
(Debian 4.0.2-9)
size=    1032kB time=6.0 bitrate=1411.3kbits/s    
--- ./data/ffmpeg.regression    2006-02-25 12:10:40.000000000 +0100
+++ /tmp/ffmpeg/tests/ffmpeg.regression.ref     2006-02-02
16:34:09.000000000 +0100
@@ -125,12 +125,12 @@
 2653642 ./data/a-ffv1.avi
 799d3db687f6cdd7a837ec156efc171f *./data/out.yuv
 stddev:  0.00 PSNR:99.99 bytes:7602176
-45faa6a70d8166f0e5d5cd9500fd69ae *./data/a-snow.avi
-170882 ./data/a-snow.avi
-4208bde8b584ceee9210cbf6f2b4cc21 *./data/out.yuv
-stddev: 23.23 PSNR:20.80 bytes:7602176
-745a5ad480b5e2a2319be85764b89012 *./data/a-snow53.avi
-3540248 ./data/a-snow53.avi
+96ee2a2a6af9d4e95dfb41b485357d31 *./data/a-snow.avi
+161058 ./data/a-snow.avi
+f665aeec3668e7d8c962e74d3a0a0692 *./data/out.yuv
+stddev: 23.20 PSNR:20.81 bytes:7602176
+11fd61ee7e67ef7a7b2a3df973691305 *./data/a-snow53.avi
+3533710 ./data/a-snow53.avi
 799d3db687f6cdd7a837ec156efc171f *./data/out.yuv
 stddev:  0.00 PSNR:99.99 bytes:7602176
 e1da20e3f52f4d2aa18e9486986161fc *./data/a-dv.dv

Regression test: Error.
make[1]: *** [codectest] Error 1
make[1]: Leaving directory `/tmp/ffmpeg/tests'
make: *** [test] Error 2

Diego