[FFmpeg-devel] [PATCH] correct make test failure from 15261 release until now (15899)

Michael Niedermayer michaelni
Sat Nov 22 03:11:42 CET 2008

On Fri, Nov 21, 2008 at 11:23:57PM +0100, Vitor Sessak wrote:
> Michael Niedermayer wrote:
> > On Fri, Nov 21, 2008 at 09:15:30PM +0100, Vitor Sessak wrote:
> >> David Geldreich wrote:
> >>> Hello Guillaume,
> >>>
> >>> Le 21 nov. 08 ? 17:25, Guillaume POIRIER a ?crit :
> >>>
> >>>> This is not the way to go. Reg tests pass on AMD64/Linux, so the code
> >>>> must be fixed to work the same on any plateform. The md5sum should not
> >>>> match X plateform results, but all plateforms result.
> >>> That's why I made another post to tell to ignore my proposed patch.
> >>>
> >>> I found no way of making sin/sinf works the same way on all the  
> >>> platform. In my case, OSX ppc and intel gives different results.
> >>>
> >>> So changes r15261 and r14982 are incomplete ... they correct the  
> >>> problem for AMD64 but breaks in on Intel32.
> >>>
> >>> We must iterate to find a "stable" sine window generating function.
> >> Even if we find a way to generate a sine window in an arch-independent 
> >> way, the codec still uses floating points in other places, so if it ever 
> >> is bit-identical across PPC and I32, I don't see any reason not to see a 
> >> different output when testing on ARM or SH or GCC 6.4 or whatever we'll 
> >> encounter in future. Unless someone tells me why it is supposed to work 
> >> as is, I think that this test should be removed...
> > 
> > ratecontrol in video uses floats, and other parts do too, we arent seeing
> > problems with these and arent disabling them
> > 
> > If you argue that the wma test should be disabled because it is not
> > matching between some important systems, thats something i can understand
> > 
> > but, arguing that itz should be disabled because it might theoretically not
> > work on some architecture or not yet existing compiler is well ...
> Ok, I didn't made myself clear. It is not just that it might 
> theoretically not work, but I see no reason it is more likely to work 
> than not in a completely valid system, and I don't find this acceptable. 

And thats where i cant follow your "logic".
What you suggest is strictly speaking to disable a test because it may or
lets even say it likely would not work in some "future" case.

Regression tests exist primarely to allow developers to test their changes,
find issues early, find out which part exactly broke, ...

Now if you drop a test you totally loose the protection this test has

But let me use a even more extreem case to make clear what i mean
lets assume every single regression test failes on every single system
except x86 and ppc.
the 99% of development that is done is done on ppc & x86, it would use
and benefit from the test like always.
the <1% of development done on other platforms would not be able to use
the regression tests

Now compare this to your suggestion of removing such tests because
they fail on some system
99% of the development suddenly looses the functionality of the regression
1% gains nothing, broken tests or no tests is pretty much the same.

And the problematic regression tests surely could be split out, but
disabling them for the 99% of developers for whom they work fine
is not a good idea IMHO

Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No great genius has ever existed without some touch of madness. -- Aristotle
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20081122/2acda042/attachment.pgp>

More information about the ffmpeg-devel mailing list