[FFmpeg-devel] A new FATE

Måns Rullgård mans
Fri Jul 30 12:13:02 CEST 2010


Mike's FATE system has done a great job over the last few years.  It
is however beginning to prove inadequate in various ways:

- Test specs can only be updated by Mike.  This is frustrating when an
  update is needed and Mike isn't around.
- Test specs are not versioned along with FFmpeg.
- No support for inexact decoders (usually floating-point audio).
- No support for parallel testing on multi-core systems.
- Only Mike can add or update systems and compilers.
- Requires non-standard tools on test systems.
- Backend and web interface are not open source.

To address the above-mentioned issues, I have been working on a
replacement system which is now ready to be announced.  These are the
main features of the new FATE:

- Test specs are part of the ffmpeg repo.  They are thus properly
  versioned, and any developer can update them as needed.
- Support for inexact tests.
- Parallel testing on multi-core systems.
- Anyone registered with FATE can add systems.
- Client side entirely in POSIX shell script and GNU make.
- Open source backend and web interface.
- Client and backend entirely decoupled.
- Anyone can contribute patches.

This is not vapour-ware.  It is already live at http://fate.ffmpeg.org/

Everything required on the client side is in FFmpeg svn, and the
source code for the server side is available from git at
http://git.mansr.com/?p=fateweb or git://git.mansr.com/fateweb

The observant will notice that this page reports many more tests than
Mike's page does.  This is due to two reasons: 1) each test in the
regression test set is reported separately, 2) a number of tests have
been added which are not in the old system.

The client configuration still needs to be documented, and there are
certainly improvements possible on the server side.  Nonetheless, what
it provides is enough to start using it.  Improvements can be made
incrementally as need arises.

Finally, I'd like to thank Mike for all the work he has done on the
test system.  The automated testing has time and again identified
issues, particularly on the "special needs" systems, which might
otherwise have gone unnoticed.  The time has now come to make this a
proper team effort and ease the maintenance burden for Mike, who is
clearly struggling to find the time keeping the system in shape
requires.

-- 
M?ns Rullg?rd
mans at mansr.com



More information about the ffmpeg-devel mailing list