[FFmpeg-user] FATE tests: FFV1

[Carl Eugen Hoyos]

Peter B. <pb <at> das-werkstatt.com> writes:

> I was surprised to see that the tests of other lossless codecs (in
> "lossless-video.mak") seem to only test decoding (e.g. Lagarith) for a
> handful of samples. It seems that only popular sample use-cases are
> tested, or for some (eg ffvhuff), it's even just encoded/decoded once.

Patch welcome!

> I was thinking about also doing only the FATE-tests done in e.g.
> Lagarith, but I've already found incompatibilities of FFV1.3 in libav
> for certain pix_fmts which would not have been tested if I had sticked
> to this very limited testset.

1) Please use "avconv", the other name was chosen with 
the intent to be as misleading as possible.

2) I smell surprise in the paragraph you wrote: Please 
feel free to test a (not too small) number of fixed 
FFmpeg tickets with avconv, you will find that there are 
several hundred regressions with avconv, don't you agree 
that it would have been more of a surprise if there were 
none in the FFV1 codec?

> ---------------------------------
> 2) Generate samples on the fly or add them to fate-suite?
> ---------------------------------
> With the help of LCOV, I tried to target as many different use cases as
> possible - including also the not-so-popular ones.
> Almost all files are now generated, so fate-suite impact is very little,
> but this means that the tests add computation time, each time FATE is
> ran completely.

This is acceptable as long as only a few frames are encoded 
(as many as necessary).

> Furthermore, I saw that encoding tests (e.g. in "vcodecs.mak") for
> lossless tests would actually require to have its own raw source file
> for every tested parameter, like pix_fmt or resolution.
> Resizing on the fly, or forcing the source to a different pix_fmt
> results in "MAXDIFF > 0".

I may misunderstand but imo this is just a question of 
the options used.

Of course creating artificial source files in different 
pix_fmts would be welcome!

Please note that all these questions are development-related 
(particularly as you intend to send patches), so the 
discussion should better take place at ffmpeg-devel (which 
by the way is an open mailing list, suited for open source 
development...)

Carl Eugen