[FFmpeg-devel] Running FATE

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> James Almer
> Sent: Thursday, 9 September 2021 22:07
> To: ffmpeg-devel at ffmpeg.org
> Subject: Re: [FFmpeg-devel] Running FATE
> 
> On 9/9/2021 5:04 PM, Soft Works wrote:
> >
> >
> >> -----Original Message-----
> >> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> >> Gyan Doshi
> >> Sent: Thursday, 9 September 2021 09:24
> >> To: ffmpeg-devel at ffmpeg.org
> >> Subject: Re: [FFmpeg-devel] Running FATE
> >>
> >>
> >>
> >> On 2021-09-09 11:03 am, Soft Works wrote:
> >>> Hi,
> >>>
> >>> I have a few questions regarding FATE:
> >>>
> >>> - Is it possible to run FATE in a way that it doesn't stop on
> >> error?
> >>
> >> make -k fate
> >>
> >>> - Is there a better way to exclude certain tests other than
> >> commenting
> >>>     them in the mak files?
> >>
> >> Don't enable/compile at least one of the dependencies of those
> tests.
> >>
> >>> - Is it possible to run just a specific test via command line?
> >>
> >> make fate-specific-test
> >>
> >> Regards,
> >> Gyan
> >
> > Hi,
> >
> > one more question I forgot to ask: is there a safe way to
> accelerate those
> > test runs?
> >
> > I had tried with something like '-j8', but this didn't produce
> > reliable results.
> > Without deeper investigation it seems that this causes tests to be
> run
> > before their dependencies have been built.
> 
> That shouldn't happen. -j# is the correct way to parallelize fate
> runs.
> Dependencies are always built before a test is run, regardless of job
> count. If they aren't, then there's a bug in the relevant Makefile
> recipe, like for example said dependencies not being correctly
> signaled.

It's consistent - running the following sequence:
(on Ubuntu)

make distclean
./configure
make fate -j8 SAMPLES=../fate-suite/
    => several test failing
make fate SAMPLES=../fate-suite/
    => all tests successful
make fate -j8 SAMPLES=../fate-suite/
    => now completes without failing tests


The following tests are affected:

flv-demux
gapless-mp3-side-data
oggopus-demux
png-frame-metadata
png-side-data
ts-demux
ts-opus-demux
ts-small-demux

They are all using ffprobe_demux at a time when ffprobe isn't built
yet:

./tests/fate-run.sh: 80: eval: /development/ffmpeg/ffprobe: not found


It seems that either demux.mak and gapless.mak need to be made dependent
on ffprobe or the 'ffprobe_demux' execution?

softworkz