[FFmpeg-user] Concat input protocol: more files, more missing streams

[Thierry Lelégard]

Le 14/09/2013 18:26, Thierry Lelégard a écrit :
 > Le 14/09/2013 18:11, Paul B Mahol a écrit :
 >> On 9/14/13, Thierry Lelegard <thierry.lelegard at free.fr> wrote:
 >>> [....]
 >>> The tests are made with 10 MB as probe size. But using any other size, from
 >>> the default size to 100 MB, does not change anything.
 >>> [....]
 >>
 >> Perhaps you actually need -analyzeduration ?

 > I thought that -probesize and -analyzeduration were two different scales for
 > doing the same thing, one in file size and one in media playout duration.
 > Am I wrong ?
 >
 > Anyway, I tried -analyzeduration 1000M, 1000 seconds while the default is 5,
 > and nothing changed.
 >
 > And remember that the problem is that the number of detected input streams
 > decreases when the input file size increases, with a constant probe size and the
 > same beginning of media content. So, the analyzed data should be the same in
 > all cases. Am I wrong?
 >
 > -Thierry

Apparently, to be efficient, both -probesize and -analyzeduration shall be
specified. Using only one of them has no effect. I did the following tests:

ffprobe VIDEO_TS.mpeg -analyzeduration 500M
  -> finds 5 streams, instant response

ffprobe VIDEO_TS.mpeg -probesize 300M
  -> finds 5 streams, instant response

ffprobe VIDEO_TS.mpeg -probesize 300M -analyzeduration 500M
  -> finds 17 streams, long time before responding

So, specifying only one of the two options seems useless. I should have realized
that before since ffprobe was too fast to respond, meaning that not much of the
file was read.

Could someone explain in depth the actual *relationship* and *dependency* between
the two options?

The documentation is pretty vague and lead to confusion:

probesize integer (input)

   Set probing size in bytes, i.e. the size of the data to analyze to get stream information.
   A higher value will allow to detect more information in case it is dispersed into the stream,
   but will increase latency. Must be an integer not lesser than 32. It is 5000000 by default.

analyzeduration integer (input)

   Specify how many microseconds are analyzed to probe the input. A higher value will allow to
   detect more accurate information, but will increase latency. It defaults to 5,000,000
   microseconds = 5 seconds.

Reading that, I can easily conclude that these are two distinct ways of indicating how much
of the file shall be analyzed.

In fact, you need to specify both values in a consistent way. That is to say you need to
specify a -probesize which is in the same order of magnitude than the -analyzeduration.
Quite easy if you know the average bitrate of the file, or the file playout duration.
Which can be found using ... ffprobe ! So, to build an efficient ffprobe command, you
need information from ... the ffprobe command you try to build !

The second issue is "how much probe size / duration should I specify?". For an automated
tool which needs to build a command which will work all the time, what should I specify?

For instance, my test file has an average bitrate of 5.55 Mb/s. I tried the various
consistent combinations of probe size / duration:

ffprobe VIDEO_TS.mpeg -probesize 300M -analyzeduration 500M
ffprobe VIDEO_TS.mpeg -probesize 250M -analyzeduration 400M
ffprobe VIDEO_TS.mpeg -probesize 150M -analyzeduration 200M
  -> finds 17 streams, long time before responding

ffprobe VIDEO_TS.mpeg -probesize 70M -analyzeduration 100M
  -> finds 14 streams, long time before responding

Below 200 seconds of analysis, streams are missing. One should argue that this depends
on the time of first appearance of the last stream. But I have strong evidences that
this is wrong. My sample DVD is a classical movie with 10 subtitle tracks in 10
languages. The first subtitle appears at the first dialogue in English, which means that
all subtitle tracks appears at the same time (or within a few seconds interval) since
all viewers in all languages want to known what the actors say, right?

What I notice is that the number of detected subtitle tracks progressively decreases
when the probe size / duration decreases. There is no threshold where all subtitle
tracks disappear. So, the (rational) explanation that ffprobe detects a stream when
it sees its first packet is wrong because it is in contradiction with what I observe.

In short, there must be some kind of bug around there but I cannot clearly
categorize it.

Good night all, I spent too much time on ffmpeg ;-)
-Thierry