[FFmpeg-user] Fwd: Understanding/preventing buffer overruns

[Simon Roberts]

> > [...snip, mostly successful scenario describing simultaneous recording
> > of four video channels...]
> >
> >> But... I get almost continuous buffer overruns after about a minute.
> >> Can someone tell me where to start learning about how I might fix this
> >> (I presume it can't be good that it's doing this even though the
> >> output seems to be OK, right?)
> > [...snip...]
> >> Here's a sample of the messages during recording (which also repeat
> >> essentially continuously)
> >>
> >> [decklink @ 0x55b2249e8580] Decklink input buffer overrun!
> >>     Last message repeated 1 times
> >> [decklink @ 0x55b2249a4040] Decklink input buffer overrun!944kB
> >> time=00:00:52.30 bitrate=549308.8kbits/s dup=35 drop=4048 speed=0.999x
> >> [decklink @ 0x55b2249e8580] Decklink input buffer overrun!
> >>     Last message repeated 13 times
> >
> > Poking around, it seems the universal answer to all questions of this
> > kind is "your computer can't handle the workload". However, I'm
> > absolutely certain this is not the case for me since if I run a single
> > ffmpeg process to capture these four video plus four audio channels, I
> > get high rates of buffer overrun reports, but, if I run five *separate
> > but simultaneous* ffmpeg processes, each one capturing/compressing a
> > single video stream, and a fifth that captures the audio (which is
> > actually coming from a four channel USB device), the recordings
> > proceed at 1x, with no buffer overruns, and no complaints of any kind
> > at the default logging level. Also, the CPU load during this
> > five-ffmpegs-at-once totals just about one-half the total available
> > CPU power of the machine.
>
> Output IO speed can also limit the ability of the process to read the
> input fast enough.

Undoubtedly true, but hypothesis that such a problem is the cause of
my observations is inconsistent with the machine's demonstrated
ability to capture, compress, and write these streams into five files,
on the same storage medium, concurrently, simply by using five
concurrent ffmpeg processes. This hardware performs the work
successfully, but when ffmpeg tries to do it as a single process (with
many threads), it gets into trouble (or at least reports that it
does--I'm still not sure if the warning messages actually represent a
real problem or not).

As a side note, I'm writing to M.2 PCIe storage that can sink 2.4
GBytes/second continuously (yes, I did run this for long enough to
overflow the caches many times over), and the output of these captures
don't come even close; the total bandwidth for them runs less than 90
MBytes/second.

> > The only real downside to this approach that I perceive is that I have
> > a tiny (perhaps one or two frames??) synchronization error between the
> > relative files, but it seems to indicate that perhaps ffmpeg's
> > internal buffer management might either allow control that I don't
> > know about that would fix this (I'd love to hear about it if this is
> > the case) or that there is a significant opportunity for improving the
> > tool in respect of higher performance modern machines and this kind of
> > capture (I'm only capturing standard 1080/30p and increasingly it
> > seems that 4K is growing in popularity.)
>
> 1-2 frames difference is unavoidable, even if you capture multiple inputs
> in the same process, syncrhronized capture is not guaranteed at all.

Fair enough. Good to know; I shan't waste time trying to pursue that further.

> > So, to that end, I now have two new questions (unless someone can tell me how to
> > tune the buffers or whatever and make this work well as a single process)
> >
> > 1) how would I report this to the developers (the ffmpeg-devel list
> > seems to say that it's for discussion of development, not for
> > reporting issues/bugs/rfes)?
>
> This is mailing list is the right place. Or trac.ffmpeg.org if you are
> confident this is a bug (I think it is not).

I agree that it's not a bug. It's an opportunity for enhancement.
ffmpeg is *functionally* correct in all respects related to the
captures that I'm performing. There is, however, a significant
opportunity for improving the *performance* (or more precisely the
capacity/throughput).

It might be that there's tuning available at the command line that can
rectify this, but nothing has been proposed to this point, and, as I
mentioned, every other question on these lines that I've seen has
ended with "your hardware can't do that much work". I have
demonstrated that in this situation, that is not the case; my hardware
has demonstrated that it *can* do it, end-to-end.

So, is this the right place for an enhancement request/proposal? Is
there a standard form for the email header?

trac.ffmpeg.org Thanks. this looks like I shall need to poke around in
the developer docs there and see if I can begin to understand how this
thing's architecture hangs together.

> > 2) What pointers for how I might be able to participate in trying to
> > work on the improvement?
>
> Complete command line you are using and the console output is a good
> start.

I was unclear, I'm contemplating whether I might try to contribute a
fix to the source code, to put some effort into improving ffmpeg.

However, this is the command that generates buffer overruns:

ffmpeg  -f decklink -i 'DeckLink Quad HDMI Recorder (1)'   -f decklink
-raw_format argb -i 'DeckLink Quad HDMI Recorder (4)'   -f decklink -i
'DeckLink Quad HDMI Recorder (2)'   -f decklink -raw_format argb -i
'DeckLink Quad HDMI Recorder (3)' -f alsa -channels 4 -i plughw:1
-map 0:v -map 1:v -map 2:v -map 3:v   -r 30 -c:v prores_ks -profile:v
3 -q:v 4 -channels 4  -map 4:a -channel_layout quad -c:a copy
-metadata title=DeckLinkCapture  -y  capture-4chan-audiocopy.mov

Splitting this into five separate commands, generating five separate
output files, running them in background simultaneously, works without
any overruns, dropped frames, or any other complaints.

> > On the latter, I have to admit that it's a longshot as, a) it's been
> > 25 years since I wrote any
> > C++ (I still code, but other languages), b) I would have to start
> > understanding this software from the ground up which would likely take
> > ages, and c) I have severely limited time that is "spare".
>
> As we all.

Ain't that the truth, and please know I appreciate everyone's input!