[FFmpeg-devel] Politics

[Calvin Walton]

On Mon, 2021-12-13 at 14:32 -0500, Calvin Walton wrote:
> like, for example, I rewrote the "fps" filter to use the activate
> callback to handle filling in large timestamp gaps a frame at a time
> rather than buffering a ton of frames all at once and causing OOM. But
> that only fixed the situation in files with single streams.
> 
> I believe that a more general solution is needed to handle working with
> multiple "synchronized" streams - i.e. produced from the same demuxer -
> where one stream can have large timestamp gaps which others do not.

Incidentally, the "fps" filter is probably one of the simplest examples
of a filter that does not involve subtitles which would benefit from a
"heartbeat" mechanism.

Currently in order to be able to output a frame, the fps filter needs
to have 2 frames buffered (or alternately, one frame and a timestampped
status like EOF). In the case of a long timestamp gap, this means that
it will produce no output frames until such time as the next frame is
read, meaning that downstream filters will be stalled, and if there are
any multi-input filters downstream, one of the inputs might end up with
too many frames in its input fifo.

If the fps filter could receive periodic activations with an indication
that "pts has increased on the input without any new frame available",
it could incrementally fill in the gap in realtime, allowing downstream
filters to proceed.

-- 
kepstin
Calvin Walton <calvin.walton at kepstin.ca>