[FFmpeg-devel] [PATCH 4/5] lavfi: drop the requirement that request_frame returns a frame.

Michael Niedermayer michael at niedermayer.cc
Thu Aug 27 03:10:18 CEST 2015

On Wed, Aug 26, 2015 at 04:46:21PM +0200, Nicolas George wrote:
> Le nonidi 9 fructidor, an CCXXIII, Paul B Mahol a écrit :
> > I'm interested.
> Glad to read it. So here it is:
> Foremost, I want to remove the recursiveness. Currently, when a frame is
> requested on buffersink, it recursively calls request_frame() on all filters
> in the chain until the source, and the source recursively calls
> filter_frame() until the sink. For a filter chain with N filters, that makes
> 2N nested function calls, this is bad for stack usage, and very annoying for
> debugging.
> First, I must make sure that request_frame() always calls a new
> ff_filter_link_close() function before returning EOF. Unlike
> avfilter_link_set_closed(), I intend to have it take a timestamp too, that
> would allow to fix the long-standing issue of the duration of the last
> frame.
> Then I will change ff_request_frame() to just set a flag on the link. The
> public API functions (mostly buffersink) will call a new function to
> repeatedly scan the graph for links with the flag and run the
> request_frame() callback when it is set. No more recursiveness for
> request_frame().
> Then I intend to add a FIFO directly in the link structure.
> ff_filter_frame() can just add the frame to FIFO, the next filter_frame()
> callback will be called by the loop that scans the graph, just like the
> request_frame() callback. No more recursiveness for filter_frame().
> (It leaves recursiveness for get_*_buffer(), but it is less of a problem.)
> These need to come first and in that order, and after that, new improvements
> become possible:
> Keep the list of filters that need a callback called in a more efficient
> data structure, to avoid scanning the whole graph each round.
> Merge the request_frame() and filter_frame() callbacks into a single
> per-filter callback, maybe activate() or something. That avoids separating
> EOF handling from normal filtering, and overall makes writing filters
> simpler. Of course, existing filters do not need to be changed, the
> framework can still call the legacy callbacks automatically; it would
> probably be able to satisfy the "needs_fifo" flag present on some filter.
> Allow filters to access the link FIFO directly. Instead of having one frame
> forcefully presented to it, the callback can access all the frames queued in
> all the inputs. This would supersedes FFBufQueue without requiring update
> for the corresponding filters.
> Keep statistics about the amount of frames and memory in the link FIFOs.
> With that, we can have a limit on the total memory used, and therefore avoid
> crashing the host for unschedulable graphs, but still have enough room for
> bursts of frames in a single link.
> Note that the last two points together should have the side effect of fixing
> the "Buffer queue overflow, dropping" that some valid filter graphs have.
> Make the filters run in threads. If the input and output properties of links
> are well separated, and the functions that manipulate them are thread-safe,
> the running the callbacks in a thread pool becomes very easy.
> In March I had started working on the same project, but I did not start by
> dropping the requirement addressed in this very patch, and it made things a
> lot more complex.

> If you have any remark or suggestion, it is of course welcome.

I think moving FIFOs into the links is a wise choice

there may be one problem though
if the parameters of Frames change and a link "contains" 5 frames
it is possible that not all these frames have the same size or
aspect ratio. Currently a link contains dimensions and aspect ratios
iam not sure what is the best solution or if this even matters ATM
or maybe is better to ignore for now and deal with at a later point
when the design is closer to finished and implemented.
One easy solution would be to split the values for pre and post fifo
dunno if that is a good or bad idea ...


Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

It is what and why we do it that matters, not just one of them.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20150827/9221ab22/attachment.sig>

More information about the ffmpeg-devel mailing list