[FFmpeg-devel] [RFC] Flush API for libavfilter
Michael Niedermayer
michaelni at gmx.at
Wed Nov 30 16:12:27 CET 2011
On Tue, Nov 29, 2011 at 08:09:39PM +0100, Michael Niedermayer wrote:
> On Tue, Nov 29, 2011 at 07:13:33PM +0100, Stefano Sabatini wrote:
> > On date Monday 2011-10-24 19:12:57 +0200, Michael Niedermayer encoded:
> > > On Mon, Oct 24, 2011 at 06:49:24PM +0200, Stefano Sabatini wrote:
> > > > On date Thursday 2011-10-20 17:37:18 +0200, Michael Niedermayer encoded:
> > > > > On Thu, Oct 20, 2011 at 05:06:21PM +0200, Stefano Sabatini wrote:
> > > > > > On date Wednesday 2011-03-23 06:25:15 +0200, Mina Nagy Zaki encoded:
> > > > > > > On Tuesday 22 March 2011 17:35:39 Stefano Sabatini wrote:
> > > > > > [...]
> > > > > > > > Check: cmdutils.c:get_filtered_audio_buffer(), it implements a pull
> > > > > > > > model, correct me if I'm wrong.
> > > > > > >
> > > > > > > I have already checked that, which is what lead me to the question in the first
> > > > > > > place. I can't really read AVERROR_EOF from request_frame() because audio
> > > > > > > filters don't(/shouldn't?) use it. get_filtered_audio_buffer() does a
> > > > > > > request_frame() on aout which propagates all the way to asrc (without passing
> > > > > > > through intermediate filters, because they don't/shouldn't use it) which then
> > > > > > > does a filter_samples that propagates all the way to aout! If I try to
> > > > > > > request_frame from the previous buffer it will reach asrc then eventually lead
> > > > > > > to calling my own effect's filter_samples, so now do I filter in filter_samples or
> > > > > > > request_frame :D And in any case, asrc never returns AVERROR_EOF anyway.
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I was trying to get the af_sox.c filter in shape when I stumbled
> > > > > > across this.
> > > > > >
> > > > > > Resuming, the problem is that when a filterchain source returns
> > > > > > AVERROR_EOF (e.g. becase the generated media has terminated) it is not
> > > > > > possible to notify the intermediate filters, which may keep cached
> > > > > > data which is never released.
> > > > > >
> > > > > > For example some sox filters cache data, which is supposed to be
> > > > > > flushed through the drain API when *all* the input has been already
> > > > > > provided (so they won't return data *at all* if such a mechanism is
> > > > > > not implemented).
> > > > > >
> > > > > > This can't be done with the current API (neither with audio nor with
> > > > > > video).
> > > > >
> > > > > I dont understand the problem for video
> > > > >
> > > > > At EOF
> > > > > A filter currently can, when it receives a request_frame()
> > > > > push the next cached frame out via
> > > > > avfilter_start_frame + avfilter_draw_slice + avfilter_end_frame
> > > > > vf_yadif does this currently for different reasons
> > > > >
> > > >
> > > > > And a filter knows its at EOF when poll & request_frame() signal so
> > > > > that is via 0 and AVERROR_EOF
> > > >
> > > > You need to implement poll() in the filter for applying this logic,
> > > > and poll() may return 0 even when no end of file was reached
> > > > (e.g. when the buffer source was emptied).
> > > >
> > > > >
> > > > > The pull system is needed or more complex networks of filters will
> > > > > have all kinds of issues.
> > > > > one example is a graph with 2 inputs and 2 outputs, we assume
> > > > > one wants to mux these 2 outputs together in the same file, the inputs
> > > > > are from seperate files. With a pull system we know from which file
> > > > > to read next with a push system we dont and could end up having to
> > > > > buffer the whole decoded file in memory. Not to mention the delay
> > > > > So i think audio should use request_frame() or a equivalent too
> > > >
> > > > I'm not proposing to drop request_frame() (which is currently
> > > > implemented for audio).
> > > >
> > > > But in the case of sox a filter may release data all at once when the
> > > > end of file is reached, and for doing this you need to know when the
> > > > input data is terminated (sox API is meant for batch processing, so
> > > > yes this will not be usable for real-time filtering).
> > >
> > > iam still not understanding the problem. Filters dont run in an
> > > isolated environment, they get calls from outside
> > >
> > > for example data can be pushed into a filter, then it has data and
> > > thus not EOF
> > > or data can be pulled out of a filter via request_frame() and for
> > > that a filter would calls its predecessors request_frame() and this
> > > either would produce data or a AVERROR_EOF
> > >
> > > from an outside view when EOF is reached on the input nothing special
> > > should happen, the pulling from the end would just continue until
> > > a EOF error is pulled out.
> > >
> > > where is the flaw in my logic ?
> >
> > Sorry for the slow reply.
> >
> > With sox API, data is cached in the filter, and released only when a
> > particular API is called (the so called drain API).
> >
> > Problem is, in order to flush this data you need to know that the
> > input ended, or in other words you need to signal the sox filter that
> > the input source reached EOF, and give a chance to the filter to flush
> > all the cached data.
> >
> > Is this currently possible? No, what happened now is that
> > request_frame() is recursively called on the current filter input. The
> > filtergraph source will eventually return EOF, the sink filter is
> > notified that the input reached EOF but the data cached in the
> > intermediate filters is never released.
>
> static int request_frame(AVFilterLink *link)
> {
> AVFilterContext *ctx = link->src;
> soxctx->did_we_output_a_frame= 0;
>
> do {
> int ret;
>
> if ((ret = avfilter_request_frame(link->src->inputs[0]))){
> if(ret == EOF){
> sox_drain()
> if(soxctx->did_we_output_a_frame)
> return 0;
> }
> return ret;
> }
> } while (!soxctx->did_we_output_a_frame);
>
> return 0;
> }
>
> filter_samples(){
>
> if we output something here
> soxctx->did_we_output_a_frame=1;
> }
>
>
>
> >
> > I see two mechanisms for dealing with this:
> >
> > 1. a filter may request frames and *force* flushing
> > (avfilter_request_frame() + FLUSH flag). In this case all the filters
> > are forced to flush cached data (via the drain API in case of libsox).
> >
> > 2. we may implement an avfilter_flush(), to be called on the filter
> > output link, to force flushing the cached frames. This could be called
> > by a source on all its output links when it reaches EOF.
> >
> > I'll try to come up with a proof of concept soon.
>
> with:
> soxfilter1 -> soxfilter2 ->out
>
> you need to flush soxfilter1 first then normally filter with 2
> and only once it runs dry flush it.
also with filters that mix video and audio links there may be problems
with mixing different APIs for flushing.
I mean for example a filter that has video output and audio input
wont recive audio specific calls on its output side.
another problem i see is that a filter that has 2 inputs
and one output, the output cannot know when to flush.
depending on the kind of filter flush may be needed when any inputs
are EOF or it may be needed only when all inputs are EOF
Only the filter itself knows when it hits EOF ...
another example would be
in->truncate->sox->out
in is a life stream without EOF
the truncate filter would reach EOF at 10 second (maybe it stops based
on detecting commercials or maybe it detects silence as a stop
criterion)
when would you call flush here?
with request_frame, out would call request_frame of sox which in
turn would call truncates request_frame which would say EOF and
sox would then call drain(), return its last samples.
and on the next run the same would happen but as drain would produce
nothing sox would return EOF too, ending playback
>
> iam happy to implement the request_frame() based flushing if someone
> provides a testcase that needs it
more precissely i need some means to test, the ffplay / ffmpeg patches
from minas branch dont apply anymore
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Incandescent light bulbs waste a lot of energy as heat so the EU forbids them.
Their replacement, compact fluorescent lamps, much more expensive, dont fit in
many old lamps, flicker, contain toxic mercury, produce a fraction of the light
that is claimed and in a unnatural spectrum rendering colors different than
in natural light. Ah and we now need to turn the heaters up more in winter to
compensate the lower wasted heat. Who wins? Not the environment, thats for sure
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20111130/61e926cc/attachment.asc>
More information about the ffmpeg-devel
mailing list