[FFmpeg-devel] [RFC] Flush API for libavfilter
Michael Niedermayer
michaelni at gmx.at
Tue Nov 29 20:09:39 CET 2011
On Tue, Nov 29, 2011 at 07:13:33PM +0100, Stefano Sabatini wrote:
> On date Monday 2011-10-24 19:12:57 +0200, Michael Niedermayer encoded:
> > On Mon, Oct 24, 2011 at 06:49:24PM +0200, Stefano Sabatini wrote:
> > > On date Thursday 2011-10-20 17:37:18 +0200, Michael Niedermayer encoded:
> > > > On Thu, Oct 20, 2011 at 05:06:21PM +0200, Stefano Sabatini wrote:
> > > > > On date Wednesday 2011-03-23 06:25:15 +0200, Mina Nagy Zaki encoded:
> > > > > > On Tuesday 22 March 2011 17:35:39 Stefano Sabatini wrote:
> > > > > [...]
> > > > > > > Check: cmdutils.c:get_filtered_audio_buffer(), it implements a pull
> > > > > > > model, correct me if I'm wrong.
> > > > > >
> > > > > > I have already checked that, which is what lead me to the question in the first
> > > > > > place. I can't really read AVERROR_EOF from request_frame() because audio
> > > > > > filters don't(/shouldn't?) use it. get_filtered_audio_buffer() does a
> > > > > > request_frame() on aout which propagates all the way to asrc (without passing
> > > > > > through intermediate filters, because they don't/shouldn't use it) which then
> > > > > > does a filter_samples that propagates all the way to aout! If I try to
> > > > > > request_frame from the previous buffer it will reach asrc then eventually lead
> > > > > > to calling my own effect's filter_samples, so now do I filter in filter_samples or
> > > > > > request_frame :D And in any case, asrc never returns AVERROR_EOF anyway.
> > > > >
> > > > > Hi,
> > > > >
> > > > > I was trying to get the af_sox.c filter in shape when I stumbled
> > > > > across this.
> > > > >
> > > > > Resuming, the problem is that when a filterchain source returns
> > > > > AVERROR_EOF (e.g. becase the generated media has terminated) it is not
> > > > > possible to notify the intermediate filters, which may keep cached
> > > > > data which is never released.
> > > > >
> > > > > For example some sox filters cache data, which is supposed to be
> > > > > flushed through the drain API when *all* the input has been already
> > > > > provided (so they won't return data *at all* if such a mechanism is
> > > > > not implemented).
> > > > >
> > > > > This can't be done with the current API (neither with audio nor with
> > > > > video).
> > > >
> > > > I dont understand the problem for video
> > > >
> > > > At EOF
> > > > A filter currently can, when it receives a request_frame()
> > > > push the next cached frame out via
> > > > avfilter_start_frame + avfilter_draw_slice + avfilter_end_frame
> > > > vf_yadif does this currently for different reasons
> > > >
> > >
> > > > And a filter knows its at EOF when poll & request_frame() signal so
> > > > that is via 0 and AVERROR_EOF
> > >
> > > You need to implement poll() in the filter for applying this logic,
> > > and poll() may return 0 even when no end of file was reached
> > > (e.g. when the buffer source was emptied).
> > >
> > > >
> > > > The pull system is needed or more complex networks of filters will
> > > > have all kinds of issues.
> > > > one example is a graph with 2 inputs and 2 outputs, we assume
> > > > one wants to mux these 2 outputs together in the same file, the inputs
> > > > are from seperate files. With a pull system we know from which file
> > > > to read next with a push system we dont and could end up having to
> > > > buffer the whole decoded file in memory. Not to mention the delay
> > > > So i think audio should use request_frame() or a equivalent too
> > >
> > > I'm not proposing to drop request_frame() (which is currently
> > > implemented for audio).
> > >
> > > But in the case of sox a filter may release data all at once when the
> > > end of file is reached, and for doing this you need to know when the
> > > input data is terminated (sox API is meant for batch processing, so
> > > yes this will not be usable for real-time filtering).
> >
> > iam still not understanding the problem. Filters dont run in an
> > isolated environment, they get calls from outside
> >
> > for example data can be pushed into a filter, then it has data and
> > thus not EOF
> > or data can be pulled out of a filter via request_frame() and for
> > that a filter would calls its predecessors request_frame() and this
> > either would produce data or a AVERROR_EOF
> >
> > from an outside view when EOF is reached on the input nothing special
> > should happen, the pulling from the end would just continue until
> > a EOF error is pulled out.
> >
> > where is the flaw in my logic ?
>
> Sorry for the slow reply.
>
> With sox API, data is cached in the filter, and released only when a
> particular API is called (the so called drain API).
>
> Problem is, in order to flush this data you need to know that the
> input ended, or in other words you need to signal the sox filter that
> the input source reached EOF, and give a chance to the filter to flush
> all the cached data.
>
> Is this currently possible? No, what happened now is that
> request_frame() is recursively called on the current filter input. The
> filtergraph source will eventually return EOF, the sink filter is
> notified that the input reached EOF but the data cached in the
> intermediate filters is never released.
static int request_frame(AVFilterLink *link)
{
AVFilterContext *ctx = link->src;
soxctx->did_we_output_a_frame= 0;
do {
int ret;
if ((ret = avfilter_request_frame(link->src->inputs[0]))){
if(ret == EOF){
sox_drain()
if(soxctx->did_we_output_a_frame)
return 0;
}
return ret;
}
} while (!soxctx->did_we_output_a_frame);
return 0;
}
filter_samples(){
if we output something here
soxctx->did_we_output_a_frame=1;
}
>
> I see two mechanisms for dealing with this:
>
> 1. a filter may request frames and *force* flushing
> (avfilter_request_frame() + FLUSH flag). In this case all the filters
> are forced to flush cached data (via the drain API in case of libsox).
>
> 2. we may implement an avfilter_flush(), to be called on the filter
> output link, to force flushing the cached frames. This could be called
> by a source on all its output links when it reaches EOF.
>
> I'll try to come up with a proof of concept soon.
with:
soxfilter1 -> soxfilter2 ->out
you need to flush soxfilter1 first then normally filter with 2
and only once it runs dry flush it.
iam happy to implement the request_frame() based flushing if someone
provides a testcase that needs it
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Dictatorship naturally arises out of democracy, and the most aggravated
form of tyranny and slavery out of the most extreme liberty. -- Plato
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20111129/a054cd92/attachment.asc>
More information about the ffmpeg-devel
mailing list