[FFmpeg-soc] Libavfilter RFC: problem with ffmpeg.c/ffplay.c integration

[Michael Niedermayer]

On Wed, Jan 02, 2008 at 10:00:26AM -0500, Bobby Bingham wrote:
> On Wed, 02 Jan 2008 12:49:24 +0100
> Vitor Sessak <vitor1001 at gmail.com> wrote:
> 
> > Hi all,
> > 
> > I've spotted a problem with the ffmpeg.c integration using the command
> > 
> > ffmpeg -vfilters fps=1 -i input.avi output.avi
> > 
> > In the way the integration is done in ffmpeg.c (my fault, I know),
> > the filter infrastructure expects one input frame for every output
> > frame. But that is not right for all filters. FFplay is also affected
> > by this problem (although it's harder to reproduce, I didn't found a
> > test case), it has a video queue and a filter will not work properly
> > if the queue is empty.
> > 
> > The root of the problem is that in the way libavfilter is designed, 
> > libavfilter must ask ffmpeg.c/ffplay.c every time it wants a frame,
> > no matter how (if any) frame was outputted by the filter chain.
> > 
> > I've thought in a few solutions:
> > 
> > Solution 1 (the closest to the actual code):
> > 
> > In the patch in soc svn, a input filter is defined in ffmpeg.c. A 
> > solution would be to move the call to av_read_frame and 
> > avcodec_decode_video to the request_frame method of the input filter,
> > so it'll decode a video frame from the file each time the filter asks
> > for one.
> > 
> > Problem: Messy, ugly to handle audio and -vcodec copy, makes ffmpeg.c 
> > even more complex than it is already, scare people away of using 
> > libavfilter in other applications.
> > 
> > Solution 2 (threading and queuing):
> > 
> > Make ffmpeg.c have a thread for audio decoding, another one for video 
> > and another for filtering. The video thread will add frames to a
> > video queue and the filtering thread will sleep and wait for frames
> > is the video queue is empty.
> > 
> > Problem: May introduce bugs, big patch, gives the impression that it
> > is impossible to use libavfilter in a single-threaded application.
> > There may also be corner cases where the queues gets too big and uses
> > gigs of ram.
> > 
> > Solution 3 (change libavfilter):
> > 
> > Allow avfilter_request_frame to give a positive value if it outputs a 
> > frame, a negative one if error or end of video and zero if it has no 
> > frame to output be may have one latter. Then make a small video queue 
> > and just make the request_frame of the input filter return 0 if the 
> > queue is empty. The advantage is that it'll be easier to use it as a 
> > library and will not increase too much the complexity of
> > ffmpeg.c/ffplay.c.
> > 
> > Problem: Every video filter will have to handle the case where the 
> > previous filter do not have a frame now but may have latter...
> > 
> > What is your opinion on that?
> 
> I'll take a closer look at this tonight, but I think solution 3 sounds
> reasonable.

As nothing has been commited yet to solve this IIRC ...

what about

    /**
     * Frame poll callback.  This returns the number of immedeately available
     * frames. That is if it returns 1 or larger than the next request_frame()
     * is guranteed to return one frame (with no delay)
     *
     * Output video pads only.
     */
    int (*poll_frame)(AVFilterLink *link);

with that we could just check each output stream (there could be more than 1)
if a frame could be output. And if yes run a request_frame() on that. If no
stream has a >0 return for poll_frame() than another input packet could be
demuxed and decoded (from a file which caused a poll()==0) and send it to one
of the input vf_filo

poll_frame() could have a trivial default implementation which just polls
its input filters and returns that, this would be sufficient for 90% of
filters.

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Everything should be made as simple as possible, but not simpler.
-- Albert Einstein
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-soc/attachments/20080107/a31ae1ee/attachment.pgp>