[FFmpeg-soc] Libavfilter RFC: problem with ffmpeg.c/ffplay.c integration

[Bobby Bingham]

On Mon, 7 Jan 2008 23:07:19 +0100
Michael Niedermayer <michaelni at gmx.at> wrote:

> On Wed, Jan 02, 2008 at 10:00:26AM -0500, Bobby Bingham wrote:
> > On Wed, 02 Jan 2008 12:49:24 +0100
> > Vitor Sessak <vitor1001 at gmail.com> wrote:
> > 
> > > Hi all,
> > > 
> > > I've spotted a problem with the ffmpeg.c integration using the
> > > command
> > > 
> > > ffmpeg -vfilters fps=1 -i input.avi output.avi
> > > 
> > > In the way the integration is done in ffmpeg.c (my fault, I know),
> > > the filter infrastructure expects one input frame for every output
> > > frame. But that is not right for all filters. FFplay is also
> > > affected by this problem (although it's harder to reproduce, I
> > > didn't found a test case), it has a video queue and a filter will
> > > not work properly if the queue is empty.
> > > 
> > > The root of the problem is that in the way libavfilter is
> > > designed, libavfilter must ask ffmpeg.c/ffplay.c every time it
> > > wants a frame, no matter how (if any) frame was outputted by the
> > > filter chain.
> > > 
> > > I've thought in a few solutions:
> > > 
> > > Solution 1 (the closest to the actual code):
> > > 
> > > In the patch in soc svn, a input filter is defined in ffmpeg.c. A 
> > > solution would be to move the call to av_read_frame and 
> > > avcodec_decode_video to the request_frame method of the input
> > > filter, so it'll decode a video frame from the file each time the
> > > filter asks for one.
> > > 
> > > Problem: Messy, ugly to handle audio and -vcodec copy, makes
> > > ffmpeg.c even more complex than it is already, scare people away
> > > of using libavfilter in other applications.
> > > 
> > > Solution 2 (threading and queuing):
> > > 
> > > Make ffmpeg.c have a thread for audio decoding, another one for
> > > video and another for filtering. The video thread will add frames
> > > to a video queue and the filtering thread will sleep and wait for
> > > frames is the video queue is empty.
> > > 
> > > Problem: May introduce bugs, big patch, gives the impression that
> > > it is impossible to use libavfilter in a single-threaded
> > > application. There may also be corner cases where the queues gets
> > > too big and uses gigs of ram.
> > > 
> > > Solution 3 (change libavfilter):
> > > 
> > > Allow avfilter_request_frame to give a positive value if it
> > > outputs a frame, a negative one if error or end of video and zero
> > > if it has no frame to output be may have one latter. Then make a
> > > small video queue and just make the request_frame of the input
> > > filter return 0 if the queue is empty. The advantage is that
> > > it'll be easier to use it as a library and will not increase too
> > > much the complexity of ffmpeg.c/ffplay.c.
> > > 
> > > Problem: Every video filter will have to handle the case where
> > > the previous filter do not have a frame now but may have latter...
> > > 
> > > What is your opinion on that?
> > 
> > I'll take a closer look at this tonight, but I think solution 3
> > sounds reasonable.
> 
> As nothing has been commited yet to solve this IIRC ...
> 
> what about
> 
>     /**
>      * Frame poll callback.  This returns the number of immedeately
> available
>      * frames. That is if it returns 1 or larger than the next
> request_frame()
>      * is guranteed to return one frame (with no delay)
>      *
>      * Output video pads only.
>      */
>     int (*poll_frame)(AVFilterLink *link);
> 
> with that we could just check each output stream (there could be more
> than 1) if a frame could be output. And if yes run a request_frame()
> on that. If no stream has a >0 return for poll_frame() than another
> input packet could be demuxed and decoded (from a file which caused a
> poll()==0) and send it to one of the input vf_filo
> 
> poll_frame() could have a trivial default implementation which just
> polls its input filters and returns that, this would be sufficient
> for 90% of filters.
> 
> [...]

Consider a filter which needs to check the timestamp or such of the
next frame to determine whether or not to output it.  This will require
it to request the frame in poll_frame and either drop it or buffer it
until it's requested by the next filter, so you can't do slice
rendering.

The idea to use the return value of request_frame avoids this.

-- 
Bobby Bingham
Never trust atoms.  Or anything made of atoms.
このメールは再利用されたバイトでできている。