[FFmpeg-soc] Libavfilter RFC: problem with ffmpeg.c/ffplay.c integration

[Vitor Sessak]

Hi all,

I've spotted a problem with the ffmpeg.c integration using the command

ffmpeg -vfilters fps=1 -i input.avi output.avi

In the way the integration is done in ffmpeg.c (my fault, I know), the 
filter infrastructure expects one input frame for every output frame. 
But that is not right for all filters. FFplay is also affected by this 
problem (although it's harder to reproduce, I didn't found a test case), 
it has a video queue and a filter will not work properly if the queue is 
empty.

The root of the problem is that in the way libavfilter is designed, 
libavfilter must ask ffmpeg.c/ffplay.c every time it wants a frame, no 
matter how (if any) frame was outputted by the filter chain.

I've thought in a few solutions:

Solution 1 (the closest to the actual code):

In the patch in soc svn, a input filter is defined in ffmpeg.c. A 
solution would be to move the call to av_read_frame and 
avcodec_decode_video to the request_frame method of the input filter, so 
it'll decode a video frame from the file each time the filter asks for one.

Problem: Messy, ugly to handle audio and -vcodec copy, makes ffmpeg.c 
even more complex than it is already, scare people away of using 
libavfilter in other applications.

Solution 2 (threading and queuing):

Make ffmpeg.c have a thread for audio decoding, another one for video 
and another for filtering. The video thread will add frames to a video 
queue and the filtering thread will sleep and wait for frames is the 
video queue is empty.

Problem: May introduce bugs, big patch, gives the impression that it is 
impossible to use libavfilter in a single-threaded application. There 
may also be corner cases where the queues gets too big and uses gigs of ram.

Solution 3 (change libavfilter):

Allow avfilter_request_frame to give a positive value if it outputs a 
frame, a negative one if error or end of video and zero if it has no 
frame to output be may have one latter. Then make a small video queue 
and just make the request_frame of the input filter return 0 if the 
queue is empty. The advantage is that it'll be easier to use it as a 
library and will not increase too much the complexity of ffmpeg.c/ffplay.c.

Problem: Every video filter will have to handle the case where the 
previous filter do not have a frame now but may have latter...

What is your opinion on that?

-Vitor