[FFmpeg-soc] [soc] lavfi audio visualization filter

[S.N. Hemanth Meenakshisundaram]

On 07/16/2010 06:53 AM, Aurelien Jacobs wrote:
> On Fri, Jul 16, 2010 at 01:44:27AM -0700, S.N. Hemanth Meenakshisundaram wrote:
>   
>> [...]
>>
>> 2. For audio visualization systems, how can I access the video
>> thread to display the visualizations.
>>     
> Ouch ! You don't want to access the video thread !
> What you want is to create a simple filter which take one audio input
> stream and generate one video output stream.
> For now it would be nice to implement (copy?) the same visualization
> effects that the ones ffplay supports.
> Then the visualization effects code could be remove from ffplay, and it
> could be replace by building a filter graph which would split audio in
> two identical streams. One stream would be feeded to a visualization
> filter and the other one would just passthru to the end of the filter
> graph.
>   

Hi All,

This is what I am trying to write now:


i/p audio filter --------> af-split ----------> o/p audio filter
                                       |
                                       ------------------> af_viz
----------> o/p visualization filter

When af_viz (or equivalent) is specified as part of -af option, the
framework sets up af_split and o/p viz filter.

af_viz has an input pad of type audio and o/p pad of type video and will
ouput video frames in YUV like any video filter.

o/p visualization filter will manage the queue of frames from af_viz.

A function replacing the video_audio_display function in ffplay.c will
call request_frame on the o/p viz filter and prepare it for SDL or
whatever else is used in the future for video output.


One issue is the formats of the af_viz. For the moment, I am giving it
the default query_formats (support all formats) behavior, so that when
avfiltergraph checks for common formats, there are no issues. Error
handling would have to be within the filter - i.e. if af_viz doesn't get
audio in a format it supports, it should complain etc.

Any comments on the design, suggestions?

Regards,