[Ffmpeg-devel] New/extended filter API discussion

[Alexander Chemeris]

Hello,

On 1/6/07, Michael Niedermayer <michaelni at gmx.at> wrote:
> On Fri, Jan 05, 2007 at 05:23:19PM -0800, Aaron Williams wrote:
> > Alexander Chemeris wrote:
> why not provide some links to the relevant source code? or documentation for
> the respective filter APIs
AVIsynth documentation was available at www.avisynth.org, but for now site
is down:
http://www.avisynth.org

You could download sources and look into them directly :
http://sourceforge.net/project/showfiles.php?group_id=57023

Or browse Internet archive documentation:
http://web.archive.org/web/20060522030238/www.avisynth.org/FilterSDK

> and i do know the filter system VirtualDub used a few years ago (i dunno if
> they still use that) but it was a ridiculous joke, IIRC 2 fixed buffers, a
> single linear 1-in-1-out chain and filters 0,2,4,... used buffer A as input
> and B as output where filters 1,3,5,... used buffer B as input and A as
> output
I did not look at VDub api deeplyand do not remember details, but seems
that you're correct - filters are very rude in old VDub versions. However
its filter system is in the middle of redesign:
http://www.virtualdub.org/blog/pivot/entry.php?id=64
This article is also a good warning about some issues in wich we could
fall or not fall.

Yeah, it somewhat sad, that everybody design its own filter API, alone,
incompatible and awkward. This field really lack standardization.

> > > I have some thoughts about video filter API, because I'm thinking about
> > > very similar subsystem for sipX media processing library. So, when I
> > > thought
> > > about benefits and drawbacks of "push", "pull" and "process" models of
> > > opperation, I came to the view, that "process" approach is simplest among
> > > others while almost as powerful as "push" and "pull". By "process" model
> > > I mean following -- each filter have 'process()' function which simply
> > > take
> > > input frames, process them and push into output buffer. Most work in this
> > > approach should be done by flowgraph class itself - it knows how filters
> > > are connected and could take output frames from preceding filter and pass
> > > them to subsequent filter.
> > >
> > > One drawback of "push" and "pull" models is that they could not have
> > > multiple inputs (for "push") or outputs (for "pull"). Lets consider
> > > "push" model.
> > > If filter graph have two inputs and one of them pushes frame behaviour
> > > of the second input is ill defined -- it could be ignored, or pulled,
> > > or smth
> > > else, but I do not see good approach here.
>
> if your filter has 2 inputs and you receive a push from input #1 theres
> nothing ill defined, you simply are then in a state of having one input frame
> and waiting for the second push, when that comes in you can output your frame
> as push to the next filter
It may be done so. But, we could fall in many traps on this way, and not all of
them are obvious now. And this way require all inputs to have clock, which
is not always true - e.g. one input could read from file. However "passive"
inputs could be timed with "slave clock" attached to some real "master clock".
One more question, should answered - what if frames come with different
framerate? I think this could be leaved on filter's decision what to do
in this situation, but we shuld be aware of blocking of all graph if one input
is failed and do not produce frames.

BTW, what do you think about multithreaded filter graphs, where different
filters run in different threads.?

> > > The same for "pull" and
> > > multiple
> > > outputs.
>
> multiple outputs are always equivalent to several filters with single outputs
> so this is independant of the filter architecture wrong
Could you illustrate this, please? May be use different assumptions, but I
think this is impossible.

> > One final thought is that filters may also want to have the ability to
> > output data to another stream (i.e. a file) for logging or multi-pass
> > processing.  I.e. my current normalization code is quite crude, but I
> > need to feed back information to the next pass for actual volume adjustment.
>
> again this is trivial, no matter which filter system calling fopen in it
> can be done ... (it can be argues that thats ugly but thats another issue)
This may be done in sane way with "to file" filter, simply writing incoming
data to file and passing it unchanged to output.


-- 
Regards,
Alexander Chemeris.