[FFmpeg-devel] [RFC] AVFilter Parser

[Vitor Sessak]

Hi

Michael Niedermayer wrote:
> Hi
> 
> Well, after various flames, discussions, some implementation and so on.
> Heres yet another proposal, this time mostly properly defined, i hope this
> will avoid flames and misuderstandings.
> This one is quite similar to the previous proposals, the main difference
> besides a clear definition of things is the feedback/feedforward syntax.
> The new syntax avoids the often cited problem with string insertion of
> predefined filter graphs and name colissions.

What I really don't get about the string insertion problem is the idea 
that user-defined filters should be implemented as string substitution. 
More on that later.

> 
> ------------------------
> 
> filter_id:
>     any alpha numeric string identifying a filter
> 
> parameters:
>     any string
> 
> filter_with_parameters:
>     filter_id
>     filter_id=parameters
>     Such a filter has a unique number of inputs and outputs.
> 
> input_redirector
>     num<id
>         redirects input from id from the right into stream num
>     id>num
>         redirects input from id from the left into stream num
>     The specified stream num is inserted thus there is 1 input more
>     for each input_redirector. The renumbering though is delayed after
>     all input_redirectors are processed.
> 
> output_redirector
>     num>id
>         redirects output from stream num into id rightward
>     id<num
>         redirects output from stream num into id leftward
>     The specified stream num is removed thus there is 1 output less
>     for each output_redirector.The renumbering though is delayed after
>     all output_redirectors are processed.

I really like the redirector with num and id. But I don't see any point 
about the extra complexity of '>','<'. Is it only useful for 
user-defined graphs?

> redirection example:
> 1<T picInPic, split T<1, split 1>T, scale, T>1 picInPic
> or
> 1<T picInPic, split T<1, split , scale;nop , picInPic
> 
> 
> --> picInPic -> split--> split --> scale --> picInPic -->
>   >                  \        \            >
> /                     \        \_________/
> \_____________________/
> 
> 
> 
> 2<X 0<L Filter0 L<0 2>Y ; Y>0 2<L Filter1 L<2 X<0
> 
>    _________
>   /         \
>   \          |
>    >        /
> ---> Filter0 --->
>    >        \
>   /          |
>   \____  ___/
>        \/
>    ____/\___
>   /         \
>   \          |
>    >        /
> ---> Filter1 --->
>    >        \
>   /          |
>   \_________/
> 
> 
> atomic_filter:
>     filter_with_parameters
>     input_redirector atomic_filter
>     atomic_filter output_redirector
> 
> paralel_filter:
>     filter1;filter2;...;filterN
>     N paralelly placed filters. N can be 1 or larger.
>     The inputs to the paralel_filter will be routed in order begining with
>     the first input of the filter filter, second input of the first filter, ...
>     outputs will be routed similarly.
> 
> filter_chain:
>     paralel_filter1,paralel_filter2,...,paralel_filterN
>     A chain of N filters. N can be 1 or larger.
>     All outputs of each filter are routed in order to
>     the next (more to the right) filters inputs.
>     The inputs of the filter_chain are exactly the inputs of paralel_filter1
>     The outputs of the filter_chain are exactly the outputs of paralel_filterN
> 
> filter:
>     atomic_filter
>     (filter_chain)
> 
> 
> Permutation filter:
>     p=a1:a2:...:aN
>         N inputs, N outputs
>         output n is matched to input an
>         not all inputs have to be used
>     p=b::a1:a2:...:aN
>         b inputs, N outputs
>         output n is matched to input an
>         not all inputs have to be used
> 
> nop=N
>     N inputs, N outputs
>     inputs passed through unchanged
>     if =N is ommited N=1 is assumed

Just to make things clear, I'll describe more verbosely also the grammar 
I though initially, since I think people understood an expanded version 
of it (but now I'm not sure if it is not better expanded):

filter_id:
     any alphanumeric string identifying a filter

parameters:
     any string

filter_with_parameters:
     filter_id
     filter_id=parameters
     Such a filter has a unique number of inputs and outputs.

label_id:
     any alpha numeric string not beginning by "in" or by "out"

label:
     (label_id)
     input_label
     output_label

input_label:
     (in#)
     Where # can be any integer and (in) is an alias for (in1)

output_label:
     (out#)
     Where # can be any integer and (out) is an alias for (out1)

A filter graph should have elements (in1) (in2) ... and (out1) (out2) 
... in the exact number of inputs and outputs it has. Each one of those 
elements can appear just once.

A filter is preceded and followed by a list of labels. Each label should 
appear exactly once preceding a filter and once following one. The 
number of input labels (minus one if preceded by a comma) should be 
equal to the number of inputs of the filter (similar for outputs).

The filters in a chain can be linked by commas. A comma link _one_ 
output stream with _one_ input stream, making things like

(in) split, picInPic (out)

syntactically invalid.

A ';' just describes two different parts of a graph without doing any 
linking (and each part should have all its inputs and outputs already 
specified). The following

vflip ;

is not valid in any context, an output to vflip is missing. Only things like

...something here... vflip (T); (A)(B) picInPic ...something...

are valid.

My idea for user defined filters is to use the same syntax.

my_filter1 = '(in1)(in2) picInPic, (in3) picInPic(out1)'

or

my_filter2 = '(in1) split (tmpa), (tmpb) picInPic(out1); (tmpa) vflip 
(tmpb)'

and used like

movie=foo.avi (A);
movie=bar.avi (B);
movie=main.avi (C);
(A)(B)(C)  my_filter1 (out1);

Notice that there are no named inputs for the user-defined filter 
("in1", "in2" and "in3" are reserved keywords and "tmpa", "tmpb" are 
local labels). To avoid name clashing, I propose to simply call 
recursively the parser, so that the user-defined filter is parsed in 
another context (and then link the graph to the inputs and outputs it 
returns).

Extensions based on other grammars suggested (which I'm mostly favourable):

1- Letting the comma link more than one pad
Advantages: Simplify the syntax of some filters
Disadvantages: Add to syntax complexity

2- Let the ';' be interpreted like vrmsss's '*'
Advantages, Disadvantages: Same as (1)

3- Allow labels to optionally specify the source/destination stream number.
Advantages: Nice simplifications, like instead of

(in1) vflip (T1); (in2) (T1) picInPic (out)

that would be equivalent to
(in1) vflip (T1:0); (in2:0) (T1:1) picInPic (out)

could be written, much more simply
(in1) vflip, (in2:0) picInPic (out)

and in this case the comma will link to the next available stream.

Well, I hope I helped clarify things, not add to confusion...

-Vitor