[FFmpeg-devel] [RFC] AVFilter Parser

[vmrsss]

Hi everybody,

	Vitor, there are a few things I don't understand in your proposal  
below. A few questions and some comments below:

On 22 Mar 2008, at 00:22, Vitor Sessak wrote:
> Just to make things clear, I'll describe more verbosely also the  
> grammar
> I though initially, since I think people understood an expanded  
> version
> of it (but now I'm not sure if it is not better expanded):
>
> filter_id:
>     any alphanumeric string identifying a filter
>
> parameters:
>     any string
>
> filter_with_parameters:
>     filter_id
>     filter_id=parameters
>     Such a filter has a unique number of inputs and outputs.
>
> label_id:
>     any alpha numeric string not beginning by "in" or by "out"
>
> label:
>     (label_id)
>     input_label
>     output_label

> input_label:
>     (in#)
>     Where # can be any integer and (in) is an alias for (in1)

> output_label:
>     (out#)
>     Where # can be any integer and (out) is an alias for (out1)

Probably counting should start at zero for consistency with the way  
ffmpeg numbers streams (and input files). Why is it important to have  
the prefixes "in" and "out"? Could it not be any string? What is the  
third kind of label (not input nor output) for? Is it to name  
intermediate streams?

> A filter graph should have elements (in1) (in2) ... and (out1) (out2)
> ... in the exact number of inputs and outputs it has. Each one of  
> those
> elements can appear just once.

Is it still your intention to use labels to describe feedback? How do  
you express that out_k is to be fed back to in_h? Or can you only do  
feedback on intermediate labels?

> A filter is preceded and followed by a list of labels. Each label  
> should
> appear exactly once preceding a filter and once following one.

do you mean each input label should appear exactly once preceding ...,  
and each ouput label should appear exactly one following ...?

> The
> number of input labels (minus one if preceded by a comma) should be
> equal to the number of inputs of the filter (similar for outputs).

> The filters in a chain can be linked by commas. A comma link _one_
> output stream with _one_ input stream,

Which one? the first? or the last? the one missing from the list?  
Perhaps I should know the answer, but I am not clear about your  
intended use of labels. Let me take an example you use below:

	 '(in1)(in2) picInPic, (in3) picInPic(out1)'

Is "(in1)(in2)" the same as "(in2)(in1)" ? I expect the answer is no,  
the second form would swap the two input stream. And (in3) refers to a  
third input from the entire chain. Is that fed to the second input of  
picInPic? In order to feed it to the first input, would I have to  
write the filter this way:

	 '(in1)(in2) picInPic(tmp), (in3)(tmp) picInPic(out1)'  ?

> making things like
>
> (in) split, picInPic (out)
>
> syntactically invalid.
>
> A ';' just describes two different parts of a graph without doing any
> linking (and each part should have all its inputs and outputs already
> specified). The following
>
> vflip ;
>
> is not valid in any context, an output to vflip is missing. Only  
> things like
>
> ...something here... vflip (T); (A)(B) picInPic ...something...
>
> are valid.
>
> My idea for user defined filters is to use the same syntax.
>
> my_filter1 = '(in1)(in2) picInPic, (in3) picInPic(out1)'

> or
>
> my_filter2 = '(in1) split (tmpa), (tmpb) picInPic(out1); (tmpa)  
> vflip (tmpb)'

> and used like
>
> movie=foo.avi (A);
> movie=bar.avi (B);
> movie=main.avi (C);
> (A)(B)(C)  my_filter1 (out1);
>
> Notice that there are no named inputs for the user-defined filter
> ("in1", "in2" and "in3" are reserved keywords and "tmpa", "tmpb" are
> local labels).

Are you saying that user-defined filters must use the in_k form, or  
that they must not?

with respect to the form above (A)(B)(C) my_filter1 (out1), if I make  
a textual substitution I find myself with

	(A)(B)(C) (in1)(in2) picInPic, (in3) picInPic(out1) (out1)

how do we explain what happens here? Is this supposed to be the same  
as writing

	(A)(B) picInPic, (C) picInPic(out1) ?

Because of course, you'll have situations like this:

	my_f1 = '(in1)something(out1)'
	my_f2 = '(in2)otherthing(out1)'

	movie=f1.avi(A);
	movie=f2.avi(B);
	(A)(B) my_f1 ; my_f2 (out1)(out2)

which I expect it is the same as : (in1)something(out1) ;  
(in2)otherthing(out2) .
Right?

> To avoid name clashing, I propose to simply call
> recursively the parser, so that the user-defined filter is parsed in
> another context (and then link the graph to the inputs and outputs it
> returns).

>
> Extensions based on other grammars suggested (which I'm mostly  
> favourable):
>
> 1- Letting the comma link more than one pad
> Advantages: Simplify the syntax of some filters
> Disadvantages: Add to syntax complexity

I think it is important that whatever the syntax is, users are allowed  
to omit elements so as to write simple filters as simply as always, eg:

	... remove_logo,deinterlace,crop,scale,pad...

In this sense, it's important to be able to leave streams implicit if  
possible.

> 2- Let the ';' be interpreted like vrmsss's '*'
> Advantages, Disadvantages: Same as (1)

I don't think the choice of a symbol is important, it is the syntactic  
mechanisms to combine streams which are important. I suppose we can  
take everybody agrees on the key elements for that (functional or  
sequential composition of filters --meaning outputs of the preceding  
filter feed into inputs of the following one-- parallel or non- 
sequential composition, rearrangement of streams via appropriate  
namings, and feedback or looping). I don't think we are yet reached  
the perfect way to express these, but we're close.

> 3- Allow labels to optionally specify the source/destination stream  
> number.
> Advantages: Nice simplifications, like instead of
>
> (in1) vflip (T1); (in2) (T1) picInPic (out)
>
> that would be equivalent to
> (in1) vflip (T1:0); (in2:0) (T1:1) picInPic (out)
>
> could be written, much more simply
> (in1) vflip, (in2:0) picInPic (out)

yes, I think this is appealing.

> and in this case the comma will link to the next available stream.

it would be the *only* one available, which is a good definition; but  
apart from the convention you use for implicit linking, don't forget  
that you have to provide the user with a way to override it.


> Well, I hope I helped clarify things, not add to confusion...
>
> -Vitor
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at mplayerhq.hu
> https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-devel