[FFmpeg-devel] Plans for libavfilter

[Paul B Mahol]

On Thu, Sep 16, 2021 at 8:47 PM Michael Niedermayer <michael at niedermayer.cc>
wrote:

> On Thu, Sep 16, 2021 at 01:55:17PM +0200, Nicolas George wrote:
> > Since it is not healthy to keep everything for myself, here is a summary
> > of the projects I have in mind for the enhancement of libavfilter.
> >
> > The thing is, there are a lot of dependencies between these projects.
> > Working on them in a proper order would make achieving the goals easier.
> > On the other hand, if task B depends on task A, then working B without
> > addressing A involves a little more work for B, but also a lot more work
> > for A, and a lot more work for everything else that depends on A.
> >
> > This is one reason I am writing this mail: to make clear what needs to
> > be done, and how tasks depend on each-other and fit together.
> >
> > The earliest tasks are not very sexy. They are about making the code
> > clearer or more robust and generic rather than adding features. It is
> > always boring. But they are necessary, because parts of the process is
> > quite shaky, and we cannot afford to build on shaky code.
> >
> > If you want to help, please consider this and you will be welcome.
> >
> > Now that is established the specifics:
> >
> > - Multi-stage negotiation.
> >
> >   That means negotiating properties that depend on other properties. I
> >   think the colorspace stuff is in that case, it cannot be negotiated
> >   until the pixel format is known.
> >
>
> > - Media type negotiation.
> >
> >   That means that filters that do not care about the contents of what
> >   they are filtering, filters that only act on the frame information
> >   fields, can also not care if they are filtering audio or video or
> >   other. There are a lot of these filters: pts manipulation, concat,
> >   split, selection, etc.
>
> +1
> the current need to duplicate filters per media type is ugly.
> Also "negotiation" is a strong term for this. Filters in general
> have either one media type on their pads or pass the type from
> input to output and support any. In that sense this is not hard
> to solve, it should not be dependant on anything else
>

Nope, it is non functional. You want to add yet another layer of complexity
for 0 gain.

>
>
> [...]
>
> >
> > - Partial graph configuration.
> >
> >   Right now, the negotiation happens in steps, first query_formats then
> >   merge, then pick. But pick can be done on some links as soon as the
> >   formats are merged to a singleton while other filters are still
> >   delaying their query_formats, and once pick is done, the graph can
> >   start running. That would allow filters with constraints more complex
> >   than what AVFilterFormats can express. (We have a mechanism for that,
> >   used by amerge and one or two other filters; I introduced it early and
> >   it was a mistake, it is much too fragile and only converges in the
> >   simplest cases.)
> >
> > - Partial graph re-configuration.
> >
> >   If we can run a graph when not all filters are configured, then we can
> >   de-configure and re-configure part of a graph. That would bring us the
> >   support for changes in pixel format.
>
> Maybe some orthogonal direction could be considered too
> completely seperating graph configuration and using some more generic
> algorithm to solve it.
> What we have basically are a bunch of constraints and then we need to
> color the links between vertexes so as to minimize a weighted average
> of computational complexity and quality loss. And each link can have
> also different colors on both ends which would correspond to an inserted
> convert filter. The "color" here would be a vector of all elements that
> need configuration.
>
> This view if it works would also allow very easy changes to the things
> needing configuration. For example in this framework one could replace
> a colorspace enum by a AxB element matrix and nothing would change on
> the solver it would just need a different quality estimation function.
>
> That means all the heuristics would be isolated in the function which
> is to be minimized. While the solver would just deal with generic
> values of some kind like ints
>
>
> [...]
>
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> The smallest minority on earth is the individual. Those who deny
> individual rights cannot claim to be defenders of minorities. - Ayn Rand
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>