[FFmpeg-devel] [RFC] Dynamic filtergraph reconfiguration

[Michael Bradshaw]

Hi,

On Wed, Oct 24, 2012 at 5:21 PM, Stefano Sabatini <stefasab at gmail.com> wrote:
> Hi,
>
> maybe it's time to finally tackle this, since it seems to be a
> recurring issue.
>
> Sometimes the input of the filterchain changes configuration
> parameters. For audio it means it could change channel layout/sample
> rate/sample format/timebase, for video it may change frame rate, size,
> pixel format, sample aspect ratio and timebase.
>
> Basically most filters assume these values to be invariant during the
> whole life of the filtergraph, others (e.g. the scale filter) monitors
> *some* of the input values and reconfigure themselves accordingly,
> sometimes calling configure on the following filters (which may or may
> not be correct). We need to make reconfiguration consistently applied
> at the framework level, and properly design it, in order to avoid
> brittle code and glitches.
>
> Problems to tackle are:
> - not all filters may support reconfigurations, for example some
>   filters rely on some specific values defined during configuration,
>   so in general we need a mechanism which allows to reconfigure only
>   the filters which can do that
>
> - some filters may support only *one* parameter change (e.g. timebase
>   may easily change).
>
> - since we don't mark codecs/containers which support variable
>   configuration parameters (do we?), we may need to "normalize" the
>   output before the terminal sink (depending on the application).
>
> Related questions: where should we check the changes (possibly in the
> framework code)?

I think that would be nice (because then I wouldn't have to do it
myself every time I use the C API)

> Where should we add the "buffer" filters which
> normalize input towards a "non-reconfigurable" filter (possibly we
> should auto-insert it only when required).
>
> Filter supporting reconfiguration should be marked as such (or
> alternatively we may mark filters *not* supporting it).

How detailed would this marking be? i.e. do you say "this filter
supports total reconfiguration" and "this filter supports
reconfiguring these 3 specific parameters" or do you say "this filter
supports at least some reconfiguration" and just fail if an
unsupported reconfiguration is attempted (and then auto-insert a
"normalization" filter).

> Also I wonder
> if we should also support filters where the *number* of inputs or
> outputs change dinamically (e.g. a movie source which adds a new
> stream).

That would be nice.

> I'll try to come with a more detailed design and proof of concept once
> we'll clear some of the design issues.

Thanks for bringing this up. I'm really looking forward to this
(however it's implemented). Are you also considering handling if one
filter dynamically changes its output configuration parameters, even
if it's in the middle of the filter graph? For example, an auto-crop
filter that crops letterboxing on frames (and maybe there's a
commercial segment with no letterboxing).

I wonder how this would be affected by subtitle filters (when we get some)...