[FFmpeg-devel] [RFC] Dynamic filtergraph reconfiguration

[Nicolas George]

Le quartidi 4 brumaire, an CCXXI, Stefano Sabatini a écrit :
> maybe it's time to finally tackle this, since it seems to be a
> recurring issue.

I agree.

> Sometimes the input of the filterchain changes configuration
> parameters. For audio it means it could change channel layout/sample
> rate/sample format/timebase, for video it may change frame rate, size,
> pixel format, sample aspect ratio and timebase.
> 
> Basically most filters assume these values to be invariant during the
> whole life of the filtergraph, others (e.g. the scale filter) monitors
> *some* of the input values and reconfigure themselves accordingly,
> sometimes calling configure on the following filters (which may or may
> not be correct). We need to make reconfiguration consistently applied
> at the framework level, and properly design it, in order to avoid
> brittle code and glitches.

Good summary.

> Problems to tackle are:
> - not all filters may support reconfigurations, for example some
>   filters rely on some specific values defined during configuration,
>   so in general we need a mechanism which allows to reconfigure only
>   the filters which can do that
> 
> - some filters may support only *one* parameter change (e.g. timebase
>   may easily change).

A bitmask on each input or input pad seems the obvious solution to both
these problems.

> - since we don't mark codecs/containers which support variable
>   configuration parameters (do we?), we may need to "normalize" the
>   output before the terminal sink (depending on the application).

The buffer sink can have an option to set the bitmask for supported
configuration change: if an application needs constant output, it leaves it
to its default 0.

> Related questions: where should we check the changes (possibly in the
> framework code)?

IMHO, calling config_props when the configuration changes is a good. That
means: for buffersrc, either detect or have an API to notify of a change,
for other sources, on a case by case basis, and for any non-source filter,
assume input is constant unless config_props is called on it, and call
config_props before changing the output.

We can provide an helper API:

unsigned avfilter_detect_format_change(AVFilterLink *link,
                                       AVFilterBufferRef *ref);

that returns the bitmask for the difference between link and ref.

>		   Where should we add the "buffer" filters which
> normalize input towards a "non-reconfigurable" filter (possibly we
> should auto-insert it only when required).

Inserting only when necessary looks like the obvious solution. I suggest
something like that:

int avfilter_link_reconfigure(AVFilterLink *outlink, unsigned changed)
{
    unsigned reconf    = changed &  outlink->dst->reconfigure_flags;
    unsigned normalize = changed & ~outlink->dst->reconfigure_flags;

    if (normalize) {
        ff_insert_normalization_filter(outlink, normalize);
    }
    if (reconf) {
        if (link->dstpad->config_props)
            link->dstpad->config_props(link);
    }
}

(with some more error checks, of course)

> Filter supporting reconfiguration should be marked as such (or
> alternatively we may mark filters *not* supporting it). Also I wonder
> if we should also support filters where the *number* of inputs or
> outputs change dinamically (e.g. a movie source which adds a new
> stream).

I do not see how we could handle it without some kind of high-level language
to describe the graph. I believe we can leave that out for now.

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20121025/2cdaaf3c/attachment.asc>