[FFmpeg-devel] [PATCH] lavfi: add interleave filters

Stefano Sabatini stefasab at gmail.com
Sat Apr 20 13:56:02 CEST 2013


On date Saturday 2013-04-20 13:27:31 +0200, Clément Bœsch encoded:
> On Thu, Apr 18, 2013 at 11:00:51PM +0200, Stefano Sabatini wrote:
> [...]
> > From cc0ea64d70218a80b4ae29c98e2b86c988e7994b Mon Sep 17 00:00:00 2001
> > From: Stefano Sabatini <stefasab at gmail.com>
> > Date: Mon, 8 Apr 2013 15:16:06 +0200
> > Subject: [PATCH] lavfi: add interleave filters
> > 
> > TODO: bump minor, add changelog entry
> > ---
> >  doc/filters.texi           |   48 ++++++++
> >  libavfilter/Makefile       |    2 +
> >  libavfilter/allfilters.c   |    2 +
> >  libavfilter/f_interleave.c |  259 ++++++++++++++++++++++++++++++++++++++++++++
> >  4 files changed, 311 insertions(+)
> >  create mode 100644 libavfilter/f_interleave.c
> > 
> > diff --git a/doc/filters.texi b/doc/filters.texi
> > index 43d7368..b739fa0 100644
> > --- a/doc/filters.texi
> > +++ b/doc/filters.texi
> > @@ -7246,6 +7246,54 @@ setpts='(RTCTIME - RTCSTART) / (TB * 1000000)'
> >  @end example
> >  @end itemize
> >  
> > + at section ainterleave, interleave
> > +
> > +Temporally interleave frames from several inputs.
> > +
> > + at code{ainterleave} works with audio inputs, @code{interleave} with video.
> > +
> > +These filters read frames from several inputs and send the oldest
> > +queued frame to the output.
> > +
> > +Input streams must have a well defined, monotonically increasing frame
> > +timestamp value.
> > +
> > +In order to submit one frame to output, these filters need to enqueue
> > +at least one frame for each input, so they cannot work in case one
> > +input is not yet terminated and will not receive incoming frames.
> > +
> > +For example consider the case when one input is a @code{select} filter
> > +which always drop input frames. The @code{interleave} filter will keep
> > +reading from that input, but it will never be able to send new frames
> > +to output until the input will send an end-of-stream signal.
> > +
> > +Also, depending on inputs synchronization, the filters may drop frames
> > +in case one input receives more frames than the other ones, and the
> > +queue is already filled.
> > +
> > +These filters accept the following options:
> > +
> > + at table @option
> > + at item nb_inputs, n
> > +Set the number of different inputs, it is 2 by default.
> > + at end table
> > +
> > + at subsection Examples
> > +
> > + at itemize
> > + at item
> > +Interleave frames belonging to different streams using @command{ffmpeg}:
> > + at example
> > +ffmpeg -i bambi.avi -i pr0n.mkv -filter_complex "[0:v][1:v] interleave" out.avi
> > + at end example
> > +
> > + at item
> > +Add flickering blur effect:
> > + at example
> 
> > +select='gt(random(0), 0.2):branch=1 [tmp], boxblur=2:2, [tmp] interleave"
> 
> Trailing or missing initial double quote, and missing simple quote in
> select expression.
> 
> Also, the branch option was implemented differently so this example needs
> adjustments. I tried replacing branch=1 with n=2 but got a bunch of buffer
> queue overflows and no output.

Command fixed.

> 
> [...]
> > diff --git a/libavfilter/f_interleave.c b/libavfilter/f_interleave.c
> > new file mode 100644
> > index 0000000..c0af198
> > --- /dev/null
> > +++ b/libavfilter/f_interleave.c
> > @@ -0,0 +1,259 @@
> > +/*
> > + * Copyright (c) 2013 Stefano Sabatini
> > + *
> > + * This file is part of FFmpeg.
> > + *
> > + * FFmpeg is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU Lesser General Public
> > + * License as published by the Free Software Foundation; either
> > + * version 2.1 of the License, or (at your option) any later version.
> > + *
> > + * FFmpeg is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > + * Lesser General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU Lesser General Public
> > + * License along with FFmpeg; if not, write to the Free Software
> > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> > + */
> > +
> > +/**
> > + * @file
> > + * audio and video interleaver
> > + */
> > +
> > +#include "libavutil/avassert.h"
> > +#include "libavutil/avstring.h"
> > +#include "libavutil/opt.h"
> > +#include "avfilter.h"
> > +#include "bufferqueue.h"
> > +#include "formats.h"
> > +#include "internal.h"
> > +#include "audio.h"
> > +#include "video.h"
> > +
> > +typedef struct {
> > +    const AVClass *class;
> > +    int nb_inputs;
> > +    struct FFBufQueue *queues;
> > +} InterleaveContext;
> > +
> > +#define OFFSET(x) offsetof(InterleaveContext, x)
> > +
> > +#define DEFINE_OPTIONS(filt_name, filt_type)                                  \
> > +static const AVOption filt_name##_options[] = {                               \
> > +    { "nb_inputs", "set number of inputs", OFFSET(nb_inputs), AV_OPT_TYPE_INT, {.i64 = 2}, 1, INT_MAX, \
> > +      .flags = AV_OPT_FLAG_##filt_type##_PARAM|AV_OPT_FLAG_FILTERING_PARAM }, \
> > +    { "n",         "set number of inputs", OFFSET(nb_inputs), AV_OPT_TYPE_INT, {.i64 = 2}, 1, INT_MAX, \
> > +      .flags = AV_OPT_FLAG_##filt_type##_PARAM|AV_OPT_FLAG_FILTERING_PARAM }, \
> > +    { NULL },                                                                 \
> > +}
> > +
> 
> nit: I prefer how it's done in select (where the
> AV_OPT_FLAG_##filt_type##_PARAM|AV_OPT_FLAG_FILTERING_PARAM for each entry
> is avoided)

Changed.

> [...]
> > +static int config_output(AVFilterLink *outlink)
> > +{
> > +    AVFilterContext *ctx = outlink->src;
> > +    AVFilterLink *inlink0 = ctx->inputs[0];
> > +    int i;
> > +
> > +    if (outlink->type == AVMEDIA_TYPE_VIDEO) {
> > +        outlink->time_base           = AV_TIME_BASE_Q;
> > +        outlink->w                   = inlink0->w;
> > +        outlink->h                   = inlink0->h;
> > +        outlink->sample_aspect_ratio = inlink0->sample_aspect_ratio;
> > +        outlink->format              = inlink0->format;
> > +        outlink->frame_rate = (AVRational) {1, 0};
> > +        for (i = 1; i < ctx->nb_inputs; i++) {
> > +            AVFilterLink *inlink = ctx->inputs[i];
> > +
> > +            if (outlink->w                       != inlink->w                       ||
> > +                outlink->h                       != inlink->h                       ||
> > +                outlink->sample_aspect_ratio.num != inlink->sample_aspect_ratio.num ||
> > +                outlink->sample_aspect_ratio.den != inlink->sample_aspect_ratio.den) {
> > +                av_log(ctx, AV_LOG_ERROR, "Parameters for input link %s "
> > +                       "(size %dx%d, SAR %d:%d) do not match the corresponding "
> > +                       "output link parameters (%dx%d, SAR %d:%d)\n",
> > +                       ctx->input_pads[i].name, inlink->w, inlink->h,
> > +                       inlink->sample_aspect_ratio.num,
> > +                       inlink->sample_aspect_ratio.den,
> > +                       outlink->w, outlink->h,
> > +                       outlink->sample_aspect_ratio.num,
> > +                       outlink->sample_aspect_ratio.den);
> > +                return AVERROR(EINVAL);
> > +            }
> > +        }
> > +    }
> 
> No consistency checks for audio required?

I based this code over concat, and no it seems there is no need for
audio checks, since we should be already sure that samplerate, channel
count and channel layout and sample formats is the same for all the
inputs.

> 
> > +
> > +    outlink->flags |= FF_LINK_FLAG_REQUEST_LOOP;
> > +    return 0;
> > +}
> > +
> 
> Rest of the code looks OK.

Will push in a day or so if I read no more comments, thank you.
-- 
FFmpeg = Furious & Furious Merciless Peaceless Enhanced Guru
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0003-lavfi-add-interleave-filters.patch
Type: text/x-diff
Size: 13333 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20130420/c4563200/attachment.bin>


More information about the ffmpeg-devel mailing list