[FFmpeg-devel] [PATCH 2/4] pan: add channel mapping capability.
Nicolas George
nicolas.george at normalesup.org
Mon Jan 23 15:05:53 CET 2012
Le quartidi 4 pluviôse, an CCXX, Clément Bœsch a écrit :
> I meant "remap == gains are pure"; remap being opposed to panning by its
> purity.
But gains are pure... what? Pure water? Pure integers?
> This debate is slightly moving to a philosophical bikeshed
Of course.
> libswresample just keeps a pointer to the channel mapping array internally, so
> at the moment it is required. This could be changed eventually.
I stand corrected.
> Mmh, I can't check it here, so I added a check in config_props() callback. But
> I'm not sure the checks are really necessary actually, since AFAIK no existing
> channel layouts have more channels than SWR_CH_MAX.
Well, I guess we could state that SWR_CH_MAX is an absolute maximum for the
number of channels lavfi can accept, and raise it if it becomes necessary.
> Well, it looks like a standard function call to me:
Exactly: with a branch instead of a function pointer, it could be a simple
jump, saving all the registers saving/restoring and arguments passing.
> - bumped lavfi micro
Beware, git rebase will silently assume the hunk has been cherry-picked
instead of showing a conflict.
> From 95f57535fc5f418173502a52bdbdeb0def4fbc20 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Cl=C3=A9ment=20B=C5=93sch?= <clement.boesch at smartjog.com>
> Date: Wed, 18 Jan 2012 12:00:16 +0100
> Subject: [PATCH] pan: add channel mapping capability.
>
> ---
> configure | 1 +
> doc/filters.texi | 45 +++++++++++++++++
> libavfilter/af_pan.c | 133 ++++++++++++++++++++++++++++++++++++++++++++++--
> libavfilter/version.h | 2 +-
> 4 files changed, 174 insertions(+), 7 deletions(-)
>
> diff --git a/configure b/configure
> index 2a3eaa1..a06e63c 100755
> --- a/configure
> +++ b/configure
> @@ -1659,6 +1659,7 @@ mp_filter_deps="gpl avcodec"
> mptestsrc_filter_deps="gpl"
> negate_filter_deps="lut_filter"
> ocv_filter_deps="libopencv"
> +pan_filter_deps="swresample"
> scale_filter_deps="swscale"
> tinterlace_filter_deps="gpl"
> yadif_filter_deps="gpl"
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 3c9f554..7d008bc 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -315,6 +315,9 @@ Ported from SoX.
> Mix channels with specific gain levels. The filter accepts the output
> channel layout followed by a set of channels definitions.
>
> +This filter is also designed to remap efficiently the channels of an audio
> +stream.
> +
> The filter accepts parameters of the form:
> "@var{l}:@var{outdef}:@var{outdef}:..."
>
> @@ -342,6 +345,8 @@ If the `=' in a channel specification is replaced by `<', then the gains for
> that specification will be renormalized so that the total is 1, thus
> avoiding clipping noise.
>
> + at subsection Mixing examples
> +
> For example, if you want to down-mix from stereo to mono, but with a bigger
> factor for the left channel:
> @example
> @@ -358,6 +363,46 @@ Note that @command{ffmpeg} integrates a default down-mix (and up-mix) system
> that should be preferred (see "-ac" option) unless you have very specific
> needs.
>
> + at subsection Remapping examples
> +
> +The channel remapping will be effective if, and only if:
> +
> + at itemize
> + at item gain coefficients are zeroes or ones,
> + at item only one input per channel output,
> + at item the number of output channels is supported by libswresample (16 at the
> + moment)
> + at c if SWR_CH_MAX changes, fix the line above.
> + at end itemize
> +
> +If all these conditions are satisfied, the filter will notify the user ("Pure
> +channel mapping detected"), and use an optimized and lossless method to do the
> +remapping.
> +
> +For example, if you have a 5.1 source and want a stereo audio stream by
> +dropping the extra channels:
> + at example
> +pan="stereo: c0=FL : c1=FR"
> + at end example
> +
> +Given the same source, you can also switch front left and front right channels
> +and keep the input channel layout:
> + at example
> +pan="5.1: c0=c1 : c1=c0 : c2=c2 : c3=c3 : c4=c4 : c5=c5"
> + at end example
> +
> +If the input is a stereo audio stream, you can mute the front left channel (and
> +still keep the stereo channel layout) with:
> + at example
> +pan="stereo:c1=c1"
> + at end example
> +
> +Still with a stereo audio stream input, you can copy the right channel in both
> +front left and right:
> + at example
> +pan="stereo: c0=FR : c1=FR"
> + at end example
> +
> @section silencedetect
>
> Detect silence in an audio stream.
> diff --git a/libavfilter/af_pan.c b/libavfilter/af_pan.c
> index a62dea1..d6bb0dc 100644
> --- a/libavfilter/af_pan.c
> +++ b/libavfilter/af_pan.c
> @@ -30,12 +30,14 @@
> #include <stdio.h>
> #include "libavutil/audioconvert.h"
> #include "libavutil/avstring.h"
> +#include "libavutil/opt.h"
> +#include "libswresample/swresample.h"
> #include "avfilter.h"
> #include "internal.h"
>
> #define MAX_CHANNELS 63
>
> -typedef struct {
> +typedef struct PanContext {
> int64_t out_channel_layout;
> union {
> double d[MAX_CHANNELS][MAX_CHANNELS];
> @@ -46,6 +48,16 @@ typedef struct {
> int need_renumber;
> int nb_input_channels;
> int nb_output_channels;
> +
> + int pure_gains;
> + void (*filter_samples)(struct PanContext*,
> + AVFilterBufferRef*,
> + AVFilterBufferRef*,
> + int);
> +
> + /* channel mapping specific */
> + int channel_map[SWR_CH_MAX];
> + struct SwrContext *swr;
> } PanContext;
>
> static int parse_channel_name(char **arg, int *rchannel, int *rnamed)
> @@ -179,6 +191,31 @@ static av_cold int init(AVFilterContext *ctx, const char *args0, void *opaque)
> return 0;
> }
>
> +static void filter_samples_channel_mapping(PanContext *pan, AVFilterBufferRef *outsamples, AVFilterBufferRef *insamples, int n);
> +static void filter_samples_panning (PanContext *pan, AVFilterBufferRef *outsamples, AVFilterBufferRef *insamples, int n);
> +
> +static int are_gains_pure(const PanContext *pan)
> +{
> + int i, j;
> +
> + for (i = 0; i < MAX_CHANNELS; i++) {
> + int nb_gain = 0;
> +
> + for (j = 0; j < MAX_CHANNELS; j++) {
> + double gain = pan->gain.d[i][j];
> +
> + /* channel mapping is effective only if 0% or 100% of a channel is
> + * selected... */
> + if (gain != 0. && gain != 1.)
> + return 0;
> + /* ...and if the output channel is only composed of one input */
> + if (gain && nb_gain++)
> + return 0;
> + }
> + }
> + return 1;
> +}
> +
> static int query_formats(AVFilterContext *ctx)
> {
> PanContext *pan = ctx->priv;
> @@ -186,11 +223,21 @@ static int query_formats(AVFilterContext *ctx)
> AVFilterLink *outlink = ctx->outputs[0];
> AVFilterFormats *formats;
>
> + if (pan->nb_output_channels <= SWR_CH_MAX)
> + pan->pure_gains = are_gains_pure(pan);
> + if (pan->pure_gains) {
> + /* libswr supports any sample and packing formats */
> + avfilter_set_common_sample_formats(ctx, avfilter_make_all_formats(AVMEDIA_TYPE_AUDIO));
> + avfilter_set_common_packing_formats(ctx, avfilter_make_all_packing_formats());
> + pan->filter_samples = filter_samples_channel_mapping;
> + } else {
> const enum AVSampleFormat sample_fmts[] = {AV_SAMPLE_FMT_S16, -1};
> const int packing_fmts[] = {AVFILTER_PACKED, -1};
>
> avfilter_set_common_sample_formats (ctx, avfilter_make_format_list(sample_fmts));
> avfilter_set_common_packing_formats(ctx, avfilter_make_format_list(packing_fmts));
> + pan->filter_samples = filter_samples_panning;
> + }
>
> // inlink supports any channel layout
> formats = avfilter_make_all_channel_layouts();
> @@ -222,6 +269,44 @@ static int config_props(AVFilterLink *link)
> }
> }
> }
> + // gains are pure, init the channel mapping
> + if (pan->pure_gains) {
> +
> + // sanity check; can't be done in query_formats since the inlink
> + // channel layout is unknown at that time
> + if (pan->nb_input_channels >= SWR_CH_MAX) {
I believe nb_input_channels == SWR_CH_MAX should be accepted.
> + av_log(ctx, AV_LOG_ERROR,
> + "libswresample support a maximum of %d channels. "
> + "Feel free to ask for a higher limit.\n", SWR_CH_MAX);
> + return AVERROR_PATCHWELCOME;
> + }
> +
> + // get channel map from the pure gains
> + for (i = 0; i < pan->nb_output_channels; i++) {
> + int ch_id = -1;
> + for (j = 0; j < pan->nb_input_channels; j++) {
> + if (pan->gain.d[i][j]) {
> + ch_id = j;
> + break;
> + }
> + }
> + pan->channel_map[i] = ch_id;
> + }
> +
> + // init libswresample context
> + pan->swr = swr_alloc_set_opts(pan->swr,
> + pan->out_channel_layout, link->format, link->sample_rate,
> + link->channel_layout, link->format, link->sample_rate,
> + 0, ctx);
> + if (!pan->swr)
> + return AVERROR(ENOMEM);
> + av_opt_set_int(pan->swr, "icl", pan->out_channel_layout, 0);
> + av_opt_set_int(pan->swr, "uch", pan->nb_output_channels, 0);
> + swr_set_channel_mapping(pan->swr, pan->channel_map);
> + r = swr_init(pan->swr);
> + if (r < 0)
> + return r;
> + } else {
> // renormalize
> for (i = 0; i < pan->nb_output_channels; i++) {
> if (!((pan->need_renorm >> i) & 1))
> @@ -239,6 +324,7 @@ static int config_props(AVFilterLink *link)
> for (j = 0; j < pan->nb_input_channels; j++)
> pan->gain.d[i][j] /= t;
> }
> + }
> // summary
> for (i = 0; i < pan->nb_output_channels; i++) {
> cur = buf;
> @@ -249,6 +335,17 @@ static int config_props(AVFilterLink *link)
> }
> av_log(ctx, AV_LOG_INFO, "o%d = %s\n", i, buf);
> }
> + // add channel mapping summary if possible
> + if (pan->pure_gains) {
> + av_log(ctx, AV_LOG_INFO, "Pure channel mapping detected:");
> + for (i = 0; i < pan->nb_output_channels; i++)
> + if (pan->channel_map[i] < 0)
> + av_log(ctx, AV_LOG_INFO, " M");
> + else
> + av_log(ctx, AV_LOG_INFO, " %d", pan->channel_map[i]);
> + av_log(ctx, AV_LOG_INFO, "\n");
> + return 0;
> + }
> // convert to integer
> for (i = 0; i < pan->nb_output_channels; i++) {
> for (j = 0; j < pan->nb_input_channels; j++) {
> @@ -261,19 +358,26 @@ static int config_props(AVFilterLink *link)
> return 0;
> }
>
> +static void filter_samples_channel_mapping(struct PanContext *pan,
struct is not necessary (but not evil either).
> + AVFilterBufferRef *outsamples,
> + AVFilterBufferRef *insamples,
> + int n)
> +{
> + swr_convert(pan->swr, outsamples->data, n, (void *)insamples->data, n);
> +}
>
> -static void filter_samples(AVFilterLink *inlink, AVFilterBufferRef *insamples)
> +static void filter_samples_panning(struct PanContext *pan,
Idem.
> + AVFilterBufferRef *outsamples,
> + AVFilterBufferRef *insamples,
> + int n)
> {
> - PanContext *const pan = inlink->dst->priv;
> - int i, o, n = insamples->audio->nb_samples;
> + int i, o;
>
> /* input */
> const int16_t *in = (int16_t *)insamples->data[0];
> const int16_t *in_end = in + n * pan->nb_input_channels;
>
> /* output */
> - AVFilterLink *const outlink = inlink->dst->outputs[0];
> - AVFilterBufferRef *outsamples = avfilter_get_audio_buffer(outlink, AV_PERM_WRITE, n);
> int16_t *out = (int16_t *)outsamples->data[0];
>
> for (; in < in_end; in += pan->nb_input_channels) {
> @@ -284,16 +388,33 @@ static void filter_samples(AVFilterLink *inlink, AVFilterBufferRef *insamples)
> *(out++) = v >> 8;
> }
> }
> +}
> +
> +static void filter_samples(AVFilterLink *inlink, AVFilterBufferRef *insamples)
> +{
> + int n = insamples->audio->nb_samples;
> + AVFilterLink *const outlink = inlink->dst->outputs[0];
> + AVFilterBufferRef *outsamples = avfilter_get_audio_buffer(outlink, AV_PERM_WRITE, n);
> + PanContext *pan = inlink->dst->priv;
> +
> + pan->filter_samples(pan, outsamples, insamples, n);
>
> avfilter_filter_samples(outlink, outsamples);
> avfilter_unref_buffer(insamples);
> }
>
> +static av_cold void uninit(AVFilterContext *ctx)
> +{
> + PanContext *pan = ctx->priv;
> + swr_free(&pan->swr);
> +}
> +
> AVFilter avfilter_af_pan = {
> .name = "pan",
> .description = NULL_IF_CONFIG_SMALL("Remix channels with coefficients (panning)."),
> .priv_size = sizeof(PanContext),
> .init = init,
> + .uninit = uninit,
> .query_formats = query_formats,
>
> .inputs = (const AVFilterPad[]) {
> diff --git a/libavfilter/version.h b/libavfilter/version.h
> index d2af525..6e19dd7 100644
> --- a/libavfilter/version.h
> +++ b/libavfilter/version.h
> @@ -30,7 +30,7 @@
>
> #define LIBAVFILTER_VERSION_MAJOR 2
> #define LIBAVFILTER_VERSION_MINOR 59
> -#define LIBAVFILTER_VERSION_MICRO 101
> +#define LIBAVFILTER_VERSION_MICRO 102
>
> #define LIBAVFILTER_VERSION_INT AV_VERSION_INT(LIBAVFILTER_VERSION_MAJOR, \
> LIBAVFILTER_VERSION_MINOR, \
> --
> 1.7.8.3
>
It looks good. Unless someone else has remarks, I believe it can be applied.
And no need to send another patch just to replace ">=" by ">" or remove
struct.
Thanks for your work.
Regards,
--
Nicolas George
More information about the ffmpeg-devel
mailing list