[FFmpeg-devel] [PATCH] Add af_resample - sample fmt and channel layout conversion filter
Michael Niedermayer
michaelni
Thu Jan 13 03:27:40 CET 2011
On Wed, Jan 12, 2011 at 10:25:26PM +0100, Stefano Sabatini wrote:
> On date Wednesday 2010-10-06 21:23:12 +0200, Michael Niedermayer encoded:
> > On Fri, Oct 01, 2010 at 06:40:24PM +0200, Stefano Sabatini wrote:
> > > On date Monday 2010-09-27 02:50:41 +0200, Michael Niedermayer encoded:
> > > > On Mon, Sep 27, 2010 at 02:41:16AM +0200, Stefano Sabatini wrote:
> > > > > On date Monday 2010-09-27 02:21:49 +0200, Michael Niedermayer encoded:
> > > > > > On Mon, Sep 27, 2010 at 12:40:31AM +0200, Stefano Sabatini wrote:
> > > > > [...]
> > > > > > > > the functions belong to libavfilter
> > > > > > > > what is the plan to move them there?
> > > > > > >
> > > > > > > Impossible as lavc uses that functions internally. The plan was to
> > > > > >
> > > > > > resample uses them, resample doesnt belong to lavc, not more than
> > > > > > sws would belong there
> > > > > >
> > > > > >
> > > > > > > directly use them, and move to something better when we'll have a
> > > > > > > revisited resampling/conversion API, a libavresample was mentioned.
> > > > > > > Since this is not going to take small time, the idea was to use the
> > > > > > > quickest solution and not stall audio lavfi development.
> > > > > >
> > > > > > the quickest is to copy the 2 pages of poor code not to export it as
> > > > > > public ABI/API from the wrong lib
> > > > > > that API/ABI requires maintaince and is not a small burden when things
> > > > > > have to be moved to the correct place or when API/ABI changes which with
> > > > > > such trashy code will happen unless development stops
> > > > >
> > > > > OK so the quickest path looks like to duplicate the code as it was
> > > > > done in the original version from Hemanth...
> > > >
> > > > yes, i dont like it but i think this is a rare case where this will cause
> > > > fewer problems
> > > >
> > > > also the code in libavcodec should be marked as deprecated as soon as possible
> > >
> > > Updated patch based on an old patch from Hemanth (Hemanth feel free to
> > > continue to discuss starting from this patch).
> > >
> > > My plan is to get -af implemented in ffplay, this requires these
> > > steps:
> > > * resample filter
> > > * resample filter auto-insertion provided by the lavfi framework
> > > * avfilter_get_audio_buffer_ref() or the equivalent, required for
> > > eliminating an unnecessary memcpy
> > > * ffplay -af
> > >
> > > Once it is possible to test somehow libavfilter with audio, then we
> > > can move to ffmpeg:
> > > * asrc_buffer
> > > * ffmpeg -af
> > > * audio lavfi tests
> > >
> > > Regards.
> > > --
> > > FFmpeg = Frenzy Forgiving Minimalistic Pitiless Epic Governor
> >
> > > Makefile | 1
> > > af_resample.c | 467 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > > allfilters.c | 1
> > > 3 files changed, 469 insertions(+)
> > > 60997f9e273101bb923c1c14bb145da860923d92 0001-Add-af_resample-sample-fmt-and-channel-layout-conver.patch
> > > From 2b9cf4fc8fa2d55aa18e6e53c9cf8ca600579133 Mon Sep 17 00:00:00 2001
> > > From: Stefano Sabatini <stefano.sabatini-lala at poste.it>
> > > Date: Fri, 1 Oct 2010 14:58:22 +0200
> > > Subject: [PATCH 1/3] Add af_resample - sample fmt and channel layout conversion filter.
> > >
> > > Patch by "S.N. Hemanth Meenakshisundaram" 5m33nak5 at uc5d.3du.
> > > ---
> > > libavfilter/Makefile | 1 +
> > > libavfilter/af_resample.c | 467 +++++++++++++++++++++++++++++++++++++++++++++
> > > libavfilter/allfilters.c | 1 +
> > > 3 files changed, 469 insertions(+), 0 deletions(-)
> > > create mode 100644 libavfilter/af_resample.c
> > >
> > > diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> > > index 74e55bb..e65995c 100644
> > > --- a/libavfilter/Makefile
> > > +++ b/libavfilter/Makefile
> > > @@ -15,6 +15,7 @@ OBJS = allfilters.o \
> > > parseutils.o \
> > >
> > > OBJS-$(CONFIG_ANULL_FILTER) += af_anull.o
> > > +OBJS-$(CONFIG_RESAMPLE_FILTER) += af_resample.o
> > >
> > > OBJS-$(CONFIG_ANULLSRC_FILTER) += asrc_anullsrc.o
> > >
> > > diff --git a/libavfilter/af_resample.c b/libavfilter/af_resample.c
> > > new file mode 100644
> > > index 0000000..2e7898e
> > > --- /dev/null
> > > +++ b/libavfilter/af_resample.c
> > > @@ -0,0 +1,467 @@
> > > +/*
> > > + * Copyright (C) 2010 S.N. Hemanth Meenakshisundaram <smeenaks at ucsd.edu>
> > > + * based on code in libavcodec/resample.c by Fabrice Bellard
> > > + * and libavcodec/audioconvert.c by Michael Neidermayer
> > > + *
> > > + * This file is part of FFmpeg.
> > > + *
> > > + * FFmpeg is free software; you can redistribute it and/or
> > > + * modify it under the terms of the GNU Lesser General Public
> > > + * License as published by the Free Software Foundation; either
> > > + * version 2.1 of the License, or (at your option) any later version.
> > > + *
> > > + * FFmpeg is distributed in the hope that it will be useful,
> > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > > + * Lesser General Public License for more details.
> > > + *
> > > + * You should have received a copy of the GNU Lesser General Public
> > > + * License along with FFmpeg; if not, write to the Free Software
> > > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> > > + */
> > > +
> > > +/**
> > > + * @file
> > > + * resample audio filter
> > > + */
> > > +
> > > +#include "avfilter.h"
> > > +#include "libavcodec/audioconvert.h"
> > > +
> >
> > > +typedef struct {
> > > + short reconfig_channel_layout; ///< set when channel layout of incoming buffer changes
> > > + short reconfig_sample_fmt; ///< set when sample format of incoming buffer changes
> >
> > short?
> >
> >
> > [...]
> > > +/**
> > > + * This is for when we have more than 2 input channels, need to downmix to mono
> > > + * and do not have a conversion formula available. We just use first two input
> > > + * channels - left and right. This is a placeholder until more conversion
> > > + * functions are written.
> > > + */
> > > +static void mono_downmix(uint8_t *out[], uint8_t *in[], int nb_samples, int in_channels)
> > > +{
> > > + int i;
> > > + short *input = (short *) in[0];
> > > + short *output = (short *) out[0];
> > > + short left, right;
> > > +
> > > + for (i = 0; i < nb_samples; i++) {
> > > + left = *input++;
> > > + right = *input++;
> >
> > > + *output++ = (left>>1)+(right>>1);
> >
> > >> after +
> >
> >
> >
> > > + input += in_channels-2;
> > > + }
> > > +}
> > > +
> >
> > > +/* Stereo to 5.1 output */
> > > +static void ac3_5p1_mux(uint8_t *out[], uint8_t *in[], int nb_samples, int in_channels)
> > > +{
> > > + int i;
> > > + short *output = (short *) out[0];
> > > + short *input = (short *) in[0];
> > > + short left, right;
> > > +
> > > + for (i = 0; i < nb_samples; i++) {
> > > + left = *input++; /* Grab next left sample */
> > > + right = *input++; /* Grab next right sample */
> > > + *output++ = left; /* left */
> > > + *output++ = right; /* right */
> > > + *output++ = (left>>1)+(right>>1); /* center */
> > > + *output++ = 0; /* low freq */
> > > + *output++ = 0; /* FIXME: left surround is either -3dB, -6dB or -9dB of stereo left */
> > > + *output++ = 0; /* FIXME: right surroud is either -3dB, -6dB or -9dB of stereo right */
> > > + }
> > > +}
> >
> > that are alot of fixmes
> >
> > also we need float versions
> >
> >
> > [...]
> > > +static void convert_channel_layout(AVFilterLink *link)
> > > +{
> > > + ResampleContext *resample = link->dst->priv;
> > > + AVFilterBufferRef *insamples = resample->s16_samples_ptr;
> > > + AVFilterBufferRef *outsamples = resample->temp_samples;
> > > + unsigned int num_ip_channels = avcodec_channel_layout_num_channels(resample->in_channel_layout);
> > > +
> > > + if (insamples)
> > > + resample->in_channel_layout = insamples->audio->channel_layout;
> > > +
> > > + /* Init stage or input channels changed, so reconfigure conversion function pointer */
> > > + if (resample->reconfig_channel_layout || !resample->channel_conversion) {
> > > +
> > > + int64_t in_channel = resample->in_channel_layout;
> > > + int64_t out_channel = resample->out_channel_layout;
> > > +
> > > + int num_channels = avcodec_channel_layout_num_channels(resample->out_channel_layout);
> > > + int out_sample_size = av_get_bits_per_sample_format(insamples->format) >> 3;
> > > +
> > > + int size = num_channels*out_sample_size*insamples->audio->samples_nb;
> > > +
> > > + if (outsamples)
> > > + avfilter_unref_buffer(outsamples);
> > > + outsamples = avfilter_get_audio_buffer(link, AV_PERM_WRITE|AV_PERM_REUSE2,
> > > + insamples->format, size,
> > > + out_channel, 0);
> > > + /*
> > > + * Pick appropriate channel conversion function based on input-output channel layouts.
> > > + * If no suitable conversion function is available, downmix to stereo and set buffer
> > > + * channel layout to stereo.
> > > + *
> > > + * FIXME: Add error handling if channel conversion is unsupported, more channel conversion
> > > + * routines and finally the ability to handle various stride lengths (sample formats).
> > > + */
> > > + if (in_channel == CH_LAYOUT_STEREO && out_channel == CH_LAYOUT_MONO)
> > > + resample->channel_conversion = stereo_to_mono;
> > > + else if (in_channel == CH_LAYOUT_MONO && out_channel == CH_LAYOUT_STEREO)
> > > + resample->channel_conversion = mono_to_stereo;
> > > + else if (in_channel == CH_LAYOUT_STEREO && out_channel == CH_LAYOUT_5POINT1)
> > > + resample->channel_conversion = ac3_5p1_mux;
> > > + else if (out_channel == CH_LAYOUT_MONO)
> > > + resample->channel_conversion = mono_downmix;
> >
> > missig {}
>
> Updated against latest SVN. Consider this a work in progress, many of
> the fixme need yet to be fixed, Michael please mark what you consider
> required for the patch to be applied.
the short types must be fixed IMHO
and its (left+right)>>1 not (left>>1)+(right>>1); (trivial)
The rest from my previous review can wait until after svn
Theres also another issue that i think must be fixed and that is the naming
There are 3 things
1. Changing the sample rate
2. Changing the sample format (int16<->int8<->float)
3. Changing the channel layout (stereo<->mono<->5.1)
1. is AFIAK called resampling
2. is AFIAK called requantization or sample format convertion
3. is AFIAK called rematrixing or up/down mixing
If someone has some authorative (_NOT_ wikipedia) source that define these
terms that cant hurt ...
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Frequently ignored awnser#1 FFmpeg bugs should be sent to our bugtracker. User
questions about the command line tools should be sent to the ffmpeg-user ML.
And questions about how to use libav* should be sent to the libav-user ML.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20110113/60029498/attachment.pgp>
More information about the ffmpeg-devel
mailing list