[FFmpeg-devel] [PATCH] Add af_resample - sample fmt and channel layout conversion filter
Stefano Sabatini
stefano.sabatini-lala
Fri Aug 27 02:31:13 CEST 2010
On date Thursday 2010-08-26 01:51:34 -0700, S.N. Hemanth Meenakshisundaram encoded:
>
> Updated to parse string channel layouts, sample formats as well. Other
> questions below remain from yesterday's email.
>
> On 08/23/2010 08:13 AM, Stefano Sabatini wrote:
> > On date Monday 2010-08-23 00:11:52 -0700, S.N. Hemanth Meenakshisundaram encoded:
> >
> >> Fixed the query formats function so that a resample filter can be
> >> inserted between filters with incompatible sample formats.
> >>
> >>
> >> +
> >> +/* Stereo to 5.1 output */
> >> +static void ac3_5p1_mux(uint8_t *out[], uint8_t *in[], int samples_nb, int in_channels)
> >> +{
> >> + int i;
> >> + short *output = (short *) out[0];
> >> + short *input = (short *) in[0];
> >> + short left, right;
> >> +
> >> + for (i = 0; i < samples_nb; i++) {
> >> + left = *input++; /* Grab next left sample */
> >> + right = *input++; /* Grab next right sample */
> >> + *output++ = left; /* left */
> >> + *output++ = right; /* right */
> >> + *output++ = (left>>1)+(right>>1); /* center */
> >> + *output++ = 0; /* low freq */
> >> + *output++ = 0; /* FIXME: left surround is either -3dB, -6dB or -9dB of stereo left */
> >> + *output++ = 0; /* FIXME: right surroud is either -3dB, -6dB or -9dB of stereo right */
> >> + }
> >> +}
> >>
> > I suppose this code is taken from libavcodec, so there is no point
> > into reviewing it. Maybe a better idea, rather than duplicate the
> > code, could be to try to export this API as internal functions
> > (ff_prefixed).
> >
> >
>
> The issue with the lavc code for channel mixing is that the functions
> have varying number of arguments. So, depending on the channel
> conversion, we would need to have a different call. Here all functions
> operate on the same arguments.
>
> Is it ok to have a switch case or if-else-if with different calls and
> use the existing lavc functions?
Sure, or even better use a table.
> >> + sscanf(args, "%d:%ld", &resample->out_sample_fmt,
> >> + &resample->out_channel_layout);
> >>
> > possibly it should take "%s:%s", sample_fmt <=> int mapping should be
> > already supported by the API, as for the channel layout there is your
> > patch waiting for review.
> >
> >
>
> Sure. Will do once that patch is in/modified otherwise we would have one
> string and one numeric argument now. I have a patch for this already.
>
> > Possibly print a string here for the sample format.
> > Same here, better to give a string representation.
> >
>
> Both these print the sample format/channel layout only if they are not
> in the table, so they would have no valid string representation.
>
> >> [...]
> >> + for (sample_fmt = 0; sample_fmt < SAMPLE_FMT_NB; sample_fmt++)
> >> + if ((ret = avfilter_add_format(&formats, sample_fmt)) < 0) {
> >> + avfilter_formats_unref(&formats);
> >> + return ret;
> >> + }
> >>
> > formats = avfilter_all_formats()
> >
>
> Done.
>
> > [...]
> > why not to call av_audio_convert()?
> >
>
> Done. This patch only works on top of the fix for
> default_get_audio_buffer that I posted just before this.
>
>
> > [...]
> > Missing docs (not high priority, feel free to add it when we're
> > approaching to the final version).
> >
>
> Will add once we have a version closer to the final one.
>
> > Regards.
> >
>
> The posted patch can be tested with
>
> ./ffplay -af resample=<sample_fmt>:<channel_layout> in.ogg
>
> Passes 'make test' and on top of SVN latest. No errors from valgrind.
>
> Regards,
[...]
> diff --git a/libavfilter/af_resample.c b/libavfilter/af_resample.c
> new file mode 100644
> index 0000000..abd0d3c
> --- /dev/null
> +++ b/libavfilter/af_resample.c
> @@ -0,0 +1,476 @@
[...]
> +/**
> + * All of the routines below are for packed audio data. SDL accepts packed data
> + * only and current ffplay also assumes packed data only at all times.
> + */
> +
> +/* Optimized stereo to mono and mono to stereo routines - common case */
> +static void stereo_to_mono(uint8_t *out[], uint8_t *in[], int samples_nb, int in_channels)
> +{
> + short *input = (short *) in[0];
> + short *output = (short *) out[0];
> +
> + while (samples_nb >= 4) {
> + output[0] = (input[0] + input[1]) >> 1;
> + output[1] = (input[2] + input[3]) >> 1;
> + output[2] = (input[4] + input[5]) >> 1;
> + output[3] = (input[6] + input[7]) >> 1;
> + output += 4;
> + input += 8;
> + samples_nb -= 4;
> + }
> + while (samples_nb > 0) {
> + output[0] = (input[0] + input[1]) >> 1;
> + output++;
> + input += 2;
> + samples_nb--;
> + }
> +}
> +
> +static void mono_to_stereo(uint8_t *out[], uint8_t *in[], int samples_nb, int in_channels)
> +{
> + int v;
> + short *input = (short *) in[0];
> + short *output = (short *) out[0];
> +
> +
> + while (samples_nb >= 4) {
> + v = input[0]; output[0] = v; output[1] = v;
> + v = input[1]; output[2] = v; output[3] = v;
> + v = input[2]; output[4] = v; output[5] = v;
> + v = input[3]; output[6] = v; output[7] = v;
> + output += 8;
> + input += 4;
> + samples_nb -= 4;
> + }
> + while (samples_nb > 0) {
> + v = input[0]; output[0] = v; output[1] = v;
> + output += 2;
> + input += 1;
> + samples_nb--;
> + }
> +}
> +
> +/**
> + * This is for when we have more than 2 input channels, need to downmix to
> + * stereo and do not have a conversion formula available. We just use first
> + * two input channels - left and right. This is a placeholder until more
> + * conversion functions are written.
> + */
> +static void stereo_downmix(uint8_t *out[], uint8_t *in[], int samples_nb, int in_channels)
> +{
> + int i;
> + short *output = (short *) out[0];
> + short *input = (short *) out[0];
> +
> + for (i = 0; i < samples_nb; i++) {
> + *output++ = *input++;
> + *output++ = *input++;
> + input+=(in_channels-2);
> + }
> +}
> +
> +/**
> + * This is for when we have more than 2 input channels, need to downmix to mono
> + * and do not have a conversion formula available. We just use first two input
> + * channels - left and right. This is a placeholder until more conversion
> + * functions are written.
> + */
> +static void mono_downmix(uint8_t *out[], uint8_t *in[], int samples_nb, int in_channels)
> +{
> + int i;
> + short *input = (short *) in[0];
> + short *output = (short *) out[0];
> + short left, right;
> +
> + for (i = 0; i < samples_nb; i++) {
> + left = *input++;
> + right = *input++;
> + *output++ = (left>>1)+(right>>1);
> + input+=(in_channels-2);
> + }
> +}
> +
> +/* Stereo to 5.1 output */
> +static void ac3_5p1_mux(uint8_t *out[], uint8_t *in[], int samples_nb, int in_channels)
> +{
> + int i;
> + short *output = (short *) out[0];
> + short *input = (short *) in[0];
> + short left, right;
> +
> + for (i = 0; i < samples_nb; i++) {
> + left = *input++; /* Grab next left sample */
> + right = *input++; /* Grab next right sample */
> + *output++ = left; /* left */
> + *output++ = right; /* right */
> + *output++ = (left>>1)+(right>>1); /* center */
> + *output++ = 0; /* low freq */
> + *output++ = 0; /* FIXME: left surround is either -3dB, -6dB or -9dB of stereo left */
> + *output++ = 0; /* FIXME: right surroud is either -3dB, -6dB or -9dB of stereo right */
> + }
> +}
Maybe all these lavc functions should be somehow exported (e.g. as internal
ff_ functions), Michael has the last word.
> +static av_cold int init(AVFilterContext *ctx, const char *args, void *opaque)
> +{
> + ResampleContext *resample = ctx->priv;
> + char sample_fmt_str[16], ch_layout_str[16];
> +
> + if (args) sscanf(args, "%15[a-z0-9]:%15[a-z0-9]", sample_fmt_str, ch_layout_str);
> +
> + resample->out_sample_fmt = avcodec_get_sample_fmt(sample_fmt_str);
> +
> + if (*sample_fmt_str && (resample->out_sample_fmt >= SAMPLE_FMT_NB ||
> + resample->out_sample_fmt <= SAMPLE_FMT_NONE)) {
> + /**
> + * SAMPLE_FMT_NONE is a valid value for out_sample_fmt and indicates no
> + * change in sample format.
> + */
> + char *tail;
> + resample->out_sample_fmt = strtol(sample_fmt_str, &tail, 10);
> + if (*tail || resample->out_sample_fmt >= SAMPLE_FMT_NB ||
> + resample->out_sample_fmt < SAMPLE_FMT_NONE) {
> + av_log(ctx, AV_LOG_ERROR, "Invalid sample format %s\n", sample_fmt_str);
> + return AVERROR(EINVAL);
> + }
> + }
> +
> + resample->out_channel_layout = *ch_layout_str ?
> + avcodec_get_channel_layout(ch_layout_str) : -1;
> +
> + if (*ch_layout_str && resample->out_channel_layout < CH_LAYOUT_STEREO) {
> + /**
> + * -1 is a valid value for out_channel_layout and indicates no change
> + * in channel layout.
> + */
> + char *tail;
> + resample->out_channel_layout = strtol(ch_layout_str, &tail, 10);
> + if (*tail || (resample->out_channel_layout < CH_LAYOUT_STEREO &&
> + resample->out_channel_layout != -1)) {
> + av_log(ctx, AV_LOG_ERROR, "Invalid channel layout %s\n", ch_layout_str);
> + return AVERROR(EINVAL);
> + }
> + }
> +
> + /* Set default values for expected incoming sample format and channel layout */
> + resample->in_channel_layout = CH_LAYOUT_STEREO;
> + resample->in_sample_fmt = SAMPLE_FMT_S16;
> + resample->in_samples_nb = 0;
> + /* We do not yet know the channel conversion function to be used */
> + resample->channel_conversion = NULL;
> +
> + return 0;
> +}
> +
> +static av_cold void uninit(AVFilterContext *ctx)
> +{
> + ResampleContext *resample = ctx->priv;
> + if (resample->s16_samples)
> + avfilter_unref_buffer(resample->s16_samples);
> + if (resample->temp_samples)
> + avfilter_unref_buffer(resample->temp_samples);
> + if (resample->out_samples)
> + avfilter_unref_buffer(resample->out_samples);
> + if (resample->conv_handle_s16)
> + av_audio_convert_free(resample->conv_handle_s16);
> + if (resample->conv_handle_out)
> + av_audio_convert_free(resample->conv_handle_out);
> +}
> +
> +static int query_formats(AVFilterContext *ctx)
> +{
> + AVFilterFormats *formats;
> +
> + if (ctx->inputs[0]) {
> + formats = NULL;
> + formats = avfilter_all_formats(AVMEDIA_TYPE_AUDIO);
> + avfilter_formats_ref(formats, &ctx->inputs[0]->out_formats);
> + }
> + if (ctx->outputs[0]) {
> + formats = NULL;
> + formats = avfilter_all_formats(AVMEDIA_TYPE_AUDIO);
> + avfilter_formats_ref(formats, &ctx->outputs[0]->in_formats);
> + }
> +
> + return 0;
> +}
> +
> +static void convert_channel_layout(AVFilterLink *link)
> +{
> + ResampleContext *resample = link->dst->priv;
> + AVFilterBufferRef *insamples = resample->s16_samples_ptr;
> + AVFilterBufferRef *outsamples = resample->temp_samples;
> + unsigned int num_ip_channels = avcodec_channel_layout_num_channels(resample->in_channel_layout);
> +
> + if (insamples)
> + resample->in_channel_layout = insamples->audio->channel_layout;
> +
> + /* Init stage or input channels changed, so reconfigure conversion function pointer */
> + if (resample->reconfig_channel_layout || !resample->channel_conversion) {
> +
> + int64_t in_channel = resample->in_channel_layout;
> + int64_t out_channel = resample->out_channel_layout;
> +
> + int num_channels = avcodec_channel_layout_num_channels(resample->out_channel_layout);
> + int out_sample_size = av_get_bits_per_sample_format(insamples->format) >> 3;
> +
> + int size = num_channels*out_sample_size*insamples->audio->samples_nb;
> +
> + if (outsamples)
> + avfilter_unref_buffer(outsamples);
> + outsamples = avfilter_get_audio_buffer(link, AV_PERM_WRITE|AV_PERM_REUSE2,
> + insamples->format, size,
> + out_channel, 0);
> + /*
> + * Pick appropriate channel conversion function based on input-output channel layouts.
> + * If no suitable conversion function is available, downmix to stereo and set buffer
> + * channel layout to stereo.
> + *
> + * FIXME: Add error handling if channel conversion is unsupported, more channel conversion
> + * routines and finally the ability to handle various stride lengths (sample formats).
> + */
> +
> + if ((in_channel == CH_LAYOUT_STEREO) &&
> + (out_channel == CH_LAYOUT_MONO))
> + resample->channel_conversion = stereo_to_mono;
> + else if ((in_channel == CH_LAYOUT_MONO) &&
> + (out_channel == CH_LAYOUT_STEREO))
> + resample->channel_conversion = mono_to_stereo;
> + else if ((in_channel == CH_LAYOUT_STEREO) &&
> + (out_channel == CH_LAYOUT_5POINT1))
> + resample->channel_conversion = ac3_5p1_mux;
> + else if (out_channel == CH_LAYOUT_MONO)
> + resample->channel_conversion = mono_downmix;
> + else {
> + resample->channel_conversion = stereo_downmix;
> + outsamples->audio->channel_layout = CH_LAYOUT_STEREO;
This maybe can improve in readability using a table.
[...]
Regards.
--
FFmpeg = Faithless and Fucking Mere Picky Enigmatic Guide
More information about the ffmpeg-devel
mailing list