[FFmpeg-devel] [OPW] OPW Project Proposal

Pallavi Kumari pallavikumari.in at gmail.com
Fri Nov 4 04:16:34 EET 2016


Updated patch attached. PFA

On Thu, Nov 3, 2016 at 2:00 AM, Michael Niedermayer <michael at niedermayer.cc>
wrote:

> On Thu, Nov 03, 2016 at 01:10:26AM +0530, Pallavi Kumari wrote:
> > Necessary changes has been done. PFA.
> >
> > Usage:
> >
> > ./ffmpeg -i kpg.mp3 -filter_complex peakpoints=wsize=16 -f null -
> >
> > On Wed, Nov 2, 2016 at 6:14 AM, Michael Niedermayer
> <michael at niedermayer.cc>
> > wrote:
> >
> > > On Wed, Nov 02, 2016 at 05:00:09AM +0530, Pallavi Kumari wrote:
> > > > Hi Michael,
> > > >
> > > > I have attached a working patch with the mail. PFA.
> > > >
> > > > Usage:
> > > >
> > >
> > > > ./ffmpeg -i kpg.mp3 -filter_complex peakpoints=input=kpg.mp3:
> wsize=16
> > >
> > > I realize now, theres a mistake in this, you must provide a output
> > > as in
> > > ./ffmpeg -i kpg.mp3 -af peakpoints -f null -
> > >
> > > without some output like  "-f null -" it wont read the file fully and
> > > wont pass it through filter_frame()
> > >
> > > you could see this failure as in:
> > > ./ffmpeg -i ~/videos/matrixbench_mpeg2.mpg -af volumedetect  -f null -
> > > vs.
> > > ./ffmpeg -i ~/videos/matrixbench_mpeg2.mpg -af volumedetect
> > >
> > > you get the histogram from the volume detect filter in the first case
> > > but not the 2nd.
> > >
> > > [...]
> > > --
> > > Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC7
> 87040B0FAB
> > >
> > > Many things microsoft did are stupid, but not doing something just
> because
> > > microsoft did it is even more stupid. If everything ms did were stupid
> they
> > > would be bankrupt already.
> > >
>
> >  af_peakpoints.c |  226 ++++++++++++++++++++++++++++++
> ++++++++++++++++++++++++++
> >  1 file changed, 226 insertions(+)
> > 206d91b47bc6066dd01db1c3369d4674ac95f04c  0001-avfilter-added-
> peakpoints-filter.patch
> > From e10f73d363d0313774bcb132b3b1f2417fcfba11 Mon Sep 17 00:00:00 2001
> > From: Atana <atana at openmailbox.org>
> > Date: Thu, 3 Nov 2016 01:05:51 +0530
> > Subject: [PATCH] avfilter: added peakpoints filter
> >
> > ---
> >  libavfilter/af_peakpoints.c | 226 ++++++++++++++++++++++++++++++
> ++++++++++++++
> >  1 file changed, 226 insertions(+)
> >  create mode 100644 libavfilter/af_peakpoints.c
>
> This is missing changes to the Makefile and libavfilter/allfilters*
>
>
> >
> > diff --git a/libavfilter/af_peakpoints.c b/libavfilter/af_peakpoints.c
> > new file mode 100644
> > index 0000000..9265c47
> > --- /dev/null
> > +++ b/libavfilter/af_peakpoints.c
> > @@ -0,0 +1,226 @@
> > +/*
> > + * Copyright (c) 2016 Atana
> > + *
> > + * This file is part of FFmpeg.
> > + *
> > + * FFmpeg is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU Lesser General Public
> > + * License as published by the Free Software Foundation; either
> > + * version 2.1 of the License, or (at your option) any later version.
> > + *
> > + * FFmpeg is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > + * Lesser General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU Lesser General Public
> > + * License along with FFmpeg; if not, write to the Free Software
> > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> 02110-1301 USA
> > + */
> > +
> > +#include "libavcodec/avcodec.h"
> > +#include "libavcodec/avfft.h"
> > +#include "libavformat/avformat.h"
> > +#include "libswscale/swscale.h"
> > +#include "avfilter.h"
> > +#include "audio.h"
> > +#include "libavutil/opt.h"
> > +
> > +
> > +/* Structure to contain peak points context */
> > +typedef struct {
> > +    const AVClass *class;
> > +    double *data;
> > +    int nsamples;
> > +    int index;
> > +    double *peaks;
> > +    int size; // number of peaks
> > +    int windowSize;
> > +    //char *inputFile;
> > +} PeakPointsContext;
> > +
> > +/* returns maximum value from an array conditioned on start and end
> index */
> > +static double getMax(double *res_arr, int startIndex, int endIndex) {
> > +    int i;
> > +    double max = res_arr[startIndex];
> > +    for (i = startIndex; i <= endIndex; i++) {
> > +         if (res_arr[i] > max) {
> > +             max = res_arr[i];
> > +         }
> > +    }
> > +    return max;
> > +}
> > +
> > +/* Stores peak frequency for each window(of chunkSize) in peaks array */
> > +static void getPeakPointInChunk(int chunkSize, double *res_arr, int
> size, double *peaks) {
> > +    int i = 0, peakIndex = 0;
> > +    int startIndex = 0;
> > +    double max;
> > +    // get a chunk and find max value in it
> > +    while (i < size) {
> > +         if (i % chunkSize-1 == 0) {
> > +            max = getMax(res_arr, startIndex, i);
> > +             peaks[peakIndex++] = max;
> > +             startIndex = startIndex + chunkSize;
> > +         }
> > +        i += 1;
> > +    }
> > +}
> > +
> > +/* Get peaks points from windowed frequency domain data*/
> > +static int getPeakPoints(PeakPointsContext *ppc) {
> > +    int i, m, k, size, chunkSize, pSize, chunkSampleSize, resSize;
> > +    double *fft_res;
> > +    void *avc;
> > +    RDFTContext *rdftC;
> > +    FFTSample *data;
> > +
> > +    size = ppc->index;
> > +    m = log2(ppc->windowSize);
> > +    chunkSize = ppc->windowSize;
> > +    chunkSampleSize = size/chunkSize;
> > +    resSize = chunkSize * chunkSampleSize;
> > +
> > +    fft_res = av_malloc(sizeof(double) * resSize);
> > +
> > +    if (!fft_res) {
> > +        av_log(avc, AV_LOG_ERROR, "Cann't allocate memmory for storing
> fft data\n");
> > +        return 0;
> > +    }
> > +
> > +
> > +    rdftC = av_rdft_init(m, DFT_R2C);
>
> > +    data = av_malloc(sizeof(FFTSample)*chunkSize);
>
> see av_malloc_array() (it avoids potential issues with the multiply
> overflowing)
>
>
> > +
> > +    if (!data) {
> > +        av_log(avc, AV_LOG_ERROR, "Cann't allocate memmory for chunk
> fft data\n");
> > +        return 0;
> > +    }
> > +    // FFT transform for windowed time domain data
> > +    // window is of size chunkSize
> > +    k = 0;
> > +    while (k < resSize) {
> > +        //copy data
> > +        for (i = 0; i < chunkSize; i++) {
> > +            data[i] = ppc->data[i+k];
> > +        }
> > +        //calculate FFT
> > +        av_rdft_calc(rdftC, data);
> > +        for (i = 0; i < chunkSize; i++) {
> > +         fft_res[i+k] = data[i];
> > +        }
> > +        k = k + chunkSize;
> > +    }
> > +
> > +    av_rdft_end(rdftC);
> > +    pSize = resSize/chunkSize;
> > +    ppc->size = pSize;
> > +    ppc->peaks = av_malloc(sizeof(double)*pSize);
> > +
> > +    if (!ppc->peaks) {
> > +        av_log(avc, AV_LOG_ERROR, "Cann't allocate memory for peak
> storage\n");
> > +        return 0;
> > +    }
> > +
> > +    getPeakPointInChunk(chunkSize, fft_res, resSize, ppc->peaks);
> > +    return 1;
> > +}
> > +
> > +
> > +#define OFFSET(x) offsetof(PeakPointsContext, x)
> > +
> > +static const AVOption peakpoints_options[] = {
> > +    { "wsize",  "set window size", OFFSET(windowSize),
> AV_OPT_TYPE_INT,    {.i64=16},    0, INT_MAX},
> > +    { NULL },
> > +};
> > +
> > +AVFILTER_DEFINE_CLASS(peakpoints);
> > +
>
> > +static av_cold int init(AVFilterContext *ctx)
> > +{
> > +    PeakPointsContext *p = ctx->priv;
> > +
> > +    if (p->windowSize < 16) {
> > +         av_log(ctx, AV_LOG_ERROR, "window size must be greater than or
> equal to 16\n");
> > +        return AVERROR(EINVAL);
> > +    }
> > +
> > +    p->index = 0;
> > +    p->size = 0;
>
> > +    p->data = av_malloc(sizeof(double)*10000);
>
> nothing gurantees that 10000 or any constant is large enough
> indeed it is not guranteed that all the decoded audio would fit in
> memory.
> What probably makes most sense is executing the
> fft from filter_frame() every time there is sufficient new data
> and overwriting the old data with new instead of trying to store
> all audio data
>
>
> > +
> > +    if (!p->data) {
> > +        av_log(ctx, AV_LOG_ERROR, "Cann't allocate memmory for audio
> data\n");
> > +        return AVERROR(EINVAL);
> > +    }
> > +
> > +    return 0;
> > +}
> > +
> > +static int filter_frame(AVFilterLink *inlink, AVFrame *samples)
> > +{
> > +    AVFilterContext *ctx = inlink->dst;
> > +    PeakPointsContext *p = ctx->priv;
> > +
> > +    // store audio data
>
> > +    p->data[p->index] = (double)*samples->data[0];
>
> There are multiple channels and multiple samples
> this uses just the first sample of the first channel
>
> samples->nb_samples contains the number of samples (as in time)
> and av_frame_get_channels(samples) is the number of channels
>
> you can see in af_volumedetect.c how to access the samples of the
> channels.
> I guess its ok to use just the first channel for now but all samples
> of the frames (timewise) should be used
>
>
> > +    p->index = p->index + 1;
> > +
> > +    return ff_filter_frame(inlink->dst->outputs[0], samples);
> > +}
> > +
> > +static void ppointsStats(AVFilterContext *ctx, PeakPointsContext *p) {
> > +    int i, ret;
> > +    ret = getPeakPoints(p);
> > +
> > +    if (ret && p->size) {
> > +     // print peaks
> > +        av_log(ctx, AV_LOG_INFO, "######## Peak points are ########\n");
> > +        for (i = 0; i < p->size; i++) {
> > +             av_log(ctx, AV_LOG_INFO, "%f\n", p->peaks[i]);
> > +        }
> > +    } else if (p->size || !ret) {
> > +        av_log(ctx, AV_LOG_ERROR, "Peak points not retrieved\n");
> > +        return;
> > +    }
> > +}
> > +
> > +static av_cold void uninit(AVFilterContext *ctx)
> > +{
> > +    PeakPointsContext *p = ctx->priv;
> > +
> > +    ppointsStats(ctx, p);
> > +
> > +    // free allocated memories
> > +    av_freep(&p->data);
> > +    av_freep(&p->peaks);
> > +}
> > +
> > +static const AVFilterPad peakpoints_inputs[] = {
> > +    {
> > +        .name         = "default",
> > +        .type         = AVMEDIA_TYPE_AUDIO,
> > +        .filter_frame = filter_frame,
> > +    },
> > +    { NULL }
> > +};
> > +
> > +static const AVFilterPad peakpoints_outputs[] = {
> > +    {
> > +        .name = "default",
> > +        .type = AVMEDIA_TYPE_AUDIO,
> > +    },
> > +    { NULL }
> > +};
> > +
> > +AVFilter ff_af_peakpoints = {
> > +    .name          = "peakpoints",
> > +    .description   = NULL_IF_CONFIG_SMALL("peak points from frequency
> domain windowed data."),
> > +    .init          = init,
> > +    .uninit        = uninit,
>
> > +    //.query_formats = query_formats,
>
> without query_formats you cannot be sure that the data you get is
> if double type, see volumedetect for an example implementation, you
> only need to adapt it so it requests double and not S16 integers
>
>
> [...]
>
>
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> The greatest way to live with honor in this world is to be what we pretend
> to be. -- Socrates
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-avfilter-added-peakpoints-filter.patch
Type: text/x-patch
Size: 9769 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20161104/bab18b20/attachment.bin>


More information about the ffmpeg-devel mailing list