[FFmpeg-devel] [PATCH] lavfi/WIP: vignette filter.

Clément Bœsch ubitux at gmail.com
Wed May 29 02:44:34 CEST 2013


On Mon, May 13, 2013 at 12:31:54AM +0200, Michael Niedermayer wrote:
> On Fri, May 10, 2013 at 06:49:22PM +0200, Clément Bœsch wrote:
> > On Thu, Apr 04, 2013 at 05:15:29PM +0200, Clément Bœsch wrote:
> > [...]
> > 
> > New version attached.
> [...]
> 
> > +typedef struct {
> > +    const AVClass *class;
> > +    const AVPixFmtDescriptor *desc;
> > +    int backward;
> > +    enum EvalMode { EVAL_MODE_INIT, EVAL_MODE_FRAME, EVAL_MODE_NB } eval_mode;
> > +#define DEF_EXPR_FIELDS(name) AVExpr *name##_pexpr; char *name##_expr; double name;
> > +    DEF_EXPR_FIELDS(a);
> > +    DEF_EXPR_FIELDS(x0);
> > +    DEF_EXPR_FIELDS(y0);
> > +    double var_values[VAR_NB];
> 
> > +    double *fmap;
> 
> isnt double overkill ?
> 

Yeah likely. Replaced with float, same for xscale and yscale.

> 
> [...]
> > +static double get_natural_factor(const VignetteContext *s, int x, int y)
> > +{
> > +    const int xx = (x - s->x0) * s->xscale;
> > +    const int yy = (y - s->y0) * s->yscale;
> > +    const double dnorm = hypot(xx, yy) / s->dmax;
> > +    if (dnorm > 1) {
> > +        return 0;
> > +    } else {
> > +        const double c = cos(s->a * dnorm);
> > +        return (c*c)*(c*c); // XXX: do not remove braces, it helps compilers
> > +    }
> > +}
> 
> this looks a bit unflexible, how well does this work for removing
> actual vignetting from pictures taken with actual lenses ?
> 

I looked more closely at the lens correction, but it actually looks like
it would belong in a separate filter to me.

Typically, with lens correction you end up with the following formula:

  r = a*r^4 + b*r^3 + c*r^2 + d*r

input r being the distance from the origin of the lens to the current
pixel (radius) and output r is the new radius. Then you likely have to
distort the image to achieve the effect; basically "moving pixels" (barrel
distortion, or its inverse pincushion) which is not what we are doing in
this filter.

The cosine 4 formula I use here seems to be the common way of simulating
the lightning vignetting effect: https://en.wikipedia.org/wiki/Vignetting
more precisely, the Natural Vignetting (which I now mention in the
filter's documentation)

Another lens filter, with different parameters (a, b, c and d) could
re-use the basis of this filter, but I believe it's a different purpose
than what I'm trying to achieve here.

[...]
> > +static int filter_frame(AVFilterLink *inlink, AVFrame *in)
> > +{
> > +    unsigned x, y;
> > +    AVFilterContext *ctx = inlink->dst;
> > +    VignetteContext *s = ctx->priv;
> > +    AVFilterLink *outlink = inlink->dst->outputs[0];
> > +    AVFrame *out;
> > +
> > +    out = ff_get_video_buffer(outlink, outlink->w, outlink->h);
> > +    if (!out) {
> > +        av_frame_free(&in);
> > +        return AVERROR(ENOMEM);
> > +    }
> > +    av_frame_copy_props(out, in);
> > +
> > +    if (s->eval_mode == EVAL_MODE_FRAME)
> > +        update_context(s, inlink, in);
> > +
> > +    if (s->desc->flags & PIX_FMT_RGB) {
> > +        uint8_t       *dst = out->data[0];
> > +        const uint8_t *src = in ->data[0];
> > +        const double *fmap = s->fmap;
> > +        const int dst_linesize = out->linesize[0];
> > +        const int src_linesize = in ->linesize[0];
> > +        const int fmap_linesize = s->fmap_linesize;
> > +
> > +        for (y = 0; y < inlink->h; y++) {
> > +            uint8_t       *dstp = dst;
> > +            const uint8_t *srcp = src;
> > +
> > +            for (x = 0; x < inlink->w; x++, dstp += 3, srcp += 3) {
> > +                const double f = fmap[x];
> > +
> > +                dstp[0] = av_clip_uint8(srcp[0] * f);
> > +                dstp[1] = av_clip_uint8(srcp[1] * f);
> > +                dstp[2] = av_clip_uint8(srcp[2] * f);
> 
> i think with just 8bit this kind of filter is prone to banding
> artifacts without dither
> 

Yep, you are right, I've added some dithering (taken from mandelbrot,
thank you).

[...]

New patch attached.

-- 
Clément B.
-------------- next part --------------
From 72d07081bac8d6bb2de4f0428f0068faf9539a3d Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Cl=C3=A9ment=20B=C5=93sch?= <ubitux at gmail.com>
Date: Tue, 2 Apr 2013 12:48:32 +0200
Subject: [PATCH] lavfi: vignette filter.

TODO: minor bump
---
 doc/filters.texi          |  98 ++++++++++++++
 libavfilter/Makefile      |   1 +
 libavfilter/allfilters.c  |   1 +
 libavfilter/vf_vignette.c | 330 ++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 430 insertions(+)
 create mode 100644 libavfilter/vf_vignette.c

diff --git a/doc/filters.texi b/doc/filters.texi
index 75a2d6a..65637ea 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -6950,6 +6950,104 @@ For example, to vertically flip a video with @command{ffmpeg}:
 ffmpeg -i in.avi -vf "vflip" out.avi
 @end example
 
+ at section vignette
+
+Make or reverse a natural vignetting effect.
+
+The filter accepts the following options:
+
+ at table @option
+ at item angle, a
+Set lens angle expression as a number of radians.
+
+The value is clipped in the @code{[0,PI/2]} range.
+
+Default value: "PI/5"
+
+ at item x0
+ at item y0
+Set center coordinates expressions. Respectively @code{"w/2"} and @code{"h/2"}
+by default.
+
+ at item mode
+Set forward/backward mode.
+
+Available modes are:
+ at table @samp
+ at item forward
+The larger the distance from the central point, the darker the image becomes.
+
+ at item backward
+The larger the distance from the central point, the brighter the image becomes.
+This can be used to reverse a vignette effect, though there is no automatic
+detection to extract the lens @option{angle} and other settings (yet).
+ at end table
+
+Default value is @samp{forward}.
+
+ at item eval
+Set evaluation mode for the expressions (@option{angle}, @option{x0}, @option{y0}).
+
+It accepts the following values:
+ at table @samp
+ at item init
+Evaluate expressions only once during the filter initialization.
+
+ at item frame
+Evaluate expressions for each incoming frame. This is way slower than the
+ at samp{init} mode since it requires all the scalers to be re-computed, but it
+allows advanced dynamic expressions.
+ at end table
+
+Default value is @samp{init}.
+ at end table
+
+ at subsection Expressions
+
+The @option{alpha}, @option{x0} and @option{y0} expressions can contain the
+following parameters.
+
+ at table @option
+ at item w
+ at item h
+input width and height
+
+ at item n
+the number of input frame, starting from 0
+
+ at item pts
+the PTS (Presentation TimeStamp) time of the filtered video frame, expressed in
+ at var{TB} units, NAN if undefined
+
+ at item r
+frame rate of the input video, NAN if the input frame rate is unknown
+
+ at item t
+the PTS (Presentation TimeStamp) of the filtered video frame,
+expressed in seconds, NAN if undefined
+
+ at item tb
+time base of the input video
+ at end table
+
+
+ at subsection Examples
+
+ at itemize
+ at item
+Apply simple strong vignetting effect:
+ at example
+vignette=PI/4
+ at end example
+
+ at item
+Make a flickering vignetting:
+ at example
+vignette='PI/4+random(1)*PI/50':eval=frame
+ at end example
+
+ at end itemize
+
 @anchor{yadif}
 @section yadif
 
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 4ee28bf..e1affbf 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -191,6 +191,7 @@ OBJS-$(CONFIG_UNSHARP_FILTER)                += vf_unsharp.o
 OBJS-$(CONFIG_VFLIP_FILTER)                  += vf_vflip.o
 OBJS-$(CONFIG_VIDSTABDETECT_FILTER)          += vidstabutils.o vf_vidstabdetect.o
 OBJS-$(CONFIG_VIDSTABTRANSFORM_FILTER)       += vidstabutils.o vf_vidstabtransform.o
+OBJS-$(CONFIG_VIGNETTE_FILTER)               += vf_vignette.o
 OBJS-$(CONFIG_YADIF_FILTER)                  += vf_yadif.o
 OBJS-$(CONFIG_ZMQ_FILTER)                    += f_zmq.o
 
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index 58bffc7..6130d09 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -188,6 +188,7 @@ void avfilter_register_all(void)
     REGISTER_FILTER(VFLIP,          vflip,          vf);
     REGISTER_FILTER(VIDSTABDETECT,  vidstabdetect,  vf);
     REGISTER_FILTER(VIDSTABTRANSFORM, vidstabtransform, vf);
+    REGISTER_FILTER(VIGNETTE,       vignette,       vf);
     REGISTER_FILTER(YADIF,          yadif,          vf);
     REGISTER_FILTER(ZMQ,            zmq,            vf);
 
diff --git a/libavfilter/vf_vignette.c b/libavfilter/vf_vignette.c
new file mode 100644
index 0000000..eb72fae
--- /dev/null
+++ b/libavfilter/vf_vignette.c
@@ -0,0 +1,330 @@
+/*
+ * Copyright (c) 2013 Clément Bœsch
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "libavutil/opt.h"
+#include "libavutil/eval.h"
+#include "libavutil/avassert.h"
+#include "libavutil/pixdesc.h"
+#include "avfilter.h"
+#include "formats.h"
+#include "internal.h"
+#include "video.h"
+
+static const char *const var_names[] = {
+    "w",    // stream width
+    "h",    // stream height
+    "n",    // frame count
+    "pts",  // presentation timestamp expressed in AV_TIME_BASE units
+    "r",    // frame rate
+    "t",    // timestamp expressed in seconds
+    "tb",   // timebase
+    NULL
+};
+
+enum var_name {
+    VAR_W,
+    VAR_H,
+    VAR_N,
+    VAR_PTS,
+    VAR_R,
+    VAR_T,
+    VAR_TB,
+    VAR_NB
+};
+
+typedef struct {
+    const AVClass *class;
+    const AVPixFmtDescriptor *desc;
+    int backward;
+    enum EvalMode { EVAL_MODE_INIT, EVAL_MODE_FRAME, EVAL_MODE_NB } eval_mode;
+#define DEF_EXPR_FIELDS(name) AVExpr *name##_pexpr; char *name##_expr; double name;
+    DEF_EXPR_FIELDS(angle);
+    DEF_EXPR_FIELDS(x0);
+    DEF_EXPR_FIELDS(y0);
+    double var_values[VAR_NB];
+    float *fmap;
+    int fmap_linesize;
+    double dmax;
+    float xscale, yscale;
+    uint32_t dither;
+    int do_dither;
+} VignetteContext;
+
+#define OFFSET(x) offsetof(VignetteContext, x)
+#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
+static const AVOption vignette_options[] = {
+    { "angle", "set lens angle", OFFSET(angle_expr), AV_OPT_TYPE_STRING, {.str="PI/5"}, .flags = FLAGS },
+    { "a",     "set lens angle", OFFSET(angle_expr), AV_OPT_TYPE_STRING, {.str="PI/5"}, .flags = FLAGS },
+    { "x0", "set circle center position on x-axis", OFFSET(x0_expr), AV_OPT_TYPE_STRING, {.str="w/2"}, .flags = FLAGS },
+    { "y0", "set circle center position on y-axis", OFFSET(y0_expr), AV_OPT_TYPE_STRING, {.str="h/2"}, .flags = FLAGS },
+    { "mode", "set forward/backward mode", OFFSET(backward), AV_OPT_TYPE_INT, {.i64 = 0}, 0, 1, FLAGS, "mode" },
+        { "forward",  NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 0}, INT_MIN, INT_MAX, FLAGS, "mode"},
+        { "backward", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = 1}, INT_MIN, INT_MAX, FLAGS, "mode"},
+    { "eval", "specify when to evaluate expressions", OFFSET(eval_mode), AV_OPT_TYPE_INT, {.i64 = EVAL_MODE_INIT}, 0, EVAL_MODE_NB-1, FLAGS, "eval" },
+         { "init",  "eval expressions once during initialization", 0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_INIT},  .flags = FLAGS, .unit = "eval" },
+         { "frame", "eval expressions for each frame",             0, AV_OPT_TYPE_CONST, {.i64=EVAL_MODE_FRAME}, .flags = FLAGS, .unit = "eval" },
+    { "dither", "set dithering", OFFSET(do_dither), AV_OPT_TYPE_INT, {.i64 = 1}, 0, 1, FLAGS },
+    { NULL }
+};
+
+AVFILTER_DEFINE_CLASS(vignette);
+
+static av_cold int init(AVFilterContext *ctx)
+{
+    VignetteContext *s = ctx->priv;
+
+#define PARSE_EXPR(name) do {                                               \
+    int ret = av_expr_parse(&s->name##_pexpr,  s->name##_expr, var_names,   \
+                            NULL, NULL, NULL, NULL, 0, ctx);                \
+    if (ret < 0) {                                                          \
+        av_log(ctx, AV_LOG_ERROR, "Unable to parse expression for '"        \
+               AV_STRINGIFY(name) "'\n");                                   \
+        return ret;                                                         \
+    }                                                                       \
+} while (0)
+
+    PARSE_EXPR(angle);
+    PARSE_EXPR(x0);
+    PARSE_EXPR(y0);
+    return 0;
+}
+
+static av_cold void uninit(AVFilterContext *ctx)
+{
+    VignetteContext *s = ctx->priv;
+    av_freep(&s->fmap);
+    av_expr_free(s->angle_pexpr);
+    av_expr_free(s->x0_pexpr);
+    av_expr_free(s->y0_pexpr);
+}
+
+static int query_formats(AVFilterContext *ctx)
+{
+    static const enum AVPixelFormat pix_fmts[] = {
+        AV_PIX_FMT_YUV444P, AV_PIX_FMT_YUV422P,
+        AV_PIX_FMT_YUV420P, AV_PIX_FMT_YUV411P,
+        AV_PIX_FMT_YUV410P, AV_PIX_FMT_YUV440P,
+        AV_PIX_FMT_RGB24,   AV_PIX_FMT_BGR24,
+        AV_PIX_FMT_GRAY8,
+        AV_PIX_FMT_NONE
+    };
+    ff_set_common_formats(ctx, ff_make_format_list(pix_fmts));
+    return 0;
+}
+
+static double get_natural_factor(const VignetteContext *s, int x, int y)
+{
+    const int xx = (x - s->x0) * s->xscale;
+    const int yy = (y - s->y0) * s->yscale;
+    const double dnorm = hypot(xx, yy) / s->dmax;
+    if (dnorm > 1) {
+        return 0;
+    } else {
+        const double c = cos(s->angle * dnorm);
+        return (c*c)*(c*c); // do not remove braces, it helps compilers
+    }
+}
+
+#define TS2D(ts)     ((ts) == AV_NOPTS_VALUE ? NAN : (double)(ts))
+#define TS2T(ts, tb) ((ts) == AV_NOPTS_VALUE ? NAN : (double)(ts) * av_q2d(tb))
+
+static void update_context(VignetteContext *s, AVFilterLink *inlink, AVFrame *frame)
+{
+    int x, y;
+    float *dst = s->fmap;
+    int dst_linesize = s->fmap_linesize;
+
+    if (frame) {
+        s->var_values[VAR_N]   = inlink->frame_count;
+        s->var_values[VAR_T]   = TS2T(frame->pts, inlink->time_base);
+        s->var_values[VAR_PTS] = TS2D(frame->pts);
+    } else {
+        s->var_values[VAR_N]   = 0;
+        s->var_values[VAR_T]   = NAN;
+        s->var_values[VAR_PTS] = NAN;
+    }
+
+    s->angle = av_clipf(av_expr_eval(s->angle_pexpr, s->var_values, NULL), 0, M_PI_2);
+    s->x0 = av_expr_eval(s->x0_pexpr, s->var_values, NULL);
+    s->y0 = av_expr_eval(s->y0_pexpr, s->var_values, NULL);
+
+    if (s->backward) {
+        for (y = 0; y < inlink->h; y++) {
+            for (x = 0; x < inlink->w; x++)
+                dst[x] = 1. / get_natural_factor(s, x, y);
+            dst += dst_linesize;
+        }
+    } else {
+        for (y = 0; y < inlink->h; y++) {
+            for (x = 0; x < inlink->w; x++)
+                dst[x] = get_natural_factor(s, x, y);
+            dst += dst_linesize;
+        }
+    }
+}
+
+static inline double get_dither_value(VignetteContext *s)
+{
+    double dv = 0;
+    if (s->do_dither) {
+        dv = s->dither / (double)(1LL<<32);
+        s->dither = s->dither * 1664525 + 1013904223;
+    }
+    return dv;
+}
+
+static int filter_frame(AVFilterLink *inlink, AVFrame *in)
+{
+    unsigned x, y;
+    AVFilterContext *ctx = inlink->dst;
+    VignetteContext *s = ctx->priv;
+    AVFilterLink *outlink = inlink->dst->outputs[0];
+    AVFrame *out;
+
+    out = ff_get_video_buffer(outlink, outlink->w, outlink->h);
+    if (!out) {
+        av_frame_free(&in);
+        return AVERROR(ENOMEM);
+    }
+    av_frame_copy_props(out, in);
+
+    if (s->eval_mode == EVAL_MODE_FRAME)
+        update_context(s, inlink, in);
+
+    if (s->desc->flags & PIX_FMT_RGB) {
+        uint8_t       *dst = out->data[0];
+        const uint8_t *src = in ->data[0];
+        const float *fmap = s->fmap;
+        const int dst_linesize = out->linesize[0];
+        const int src_linesize = in ->linesize[0];
+        const int fmap_linesize = s->fmap_linesize;
+
+        for (y = 0; y < inlink->h; y++) {
+            uint8_t       *dstp = dst;
+            const uint8_t *srcp = src;
+
+            for (x = 0; x < inlink->w; x++, dstp += 3, srcp += 3) {
+                const double f = fmap[x];
+
+                dstp[0] = av_clip_uint8(srcp[0] * f + get_dither_value(s));
+                dstp[1] = av_clip_uint8(srcp[1] * f + get_dither_value(s));
+                dstp[2] = av_clip_uint8(srcp[2] * f + get_dither_value(s));
+            }
+            dst += dst_linesize;
+            src += src_linesize;
+            fmap += fmap_linesize;
+        }
+    } else {
+        int plane;
+
+        for (plane = 0; plane < 4 && in->data[plane]; plane++) {
+            uint8_t       *dst = out->data[plane];
+            const uint8_t *src = in ->data[plane];
+            const float *fmap = s->fmap;
+            const int dst_linesize = out->linesize[plane];
+            const int src_linesize = in ->linesize[plane];
+            const int fmap_linesize = s->fmap_linesize;
+            const int chroma = plane == 1 || plane == 2;
+            const int hsub = chroma ? s->desc->log2_chroma_w : 0;
+            const int vsub = chroma ? s->desc->log2_chroma_h : 0;
+            const int w = FF_CEIL_RSHIFT(inlink->w, hsub);
+            const int h = FF_CEIL_RSHIFT(inlink->h, vsub);
+
+            for (y = 0; y < h; y++) {
+                uint8_t *dstp = dst;
+                const uint8_t *srcp = src;
+
+                for (x = 0; x < w; x++) {
+                    const double dv = get_dither_value(s);
+                    if (chroma) *dstp++ = av_clip_uint8(fmap[x << hsub] * (*srcp++ - 127) + 127 + dv);
+                    else        *dstp++ = av_clip_uint8(fmap[x        ] *  *srcp++              + dv);
+                }
+                dst += dst_linesize;
+                src += src_linesize;
+                fmap += fmap_linesize << vsub;
+            }
+        }
+    }
+
+    return ff_filter_frame(outlink, out);
+}
+
+static int config_props(AVFilterLink *inlink)
+{
+    VignetteContext *s = inlink->dst->priv;
+
+    s->desc = av_pix_fmt_desc_get(inlink->format);
+    s->var_values[VAR_W]  = inlink->w;
+    s->var_values[VAR_H]  = inlink->h;
+    s->var_values[VAR_TB] = av_q2d(inlink->time_base);
+    s->var_values[VAR_R]  = inlink->frame_rate.num == 0 || inlink->frame_rate.den == 0 ?
+        NAN : av_q2d(inlink->frame_rate);
+
+    if (inlink->sample_aspect_ratio.num > inlink->sample_aspect_ratio.den) {
+        s->xscale = av_q2d(inlink->sample_aspect_ratio);
+        s->yscale = 1;
+        s->dmax = hypot(inlink->w / 2., s->yscale * inlink->h / 2.);
+    } else {
+        s->yscale = av_q2d(inlink->sample_aspect_ratio);
+        s->xscale = 1;
+        s->dmax = hypot(s->xscale * inlink->w / 2., inlink->h / 2.);
+    }
+
+    s->fmap_linesize = FFALIGN(inlink->w, 32);
+    s->fmap = av_malloc(s->fmap_linesize * inlink->h * sizeof(*s->fmap));
+    if (!s->fmap)
+        return AVERROR(ENOMEM);
+
+    if (s->eval_mode == EVAL_MODE_INIT)
+        update_context(s, inlink, NULL);
+
+    return 0;
+}
+
+static const AVFilterPad vignette_inputs[] = {
+    {
+        .name         = "default",
+        .type         = AVMEDIA_TYPE_VIDEO,
+        .filter_frame = filter_frame,
+        .config_props = config_props,
+    },
+    { NULL }
+};
+
+static const AVFilterPad vignette_outputs[] = {
+     {
+         .name = "default",
+         .type = AVMEDIA_TYPE_VIDEO,
+     },
+     { NULL }
+};
+
+AVFilter avfilter_vf_vignette = {
+    .name          = "vignette",
+    .description   = NULL_IF_CONFIG_SMALL("Make or reverse a vignette effect."),
+    .priv_size     = sizeof(VignetteContext),
+    .init          = init,
+    .uninit        = uninit,
+    .query_formats = query_formats,
+    .inputs        = vignette_inputs,
+    .outputs       = vignette_outputs,
+    .priv_class    = &vignette_class,
+    .flags         = AVFILTER_FLAG_SUPPORT_TIMELINE_GENERIC,
+};
-- 
1.8.3

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20130529/b23996f5/attachment.asc>


More information about the ffmpeg-devel mailing list