[FFmpeg-devel] [Patch] [avfilter] refactor floating point based interpolation and introduce in vf_lenscorrection
Michael Niedermayer
michaelni at gmx.at
Wed Aug 20 03:16:49 CEST 2014
On Wed, Aug 20, 2014 at 12:12:49AM +0200, Daniel Oberhoff wrote:
> Hello,
>
> As a follow-up to my last patch I now factored out the floating point based interpolation from transform.h/transform.c and applied it in the vf_lenscorrection filter I introduced in my last patch. What I did not do is also factor out fixed point based interpolation as used in vf_rotate and vf_perspective and maybe others as could probably also be done. Also I did not look further for uses of floating point based interpolation. Basically I just wanted to introduce interpolation in vf_lenscorrection without code duplication. As a side note I also tried to introduce fixed point calculations to vf_lenscorrection but found myself effectively doing floating point “by hand” since due to the high order of the algorithm (up to 4th order) it is very hard to keep track of the right amount of pre/post-comma digits for a given step in the algorithm and given parameters and it felt rather futile after a while.
>
> Looking forward to reviews :).
why did you use the deshake code and not the vf_perspective code ?!
i suspect this will be quite a bit harder to get into shape for a
common API
>
> From 4b271f72946aeebf5603cc8779f6b61ff0c1bd49 Mon Sep 17 00:00:00 2001
> From: James Almer <jamrial at gmail.com>
> Date: Sun, 10 Aug 2014 02:24:01 -0300
> Subject: [PATCH] afvilter: re-factor/re-use floating point based interpolation
> from vf_perspective
>
> ---
> doc/filters.texi | 11 +++
> libavfilter/interpolate.h | 144 ++++++++++++++++++++++++++++++++++++++++
> libavfilter/transform.c | 91 ++-----------------------
> libavfilter/transform.h | 14 +---
> libavfilter/vf_lenscorrection.c | 21 ++++--
> 5 files changed, 176 insertions(+), 105 deletions(-)
> create mode 100644 libavfilter/interpolate.h
>
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 0ca1d6f..2edefc4 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -5582,6 +5582,17 @@ height.
> Coefficient of the quadratic correction term. 0.5 means no correction.
> @item k2
> Coefficient of the double quadratic correction term. 0.5 means no correction.
> + at item interpolation
> +Set the interpolation method for the transformation
> +
> +It accepts the following values:
> + at table @samp
> + at item nearest
> + at item linear
> + at item cubic
> + at end table
> +Default value is @samp{linear}.
> +
> @end table
>
> The formula that generates the correction is:
> diff --git a/libavfilter/interpolate.h b/libavfilter/interpolate.h
> new file mode 100644
> index 0000000..6f7a849
> --- /dev/null
> +++ b/libavfilter/interpolate.h
> @@ -0,0 +1,144 @@
> +/*
> + * Copyright (C) 2010 Georg Martius <georg.martius at web.de>
> + * Copyright (C) 2010 Daniel G. Taylor <dan at programmer-art.org>
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#ifndef AVFILTER_INTERPOLATE_H
> +#define AVFILTER_INTERPOLATE_H
> +
> +enum InterpolateMethod {
> + INTERPOLATE_NEAREST, //< Nearest-neighbor (fast)
> + INTERPOLATE_BILINEAR, //< Bilinear
> + INTERPOLATE_BIQUADRATIC, //< Biquadratic (best)
> + INTERPOLATE_COUNT, //< Number of interpolation methods
> +};
> +
> +// Shortcuts for the fastest and best interpolation methods
> +#define INTERPOLATE_DEFAULT INTERPOLATE_BILINEAR
> +#define INTERPOLATE_FAST INTERPOLATE_NEAREST
> +#define INTERPOLATE_BEST INTERPOLATE_BIQUADRATIC
> +
> +#define INTERPOLATE_METHOD(name) \
> + static av_always_inline uint8_t name(float x, float y, const uint8_t *src, \
> + int width, int height, int stride, uint8_t def)
> +
> +/**
> + * Nearest neighbor interpolation
> + */
> +INTERPOLATE_METHOD(interpolate_nearest)
> +{
> + if (x < 0 || x >= width || y < 0 || y >= height) {
> + return def;
> + } else {
> + return src[(int)(x + 0.5f) + stride * (int)(y + 0.5f)];
> + }
> +}
i dont think using float in a nearest neighbor resampler is acceptable
> +
> +/**
> + * Bilinear interpolation
> + */
> +INTERPOLATE_METHOD(interpolate_bilinear)
> +{
> + int x_c, x_f, y_c, y_f;
> + int v1, v2, v3, v4;
> + const uint8_t *line_y_f, *line_y_c;
> +
> + if (x < 0 || x >= width || y < 0 || y >= height) {
> + return def;
> + } else {
> + x_f = (int)x;
> + x_c = x_f + 1;
> +
> + y_f = (int)y;
> + y_c = y_f + 1;
> +
> + line_y_f = src + stride * y_c;
> + line_y_c = line_y_f + stride;
> +
> + v1 = line_y_c[x_c];
> + v2 = line_y_f[x_c];
> + v3 = line_y_c[x_f];
> + v4 = line_y_f[x_f];
> +
> + return (v1*(x - x_f)*(y - y_f) + v2*((x - x_f)*(y_c - y)) +
> + v3*(x_c - x)*(y - y_f) + v4*((x_c - x)*(y_c - y)));
> + }
> +}
> +
> +/**
> + * Biquadratic interpolation
> + */
> +INTERPOLATE_METHOD(interpolate_biquadratic)
> +{
> + int x_c, x_f, y_c, y_f;
> + uint8_t v1, v2, v3, v4;
> + float f1, f2, f3, f4;
> + const uint8_t *line_y_f, *line_y_c;
> +
> + if (x < 0 || x >= width || y < 0 || y >= height) {
> + return def;
> + } else {
> + x_f = (int)x;
> + x_c = x_f + 1;
> + y_f = (int)y;
> + y_c = y_f + 1;
> +
> + line_y_f = src + stride * y_c;
> + line_y_c = line_y_f + stride;
> +
> + v1 = line_y_c[x_c];
> + v2 = line_y_f[x_c];
> + v3 = line_y_c[x_f];
> + v4 = line_y_f[x_f];
> +
> + f1 = 1 - sqrt((x_c - x) * (y_c - y));
> + f2 = 1 - sqrt((x_c - x) * (y - y_f));
> + f3 = 1 - sqrt((x - x_f) * (y_c - y));
> + f4 = 1 - sqrt((x - x_f) * (y - y_f));
> + return (v1 * f1 + v2 * f2 + v3 * f3 + v4 * f4) / (f1 + f2 + f3 + f4);
> + }
> +}
There should be bilinear and bicubic filters.
If you want to add windowed sinc filters or spline based ones i dont
mind.
But ive no faith that this odd filter above which has the order of a
bilinear one will look all that great. And it sure wont be fast with
sqrt() per sample
also
one way to do higher order filters is to create a LUT with the filter
Coefficients, for example if you want 8bit sub pixel precission and
a bicubic filter that makes just 256 * 4 entries, 4 Coefficients for
each 256 subpixel positions.
That LUT can be filled by using the float code from vf_perspective
or if done without LUT the code from vf_perspective can also be used
directly and doesnt need any functions like sqrt() and at the
same time should be higher quality
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Complexity theory is the science of finding the exact solution to an
approximation. Benchmarking OTOH is finding an approximation of the exact
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20140820/ee8a28a5/attachment.asc>
More information about the ffmpeg-devel
mailing list