[FFmpeg-devel] [PATCH v4] avfilter: add libdewobble filter
Daniel Playfair Cal
daniel.playfair.cal at gmail.com
Thu Aug 26 16:58:39 EEST 2021
All of the existing filters for video stabilization use an affine model
(or a limited version of it) to represent the movement of the camera. When
used with cameras with a very wide field of view and/or where the camera
shaking is severe, the corrections result in significant geometric
distortion ("wobbling").
Dewobble (https://git.sr.ht/~hedgepigdaniel/dewobble) is a library built
to solve this problem. It requires knowledge of the projection used by
the input camera, and it performs stabilization using a homography
model, which is limited to include only changes in camera orientation.
Additionally, it can perform projection change by specifying a different
projection for the output camera. This is more efficient and results in
less loss of information than using separate filters to perform
stabilization and projection change.
The libdewobble filter is a wrapper for Dewobble. Dewobble supports
input and output in OpenCL buffers containing NV12 frames. Hence, the
filter has the same limitations. Currently
all of the options of Dewobble are supported. Of the two types of filter
available in Dewobble (FilterSync and FilterThreaded), FilterThreaded is
used. The API is synchronous, but the transformations are done in a
separate thread. The purpose of this is to isolate the global per thread
OpenCL context used by OpenCV, which Dewobble uses internally. This
prevents dewobble_opencl from interfering with any other usage of OpenCV
from within FFmpeg.
Signed-off-by: Daniel Playfair Cal <daniel.playfair.cal at gmail.com>
---
Changelog v4:
- correct rename in documentation
- update examples to include hwupload/hwdownload, now that name does
not contain _opencl postfix.
---
Changelog | 1 +
LICENSE.md | 2 +-
configure | 4 +
doc/filters.texi | 149 ++++
libavfilter/Makefile | 1 +
libavfilter/allfilters.c | 1 +
libavfilter/vf_libdewobble.c | 1273 ++++++++++++++++++++++++++++++++++
7 files changed, 1430 insertions(+), 1 deletion(-)
create mode 100644 libavfilter/vf_libdewobble.c
diff --git a/Changelog b/Changelog
index 20d71bed88..b4e0a4f4f9 100644
--- a/Changelog
+++ b/Changelog
@@ -12,6 +12,7 @@ version <next>:
- audio and video segment filters
- Apple Graphics (SMC) encoder
- hsvkey and hsvhold video filters
+- libdewobble video filter
version 4.4:
diff --git a/LICENSE.md b/LICENSE.md
index 613070e1b6..dfdf010d8e 100644
--- a/LICENSE.md
+++ b/LICENSE.md
@@ -112,7 +112,7 @@ The VMAF, mbedTLS, RK MPI, OpenCORE and VisualOn libraries are under the Apache
version 3 of those licenses. So to combine these libraries with FFmpeg, the
license version needs to be upgraded by passing `--enable-version3` to configure.
-The smbclient library is under the GPL v3, to combine it with FFmpeg,
+The dewobble and smbclient libraries are under the GPL v3, to combine them with FFmpeg,
the options `--enable-gpl` and `--enable-version3` have to be passed to
configure to upgrade FFmpeg to the GPL v3.
diff --git a/configure b/configure
index 9249254b70..00f3e8deea 100755
--- a/configure
+++ b/configure
@@ -230,6 +230,7 @@ External library support:
--enable-libdavs2 enable AVS2 decoding via libdavs2 [no]
--enable-libdc1394 enable IIDC-1394 grabbing using libdc1394
and libraw1394 [no]
+ --enable-libdewobble enable video stabilization via libdewobble [no]
--enable-libfdk-aac enable AAC de/encoding via libfdk-aac [no]
--enable-libflite enable flite (voice synthesis) support via libflite [no]
--enable-libfontconfig enable libfontconfig, useful for drawtext filter [no]
@@ -1781,6 +1782,7 @@ EXTERNAL_LIBRARY_VERSION3_LIST="
"
EXTERNAL_LIBRARY_GPLV3_LIST="
+ libdewobble
libsmbclient
"
@@ -3606,6 +3608,7 @@ interlace_filter_deps="gpl"
kerndeint_filter_deps="gpl"
ladspa_filter_deps="ladspa libdl"
lensfun_filter_deps="liblensfun version3"
+libdewobble_filter_deps="libdewobble opencl"
lv2_filter_deps="lv2"
mcdeint_filter_deps="avcodec gpl"
metadata_filter_deps="avformat"
@@ -6406,6 +6409,7 @@ enabled libcodec2 && require libcodec2 codec2/codec2.h codec2_create -lc
enabled libdav1d && require_pkg_config libdav1d "dav1d >= 0.5.0" "dav1d/dav1d.h" dav1d_version
enabled libdavs2 && require_pkg_config libdavs2 "davs2 >= 1.6.0" davs2.h davs2_decoder_open
enabled libdc1394 && require_pkg_config libdc1394 libdc1394-2 dc1394/dc1394.h dc1394_new
+enabled libdewobble && require_pkg_config libdewobble dewobble dewobble/filter.h dewobble_filter_create_threaded
enabled libdrm && require_pkg_config libdrm libdrm xf86drm.h drmGetVersion
enabled libfdk_aac && { check_pkg_config libfdk_aac fdk-aac "fdk-aac/aacenc_lib.h" aacEncOpen ||
{ require libfdk_aac fdk-aac/aacenc_lib.h aacEncOpen -lfdk-aac &&
diff --git a/doc/filters.texi b/doc/filters.texi
index 47ceeb18c6..1a41f0dd0a 100644
--- a/doc/filters.texi
+++ b/doc/filters.texi
@@ -14213,6 +14213,155 @@ ffmpeg -i input.mov -vf lensfun=make=Canon:model="Canon EOS 100D":lens_model="Ca
@end itemize
+ at section libdewobble
+
+Apply motion stabilization with awareness of lens projection and/or lens projection change using libdewobble (@url{https://git.sr.ht/~hedgepigdaniel/dewobble}).
+
+To enable compilation of this filter you need to configure FFmpeg with
+ at code{--enable-libdewobble}.
+
+This filter accepts the following options:
+
+ at table @option
+ at item in_p
+ at item out_p
+Set the lens projection model for the input and output.
+
+Available values are:
+ at table @samp
+ at item rect
+Rectilinear projection.
+
+ at item fish
+Equidistant fisheye projection.
+
+ at end table
+
+ at item in_dfov
+ at item out_dfov
+Diagonal field of view in degrees for the input and output.
+
+ at item in_fx
+ at item in_fy
+ at item out_fx
+ at item out_fy
+Location of the focal point in the input and output image.
+Default value is the image centre in both cases.
+
+ at item out_w
+ at item out_h
+Dimensions of the output image.
+Default value is the same as in input image.
+
+ at item stab
+Motion stabilization algorithm.
+
+Available values are:
+ at table @samp
+ at item fixed
+Fix the camera orientation after the first frame.
+
+ at item none
+No not apply stabilization.
+
+ at item sg
+Smooth the camera motion using a Savitzky-Golay filter.
+
+ at end table
+
+Default value is @samp{sg}.
+
+ at item stab_r
+For Savitzky-Golay smoothing: the number of frames to look ahead and behind.
+Higher values result in a smoother output camera path.
+
+Default value is 15.
+
+Higher values increase (OpenCL) memory usage.
+
+ at item stab_h
+For stabilization: the number of frames to look ahead to interpolate input camera rotation in frames where it cannot be detected.
+
+Default value is 30.
+
+Higher values increase (OpenCL) memory usage.
+
+ at item interp
+Pixel interpolation algorithm.
+
+Available values are:
+ at table @samp
+ at item nearest
+Nearest neighbour interpolation (fast OpenCL implementation).
+
+ at item linear
+Bilinear interpolation (fast OpenCL implementation).
+
+ at item cubic
+Bicubic interpolation (CPU implementation).
+
+ at item lanczos
+Lanczos4 interpolation in an 8x8 neighbourhood (CPU implementation).
+
+ at end table
+
+Default value is @samp{linear}.
+
+ at item border
+Border extrapolation algorithm (determines how to color pixels in the output that do not map to the input).
+
+Available values are:
+ at table @samp
+ at item constant
+Constant color.
+
+ at item reflect
+Reflection of the input horizontally or vertically about the edge.
+
+ at item reflect101
+Reflection of the input horizontally or vertically about the point half a pixel from the edge.
+
+ at item replicate
+Replicate the pixel on the edge in a vertical or horizontal direction.
+
+ at item wrap
+Wrap around to the opposite side of the source image.
+
+ at end table
+
+Default value is @samp{constant}.
+
+ at item border_r
+ at item border_g
+ at item border_b
+For @samp{constant} border, the color to fill with (red, green, blue components).
+
+Default value is black.
+
+ at item debug
+Include a suite of debugging information in the output.
+
+Default value is disabled.
+
+ at end table
+
+ at subsection Examples
+
+ at itemize
+ at item
+Apply motion stabilization to video from a popular action cam in a certain capture mode:
+ at example
+ffmpeg -init_hw_device opencl=ocl:DEVICE -filter_hw_device ocl -i INPUT -vf 'format=nv12,hwupload,libdewobble=in_p=fish:in_dfov=145.8:out_p=fish:out_dfov=145.8:stab=sg,hwdownload,format=nv12' OUTPUT
+ at end example
+
+ at item
+Apply stabilization and lens projection change:
+ at example
+ffmpeg -init_hw_device opencl=ocl:DEVICE -filter_hw_device ocl -i INPUT -vf 'format=nv12,hwupload,libdewobble=in_p=fish:in_dfov=145.8:out_p=rect:out_dfov=145.8:stab=sg,hwdownload,format=nv12' OUTPUT
+ at end example
+
+ at end itemize
+
@section libvmaf
Obtain the VMAF (Video Multi-Method Assessment Fusion)
diff --git a/libavfilter/Makefile b/libavfilter/Makefile
index 3166e70b58..6e495c810e 100644
--- a/libavfilter/Makefile
+++ b/libavfilter/Makefile
@@ -315,6 +315,7 @@ OBJS-$(CONFIG_KIRSCH_FILTER) += vf_convolution.o
OBJS-$(CONFIG_LAGFUN_FILTER) += vf_lagfun.o
OBJS-$(CONFIG_LENSCORRECTION_FILTER) += vf_lenscorrection.o
OBJS-$(CONFIG_LENSFUN_FILTER) += vf_lensfun.o
+OBJS-$(CONFIG_LIBDEWOBBLE_FILTER) += vf_libdewobble.o opencl.o
OBJS-$(CONFIG_LIBVMAF_FILTER) += vf_libvmaf.o framesync.o
OBJS-$(CONFIG_LIMITER_FILTER) += vf_limiter.o
OBJS-$(CONFIG_LOOP_FILTER) += f_loop.o
diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
index c49c831203..9cdde96a0f 100644
--- a/libavfilter/allfilters.c
+++ b/libavfilter/allfilters.c
@@ -300,6 +300,7 @@ extern const AVFilter ff_vf_kirsch;
extern const AVFilter ff_vf_lagfun;
extern const AVFilter ff_vf_lenscorrection;
extern const AVFilter ff_vf_lensfun;
+extern const AVFilter ff_vf_libdewobble;
extern const AVFilter ff_vf_libvmaf;
extern const AVFilter ff_vf_limiter;
extern const AVFilter ff_vf_loop;
diff --git a/libavfilter/vf_libdewobble.c b/libavfilter/vf_libdewobble.c
new file mode 100644
index 0000000000..45d17dd745
--- /dev/null
+++ b/libavfilter/vf_libdewobble.c
@@ -0,0 +1,1273 @@
+/*
+ * Copyright (c) 2021 Daniel Playfair Cal <daniel.playfair.cal at gmail.com>
+ *
+ * This file is part of FFmpeg.
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <https://www.gnu.org/licenses/>.
+ */
+#include <dewobble/camera.h>
+#include <dewobble/filter.h>
+#include <dewobble/stabilizer.h>
+#include <float.h>
+#include <pthread.h>
+#include <signal.h>
+
+#include "libavutil/avassert.h"
+#include "libavutil/common.h"
+#include "libavutil/imgutils.h"
+#include "libavutil/mem.h"
+#include "libavutil/opt.h"
+#include "libavutil/pixdesc.h"
+#include "libavutil/thread.h"
+
+#include "avfilter.h"
+#include "filters.h"
+#include "internal.h"
+#include "opencl.h"
+#include "opencl_source.h"
+#include "transpose.h"
+#include "video.h"
+
+/**
+ * @file
+ * Apply motion stabilization with awareness of lens projection and/or change
+ * camera projection.
+ *
+ * This filter is essentially a wrapper around dewobble
+ * (https://git.sr.ht/~hedgepigdaniel/dewobble).
+ *
+ * @par Queued frames
+ *
+ * libdewobble requires a queue of frames before it can provide output because
+ * it looks ahead to calculate a smooth camera path and to interpolate camera
+ * positions from frames where it fails to detect motion. The number of queued
+ * frames required is determined by libdewobble.
+ *
+ * @par Hardware frame allocation
+ *
+ * Input OpenCL hardware frames contain `cl_image`s but these must be converted
+ * to `cl_buffer`s for libdewobble. Although the filter keeps a reference to
+ * the input frame until the output frame is sent, it unreferences the original
+ * hardware buffers immediately after copying them to a `cl_buffer` in
+ * `consume_input_frame`. This avoids OOM issues for example when using input
+ * frames mapped from VA-API hardware frames where there is a low limit for how
+ * many can be allocated at once. The filter only owns a single input/output
+ * hardware frame buffer at any time, although internally it allocates OpenCL
+ * buffers to store the contents of a queue of frames.
+ */
+
+/**
+ * Camera properties, mirroring those present in libdewobble's camera object.
+ */
+typedef struct Camera {
+ /**
+ * Camera projection model, e.g. `DEWOBBLE_PROJECTION_RECTILINEAR`
+ */
+ int model;
+
+ /**
+ * Camera diagonal field of view in degrees
+ */
+ double diagonal_fov;
+
+ /**
+ * Width in pixels
+ */
+ int width;
+
+ /**
+ * Height in pixels
+ */
+ int height;
+
+ /**
+ * Horizonal coordinate of focal point in pixels
+ */
+ double focal_point_x;
+
+ /**
+ * Vertical coordinate of focal point in pixels
+ */
+ double focal_point_y;
+} Camera;
+
+/**
+ * Motion stabilization algorithm, mirroring those available in libdewobble.
+ */
+typedef enum StabilizationAlgorithm {
+
+ /**
+ * Do not apply stabilization
+ */
+ STABILIZATION_ALGORITHM_ORIGINAL,
+
+ /**
+ * Keep the camera orientation fixed at its orientation in the first frame
+ */
+ STABILIZATION_ALGORITHM_FIXED,
+
+ /**
+ * Smooth camera orientation with a Savitsky-Golay filter
+ */
+ STABILIZATION_ALGORITHM_SMOOTH,
+
+ /**
+ * Number of stabilization algorithms
+ */
+ NB_STABILIZATION_ALGORITHMS,
+
+} StabilizationAlgorithm;
+
+/**
+ * libdewobble filter context
+ */
+typedef struct LibDewobbleOpenCLContext {
+
+ /**
+ * Generic OpenCL filter context
+ */
+ OpenCLFilterContext ocf;
+
+ /**
+ * OpenCL command queue
+ */
+ cl_command_queue command_queue;
+
+ /**
+ * Input camera (projection, focal length, etc)
+ */
+ Camera input_camera;
+
+ /**
+ * Output camera (projection, focal length, etc)
+ */
+ Camera output_camera;
+
+ /**
+ * Stabilization algorithm applied by the filter
+ * (@ref StabilizationAlgorithm)
+ */
+ int stabilization_algorithm;
+
+ /**
+ * The number of frames to look ahead and behind for the purpose of
+ * stabilizing each frame
+ */
+ int stabilization_radius;
+
+ /**
+ * The number of frames to look ahead for the purpose of interpolating
+ * frame rotation for frames where detection fails
+ */
+ int stabilization_horizon;
+
+ /**
+ * The algorithm to interpolate the value between source image pixels
+ * (e.g.\ `DEWOBBLE_INTERPOLATION_LINEAR`)
+ */
+ int interpolation_algorithm;
+
+ /**
+ * The algorithm used to fill in unmapped areas of the output (e.g.\
+ * `DEWOBBLE_BORDER_CONSTANT`)
+ */
+ int border_type;
+
+ /**
+ * The color used to fill unmapped areas of the output when
+ * @ref border_type is `DEWOBBLE_BORDER_CONSTANT`
+ */
+ double border_color[4];
+
+ /**
+ * Whether to include debugging information in the output
+ */
+ int debug;
+
+ /**
+ * Whether the filter has been initialized
+ */
+ int initialized;
+
+ /**
+ * The status of the input link
+ */
+ int input_status;
+
+ /**
+ * The time that the input status was reached
+ */
+ int64_t input_status_pts;
+
+ /**
+ * Number of frame jobs currently in progress (read from inlink but not
+ * yet sent to outlink)
+ */
+ int nb_frames_in_progress;
+
+ /**
+ * Number of frames consumed so far
+ */
+ long nb_frames_consumed;
+
+ /**
+ * The instance of libdewobble's filter
+ */
+ DewobbleFilter dewobble_filter;
+
+} LibDewobbleOpenCLContext;
+
+/**
+ * Convert degrees to radians.
+ * @param degrees the number of degrees
+ * @return the equivalent number of radians
+ */
+static double degrees_to_radians(double degrees)
+{
+ return degrees * M_PI / 180;
+}
+
+/**
+ * Initialize the libdewobble filter instance.
+ * @param avctx the filter context
+ * @return 0 on success, otherwise a negative error code
+ */
+static int init_libdewobble_filter(AVFilterContext *avctx)
+{
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+ DewobbleStabilizer stabilizer = NULL;
+ DewobbleCamera input_camera = NULL, output_camera = NULL;
+ DewobbleFilterConfig config = NULL;
+
+ input_camera = dewobble_camera_create(
+ ctx->input_camera.model, degrees_to_radians(ctx->input_camera.diagonal_fov),
+ ctx->input_camera.width, ctx->input_camera.height,
+ ctx->input_camera.focal_point_x, ctx->input_camera.focal_point_y);
+
+ if (input_camera == NULL)
+ goto fail;
+
+ output_camera = dewobble_camera_create(
+ ctx->output_camera.model,
+ degrees_to_radians(ctx->output_camera.diagonal_fov),
+ ctx->output_camera.width, ctx->output_camera.height,
+ ctx->output_camera.focal_point_x, ctx->output_camera.focal_point_y);
+
+ if (output_camera == NULL)
+ goto fail;
+
+ switch (ctx->stabilization_algorithm) {
+ case STABILIZATION_ALGORITHM_ORIGINAL:
+ stabilizer = dewobble_stabilizer_create_none();
+ break;
+ case STABILIZATION_ALGORITHM_FIXED:
+ stabilizer = dewobble_stabilizer_create_fixed(input_camera,
+ ctx->stabilization_horizon);
+
+ break;
+ case STABILIZATION_ALGORITHM_SMOOTH:
+ stabilizer = dewobble_stabilizer_create_savitzky_golay(
+ input_camera, ctx->stabilization_radius, ctx->stabilization_horizon);
+
+ break;
+ }
+
+ if (stabilizer == NULL)
+ goto fail;
+
+ config = dewobble_filter_config_create(input_camera, output_camera, stabilizer);
+
+ dewobble_filter_config_set_opencl_context(config, ctx->ocf.hwctx->context);
+ dewobble_filter_config_set_opencl_device(config, ctx->ocf.hwctx->device_id);
+ dewobble_filter_config_set_interpolation(config, ctx->interpolation_algorithm);
+ dewobble_filter_config_set_border_type(config, ctx->border_type);
+ dewobble_filter_config_set_border_color(config, ctx->border_color);
+ dewobble_filter_config_set_debug(config, ctx->debug);
+
+ ctx->dewobble_filter = dewobble_filter_create_threaded(config);
+
+ dewobble_filter_config_destroy(&config);
+
+ if (ctx->dewobble_filter == NULL)
+ goto fail;
+
+ dewobble_stabilizer_destroy(&stabilizer);
+
+ return 0;
+
+fail:
+ dewobble_stabilizer_destroy(&stabilizer);
+ dewobble_camera_destroy(&input_camera);
+ dewobble_camera_destroy(&output_camera);
+
+ return AVERROR(ENOMEM);
+}
+
+/**
+ * Initialize the filter based on the options
+ * @param avctx the filter context
+ * @return 0 on success, otherwise a negative error code
+ */
+static int libdewobble_init(AVFilterContext *avctx)
+{
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+
+ av_log(avctx, AV_LOG_VERBOSE, "Init\n");
+
+ if (ctx->input_camera.model == DEWOBBLE_NB_PROJECTIONS
+ || ctx->output_camera.model == DEWOBBLE_NB_PROJECTIONS) {
+
+ av_log(avctx, AV_LOG_ERROR, "both in_p and out_p must be set\n");
+ return AVERROR(EINVAL);
+ }
+
+ if (ctx->input_camera.diagonal_fov == 0 || ctx->output_camera.diagonal_fov == 0) {
+ av_log(avctx, AV_LOG_ERROR, "both in_dfov and out_dfov must be set\n");
+ return AVERROR(EINVAL);
+ }
+
+ if (ctx->stabilization_algorithm == STABILIZATION_ALGORITHM_ORIGINAL)
+ ctx->stabilization_horizon = 0;
+
+ return ff_opencl_filter_init(avctx);
+}
+
+/**
+ * Clean up the filter on destruction.
+ * @param avctx the filter context
+ */
+static void libdewobble_uninit(AVFilterContext *avctx)
+{
+ cl_int cle;
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+
+ av_log(avctx, AV_LOG_VERBOSE, "Uninit\n");
+
+ if (ctx->command_queue) {
+ cle = clReleaseCommandQueue(ctx->command_queue);
+
+ if (cle != CL_SUCCESS)
+ av_log(avctx, AV_LOG_ERROR,
+ "Failed to release command queue: %d.\n", cle);
+ }
+
+ dewobble_filter_destroy(&ctx->dewobble_filter);
+ ff_opencl_filter_uninit(avctx);
+}
+
+/**
+ * Perform further initialization of the filter when the first input frame is
+ * available.
+ * @param avctx the filter context
+ * @param first_frame the first input frame
+ * @return 0 on success, otherwise a negative error code
+ */
+static int libdewobble_frames_init(AVFilterContext *avctx, AVFrame *first_frame)
+{
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+ AVFilterLink *inlink = avctx->inputs[0];
+ cl_int cle;
+ int err;
+
+ if (first_frame->crop_top % 2 == 1 || first_frame->crop_bottom % 2 == 1
+ || first_frame->crop_left % 2 == 1 || first_frame->crop_right % 2 == 1) {
+
+ av_log(avctx, AV_LOG_ERROR,
+ "Cropping by an odd number of pixels is not supported!\n");
+ return AVERROR(EINVAL);
+ }
+
+ if ((first_frame->crop_top || first_frame->crop_bottom)
+ && (ctx->output_camera.height == 0
+ || ctx->output_camera.focal_point_y == DBL_MAX))
+ av_log(avctx, AV_LOG_WARNING,
+ "Input is vertically cropped, but output height or vertical "
+ "focal point is not set. The default values are based on the "
+ "uncropped input!\n");
+
+ if ((first_frame->crop_left || first_frame->crop_right)
+ && (ctx->output_camera.width == 0
+ || ctx->output_camera.focal_point_x == DBL_MAX))
+ av_log(avctx, AV_LOG_WARNING,
+ "Input is horizontally cropped, but output width or horizontal "
+ "focal point is not set. The default values are based on the "
+ "uncropped input!\n");
+
+ ctx->input_camera.width
+ = inlink->w - first_frame->crop_left - first_frame->crop_right;
+ ctx->input_camera.height
+ = inlink->h - first_frame->crop_top - first_frame->crop_bottom;
+
+ /* Output camera width must match the filter output */
+ ctx->output_camera.width = ctx->ocf.output_width;
+ ctx->output_camera.height = ctx->ocf.output_height;
+
+ /* Focal points default to the image center (disregarding cropping) */
+ if (ctx->input_camera.focal_point_x == DBL_MAX)
+ ctx->input_camera.focal_point_x
+ = (inlink->w - 1) / 2.0 - first_frame->crop_left;
+
+ if (ctx->input_camera.focal_point_y == DBL_MAX)
+ ctx->input_camera.focal_point_y
+ = (inlink->h - 1) / 2.0 - first_frame->crop_top;
+
+ if (ctx->output_camera.focal_point_x == DBL_MAX)
+ ctx->output_camera.focal_point_x = (ctx->output_camera.width - 1) / 2.0;
+
+ if (ctx->output_camera.focal_point_y == DBL_MAX)
+ ctx->output_camera.focal_point_y = (ctx->output_camera.height - 1) / 2.0;
+
+ ctx->command_queue = clCreateCommandQueue(ctx->ocf.hwctx->context,
+ ctx->ocf.hwctx->device_id, 0, &cle);
+
+ if (cle) {
+ av_log(avctx, AV_LOG_ERROR,
+ "Failed to create OpenCL command queue %d.\n", cle);
+ return AVERROR(EIO);
+ }
+
+ err = init_libdewobble_filter(avctx);
+ if (err) {
+ av_log(avctx, AV_LOG_ERROR,
+ "Failed to initialise libdewobble filter %d.\n", err);
+ return AVERROR(EIO);
+ }
+
+ ctx->initialized = 1;
+
+ return 0;
+}
+
+/**
+ * Perform initialization based on the input filter link.
+ * @param inlink the input filter link
+ * @return 0 on success, otherwise a negative error code
+ */
+static int libdewobble_config_input(AVFilterLink *inlink)
+{
+ AVFilterContext *avctx = inlink->dst;
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+ int ret;
+
+ ret = ff_opencl_filter_config_input(inlink);
+
+ if (ret < 0)
+ return ret;
+
+ if (ctx->ocf.output_format != AV_PIX_FMT_NV12) {
+ av_log(avctx, AV_LOG_ERROR, "Only NV12 input is supported!\n");
+ return AVERROR(ENOSYS);
+ }
+
+ if (inlink->w % 2 == 1 || inlink->h % 2 == 1) {
+ av_log(avctx, AV_LOG_ERROR, "Input with odd dimensions is not supported!\n");
+ return AVERROR(EINVAL);
+ }
+
+ if (ctx->output_camera.width % 2 == 1 || ctx->output_camera.height % 2 == 1) {
+ av_log(avctx, AV_LOG_ERROR, "Output camera must have even dimensions!\n");
+ return AVERROR(EINVAL);
+ }
+
+ /* Output dimensions default to the input dimensions (disregarding cropping) */
+ ctx->ocf.output_width
+ = ctx->output_camera.width ? ctx->output_camera.width : inlink->w;
+
+ ctx->ocf.output_height
+ = ctx->output_camera.height ? ctx->output_camera.height : inlink->h;
+
+ return 0;
+}
+
+/**
+ * Copy the contents of an input frame to an OpenCL buffer.
+ * @param avctx the filter context
+ * @param context the OpenCL context to use
+ * @param command_queue the OpenCL command queue to use
+ * @param frame the input @ref AVFrame
+ * @param input_buffer the OpenCL buffer to copy the frame into
+ * @return 0 on success, otherwise a negative error code
+ */
+static cl_int copy_frame_to_buffer(AVFilterContext *avctx, cl_context context,
+ cl_command_queue command_queue,
+ AVFrame *frame, cl_mem input_buffer)
+{
+ int err;
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+ cl_mem luma = (cl_mem)frame->data[0];
+ cl_mem chroma = (cl_mem)frame->data[1];
+ cl_int cle = 0;
+ size_t src_luma_origin[3] = { frame->crop_left, frame->crop_top, 0 };
+
+ size_t src_chroma_origin[3] = {
+ frame->crop_left / 2,
+ frame->crop_top / 2,
+ 0,
+ };
+
+ size_t luma_region[3] = {
+ ctx->input_camera.width,
+ ctx->input_camera.height,
+ 1,
+ };
+
+ size_t chroma_region[3] = {
+ ctx->input_camera.width / 2,
+ ctx->input_camera.height / 2,
+ 1,
+ };
+
+ cl_event copy_finished[2];
+
+ cle = clEnqueueCopyImageToBuffer(command_queue, luma, input_buffer,
+ src_luma_origin, luma_region, 0, 0, NULL,
+ ©_finished[0]);
+
+ CL_FAIL_ON_ERROR(AVERROR(EINVAL),
+ "Failed to enqueue copy luma image to buffer: %d\n", cle);
+
+ cle = clEnqueueCopyImageToBuffer(
+ command_queue, chroma, input_buffer, src_chroma_origin, chroma_region,
+ ctx->input_camera.width * ctx->input_camera.height * 1, 0, NULL,
+ ©_finished[1]);
+
+ CL_FAIL_ON_ERROR(AVERROR(EINVAL),
+ "Failed to enqueue copy chroma image to buffer: %d\n", cle);
+
+ cle = clWaitForEvents(2, copy_finished);
+
+ CL_FAIL_ON_ERROR(AVERROR(EINVAL), "Failed to copy images to buffer: %d\n", cle);
+
+ return 0;
+
+fail:
+ return err;
+}
+
+/**
+ * Copy the contents of an OpenCL buffer to an output frame.
+ * @param avctx the filter context
+ * @param buffer the OpenCL buffer
+ * @param output_frame the output frame
+ * @return 0 on success, otherwise a negative error code
+ */
+static int copy_buffer_to_frame(AVFilterContext *avctx, cl_mem buffer,
+ AVFrame *output_frame)
+{
+ int err;
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+ cl_mem luma = (cl_mem)output_frame->data[0];
+ cl_mem chroma = (cl_mem)output_frame->data[1];
+ cl_int cle = 0;
+ size_t dst_origin[3] = { 0, 0, 0 };
+ size_t luma_region[3] = { output_frame->width, output_frame->height, 1 };
+
+ size_t chroma_region[3] = {
+ output_frame->width / 2,
+ output_frame->height / 2,
+ 1,
+ };
+
+ cl_event copy_finished[2];
+
+ cle = clEnqueueCopyBufferToImage(ctx->command_queue, buffer, luma, 0, dst_origin,
+ luma_region, 0, NULL, ©_finished[0]);
+
+ CL_FAIL_ON_ERROR(AVERROR(EINVAL),
+ "Failed to enqueue copy buffer to luma image: %d\n", cle);
+
+ cle = clEnqueueCopyBufferToImage(ctx->command_queue, buffer, chroma,
+ output_frame->width * output_frame->height * 1,
+ dst_origin, chroma_region, 0, NULL,
+ ©_finished[1]);
+
+ CL_FAIL_ON_ERROR(AVERROR(EINVAL),
+ "Failed to enqueue copy buffer to luma image: %d\n", cle);
+
+ cle = clWaitForEvents(2, copy_finished);
+ CL_FAIL_ON_ERROR(AVERROR(EINVAL), "Failed to copy buffer to images: %d\n", cle);
+
+ return 0;
+
+fail:
+ return err;
+}
+
+/**
+ * Consume an input frame and push it to the libdewobble filter.
+ * @param avctx the filter context
+ * @param input_frame the input frame
+ * @return 0 on success, otherwise a negative error code
+ */
+static int consume_input_frame(AVFilterContext *avctx, AVFrame *input_frame)
+{
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+ cl_mem input_buffer;
+ int err = 0;
+ cl_int cle;
+
+ if (!input_frame->hw_frames_ctx)
+ return AVERROR(EINVAL);
+
+ if (!ctx->initialized) {
+ av_log(avctx, AV_LOG_VERBOSE, "Initializing\n");
+ err = libdewobble_frames_init(avctx, input_frame);
+
+ if (err < 0)
+ return err;
+ }
+
+ input_buffer
+ = dewobble_filter_get_input_frame_buffer(ctx->dewobble_filter, &cle);
+
+ CL_FAIL_ON_ERROR(AVERROR(ENOMEM), "Failed to create buffer: %d\n", cle);
+
+ err = copy_frame_to_buffer(avctx, ctx->ocf.hwctx->context,
+ ctx->command_queue, input_frame, input_buffer);
+
+ if (err)
+ goto fail;
+
+ /* Free original input frame buffers */
+ for (int i = 0; input_frame->buf[i] != NULL; i++)
+ av_buffer_unref(&input_frame->buf[i]);
+
+ dewobble_filter_push_frame(ctx->dewobble_filter, input_buffer,
+ (void **)input_frame);
+
+ ctx->nb_frames_in_progress += 1;
+ ctx->nb_frames_consumed += 1;
+
+ return 0;
+
+fail:
+ return err;
+}
+
+/**
+ * Create and send on an output frame using an output buffer pulled from the
+ * libdewobble filter.
+ * @param avctx the filter context
+ * @return 0 on success, otherwise a negative error code.
+ */
+static int send_output_frame(AVFilterContext *avctx)
+{
+ AVFilterLink *inlink = avctx->inputs[0];
+ AVFilterLink *outlink = avctx->outputs[0];
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+ AVFrame *input_frame;
+ AVFrame *output_frame = NULL;
+ cl_mem output_buffer = NULL, input_buffer;
+ int err;
+
+ dewobble_filter_pull_frame(ctx->dewobble_filter, &output_buffer,
+ &input_buffer, (void **)&input_frame);
+
+ dewobble_filter_release_input_frame_buffer(ctx->dewobble_filter, &input_buffer);
+
+ output_frame = ff_get_video_buffer(outlink, outlink->w, outlink->h);
+ if (output_frame == NULL) {
+ err = AVERROR(ENOMEM);
+ goto fail;
+ }
+
+ err = av_frame_copy_props(output_frame, input_frame);
+ if (err)
+ goto fail;
+
+ output_frame->crop_top = 0;
+ output_frame->crop_bottom = 0;
+ output_frame->crop_left = 0;
+ output_frame->crop_right = 0;
+
+ err = copy_buffer_to_frame(avctx, output_buffer, output_frame);
+ if (err)
+ goto fail;
+
+ dewobble_filter_release_output_frame_buffer(ctx->dewobble_filter, &output_buffer);
+
+ av_log(avctx, AV_LOG_VERBOSE, "Sending output frame %ld (%d in progress)\n",
+ ctx->nb_frames_consumed - ctx->nb_frames_in_progress,
+ ctx->nb_frames_in_progress);
+
+ ctx->nb_frames_in_progress -= 1;
+
+ err = ff_filter_frame(outlink, output_frame);
+ if (err < 0)
+ goto fail;
+
+ if (!dewobble_filter_frame_ready(ctx->dewobble_filter))
+ ff_inlink_request_frame(inlink);
+
+ if (ctx->input_status && ctx->nb_frames_in_progress == 0) {
+ av_log(avctx, AV_LOG_VERBOSE, "Output reached EOF\n");
+ ff_outlink_set_status(outlink, ctx->input_status, ctx->input_status_pts);
+ }
+
+ av_frame_free(&input_frame);
+
+ return 0;
+
+fail:
+ av_frame_free(&input_frame);
+ av_log(avctx, AV_LOG_ERROR, "Failed to send output frame: %d\n", err);
+ av_frame_free(&output_frame);
+
+ return err;
+}
+
+/**
+ * Attempt to consume an input frame, and push it to the libdewobble filter
+ * if one is available.
+ * @param avctx the filter context
+ * @return 0 on success, otherwise a negative error code
+ */
+static int try_consume_input_frame(AVFilterContext *avctx)
+{
+ AVFilterLink *inlink = avctx->inputs[0];
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+ int err = 0;
+ AVFrame *input_frame;
+
+ /* If necessary, attempt to consume a frame from the input */
+ if (!ctx->initialized || !dewobble_filter_frame_ready(ctx->dewobble_filter)) {
+ err = ff_inlink_consume_frame(inlink, &input_frame);
+ if (err < 0) {
+ av_log(avctx, AV_LOG_ERROR, "Failed to read input frame\n");
+
+ return err;
+ } else if (err > 0) {
+ av_log(avctx, AV_LOG_VERBOSE,
+ "Consuming input frame %ld (%d in progress)\n",
+ ctx->nb_frames_consumed, ctx->nb_frames_in_progress);
+
+ err = consume_input_frame(avctx, input_frame);
+ if (err) {
+ av_log(avctx, AV_LOG_ERROR,
+ "Failed to consume input frame: %d\n", err);
+
+ return err;
+ }
+ }
+ }
+
+ return err;
+}
+
+/**
+ * Read the input status and update the filter state and output status as
+ * appropriate.
+ * @param avctx the filter context
+ */
+static void check_input_status(AVFilterContext *avctx)
+{
+ AVFilterLink *inlink = avctx->inputs[0];
+ AVFilterLink *outlink = avctx->outputs[0];
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+
+ /* Check for end of input */
+ if (!ctx->input_status
+ && ff_inlink_acknowledge_status(inlink, &ctx->input_status,
+ &ctx->input_status_pts)) {
+
+ if (ctx->input_status == AVERROR_EOF) {
+ av_log(avctx, AV_LOG_VERBOSE, "Reached input EOF\n");
+
+ dewobble_filter_end_input(ctx->dewobble_filter);
+ } else
+ av_log(avctx, AV_LOG_ERROR, "Input status: %d\n", ctx->input_status);
+
+ if (ctx->nb_frames_in_progress == 0) {
+ av_log(avctx, AV_LOG_VERBOSE, "Sending output EOF\n");
+
+ ff_outlink_set_status(outlink, ctx->input_status, ctx->input_status_pts);
+ }
+ }
+}
+
+/**
+ * Perform some work to advance the filtering process
+ * @param avctx the filter context
+ * @return 0 if progress was made, otherwise a negative error code
+ */
+static int activate(AVFilterContext *avctx)
+{
+ LibDewobbleOpenCLContext *ctx = avctx->priv;
+ AVFilterLink *inlink = avctx->inputs[0];
+ AVFilterLink *outlink = avctx->outputs[0];
+ int err = 0;
+
+ /* Forward any output status to input */
+ err = ff_outlink_get_status(outlink);
+ if (err) {
+ av_log(avctx, AV_LOG_VERBOSE, "forwarding status to inlink: %d\n", err);
+ ff_inlink_set_status(inlink, err);
+ return 0;
+ }
+
+ /* Consume an input frame if possible */
+ err = try_consume_input_frame(avctx);
+ if (err) {
+ av_log(avctx, AV_LOG_ERROR, "try_consume_input_frame failed: %d\n", err);
+ return 0;
+ }
+
+ /* Check input status, including detecting EOF */
+ check_input_status(avctx);
+
+ /* If possible, send an output frame */
+ if (dewobble_filter_frame_ready(ctx->dewobble_filter)) {
+ err = send_output_frame(avctx);
+ if (err < 0) {
+ av_log(avctx, AV_LOG_ERROR, "send_output_frame failed: %d\n", err);
+ goto fail;
+ }
+ }
+
+ /* Schedule the next activation */
+ if (ff_inlink_check_available_frame(inlink))
+ /* Immediately, if input frames are still queued */
+ ff_filter_set_ready(avctx, 1);
+ else if (dewobble_filter_frame_ready(ctx->dewobble_filter))
+ /* Immediately, if output frames are ready */
+ ff_filter_set_ready(avctx, 1);
+ else
+ /* Otherwise when more input frames are ready */
+ ff_inlink_request_frame(inlink);
+
+ return FFERROR_NOT_READY;
+
+fail:
+ ff_outlink_set_status(outlink, AVERROR_UNKNOWN, 0);
+ return err;
+}
+
+/**
+ * Get the offset of a member in @ref LibDewobbleOpenCLContext
+ */
+#define OFFSET(x) offsetof(LibDewobbleOpenCLContext, x)
+
+/**
+ * Get the offset of a member in @ref Camera
+ */
+#define OFFSET_CAMERA(x) offsetof(Camera, x)
+
+#define FLAGS (AV_OPT_FLAG_FILTERING_PARAM | AV_OPT_FLAG_VIDEO_PARAM)
+
+static const AVOption libdewobble_options[] = {
+ /* Input camera options */
+ {
+ "in_p",
+ "input camera projection model",
+ OFFSET(input_camera) + OFFSET_CAMERA(model),
+ AV_OPT_TYPE_INT,
+ { .i64 = DEWOBBLE_PROJECTION_EQUIDISTANT_FISHEYE },
+ 0,
+ DEWOBBLE_NB_PROJECTIONS - 1,
+ FLAGS,
+ "model",
+ },
+ {
+ "in_dfov",
+ "input camera diagonal field of view in degrees",
+ OFFSET(input_camera) + OFFSET_CAMERA(diagonal_fov),
+ AV_OPT_TYPE_DOUBLE,
+ { .dbl = 0 },
+ 0,
+ DBL_MAX,
+ .flags = FLAGS,
+ },
+ {
+ "in_fx",
+ "horizontal coordinate of focal point in input camera (default: "
+ "center)",
+ OFFSET(input_camera) + OFFSET_CAMERA(focal_point_x),
+ AV_OPT_TYPE_DOUBLE,
+ { .dbl = DBL_MAX },
+ -DBL_MAX,
+ DBL_MAX,
+ .flags = FLAGS,
+ },
+ {
+ "in_fy",
+ "vertical coordinate of focal point in input camera (default: center)",
+ OFFSET(input_camera) + OFFSET_CAMERA(focal_point_y),
+ AV_OPT_TYPE_DOUBLE,
+ { .dbl = DBL_MAX },
+ -DBL_MAX,
+ DBL_MAX,
+ .flags = FLAGS,
+ },
+
+ /* Output camera options */
+ {
+ "out_p",
+ "output camera projection model",
+ OFFSET(output_camera) + OFFSET_CAMERA(model),
+ AV_OPT_TYPE_INT,
+ { .i64 = DEWOBBLE_PROJECTION_RECTILINEAR },
+ 0,
+ DEWOBBLE_NB_PROJECTIONS - 1,
+ FLAGS,
+ "model",
+ },
+ {
+ "out_dfov",
+ "output camera diagonal field of view in degrees",
+ OFFSET(output_camera) + OFFSET_CAMERA(diagonal_fov),
+ AV_OPT_TYPE_DOUBLE,
+ { .dbl = 0 },
+ 0,
+ DBL_MAX,
+ .flags = FLAGS,
+ },
+ {
+ "out_w",
+ "output camera width in pixels (default: same as input)",
+ OFFSET(output_camera) + OFFSET_CAMERA(width),
+ AV_OPT_TYPE_INT,
+ { .i64 = 0 },
+ 0,
+ SHRT_MAX,
+ .flags = FLAGS,
+ },
+ {
+ "out_h",
+ "output camera height in pixels (default: same as input)",
+ OFFSET(output_camera) + OFFSET_CAMERA(height),
+ AV_OPT_TYPE_INT,
+ { .i64 = 0 },
+ 0,
+ SHRT_MAX,
+ .flags = FLAGS,
+ },
+ {
+ "out_fx",
+ "horizontal coordinate of focal point in output camera "
+ "(default: center)",
+ OFFSET(output_camera) + OFFSET_CAMERA(focal_point_x),
+ AV_OPT_TYPE_DOUBLE,
+ { .dbl = DBL_MAX },
+ -DBL_MAX,
+ DBL_MAX,
+ .flags = FLAGS,
+ },
+ {
+ "out_fy",
+ "vertical coordinate of focal point in output camera "
+ "(default: center)",
+ OFFSET(output_camera) + OFFSET_CAMERA(focal_point_y),
+ AV_OPT_TYPE_DOUBLE,
+ { .dbl = DBL_MAX },
+ -DBL_MAX,
+ DBL_MAX,
+ .flags = FLAGS,
+ },
+
+ /* Stabilization options */
+ {
+ "stab",
+ "camera orientation stabilization algorithm",
+ OFFSET(stabilization_algorithm),
+ AV_OPT_TYPE_INT,
+ { .i64 = STABILIZATION_ALGORITHM_SMOOTH },
+ 0,
+ NB_STABILIZATION_ALGORITHMS - 1,
+ FLAGS,
+ "stab",
+ },
+ {
+ "stab_r",
+ "for Savitzky-Golay smoothing: the number of frames "
+ "to look ahead and behind",
+ OFFSET(stabilization_radius),
+ AV_OPT_TYPE_INT,
+ { .i64 = 15 },
+ 1,
+ INT_MAX,
+ FLAGS,
+ },
+ {
+ "stab_h",
+ "for stabilization: the number of frames to look "
+ "ahead to interpolate rotation in frames where it cannot be detected",
+ OFFSET(stabilization_horizon),
+ AV_OPT_TYPE_INT,
+ { .i64 = 30 },
+ 0,
+ INT_MAX,
+ FLAGS,
+ },
+
+ /* General options */
+ {
+ "interp",
+ "interpolation algorithm",
+ OFFSET(interpolation_algorithm),
+ AV_OPT_TYPE_INT,
+ { .i64 = DEWOBBLE_INTERPOLATION_LINEAR },
+ 0,
+ DEWOBBLE_NB_INTERPOLATIONS - 1,
+ FLAGS,
+ "interpolation",
+ },
+ {
+ "border",
+ "border fill mode",
+ OFFSET(border_type),
+ AV_OPT_TYPE_INT,
+ { .i64 = DEWOBBLE_BORDER_CONSTANT },
+ 0,
+ DEWOBBLE_NB_BORDER_TYPES - 1,
+ FLAGS,
+ "border_type",
+ },
+ {
+ "border_r",
+ "border fill color (red component)",
+ OFFSET(border_color) + sizeof(double) * 2,
+ AV_OPT_TYPE_DOUBLE,
+ { .i64 = 0 },
+ 0,
+ 255,
+ FLAGS,
+ },
+ {
+ "border_g",
+ "border fill color (green component)",
+ OFFSET(border_color) + sizeof(double) * 1,
+ AV_OPT_TYPE_DOUBLE,
+ { .i64 = 0 },
+ 0,
+ 255,
+ FLAGS,
+ },
+ {
+ "border_b",
+ "border fill color (blue component)",
+ OFFSET(border_color) + sizeof(double) * 0,
+ AV_OPT_TYPE_DOUBLE,
+ { .i64 = 0 },
+ 0,
+ 255,
+ FLAGS,
+ },
+ {
+ "debug",
+ "whether to include debugging information in the output",
+ OFFSET(debug),
+ AV_OPT_TYPE_BOOL,
+ { .i64 = 0 },
+ 0,
+ 1,
+ FLAGS,
+ },
+
+ /* Camera models */
+ {
+ "rect",
+ "rectilinear projection",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = DEWOBBLE_PROJECTION_RECTILINEAR },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "model",
+ },
+ {
+ "fish",
+ "equidistant fisheye projection",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = DEWOBBLE_PROJECTION_EQUIDISTANT_FISHEYE },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "model",
+ },
+
+ /* Stabilization algorithms */
+ {
+ "fixed",
+ "fix the camera orientation after the first frame",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = STABILIZATION_ALGORITHM_FIXED },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "stab",
+ },
+ {
+ "none",
+ "do not apply stabilization",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = STABILIZATION_ALGORITHM_ORIGINAL },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "stab",
+ },
+ {
+ "sg",
+ "smooth the camera orientation using a Savitzky-Golay filter",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = STABILIZATION_ALGORITHM_SMOOTH },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "stab",
+ },
+
+ /* Interpolation algorithms */
+ {
+ "nearest",
+ "nearest neighbour interpolation (fast)",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = DEWOBBLE_INTERPOLATION_NEAREST },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "interpolation",
+ },
+ {
+ "linear",
+ "bilinear interpolation (fast)",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = DEWOBBLE_INTERPOLATION_LINEAR },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "interpolation",
+ },
+ {
+ "cubic",
+ "bicubic interpolation (medium)",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = DEWOBBLE_INTERPOLATION_CUBIC },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "interpolation",
+ },
+ {
+ "lanczos",
+ "Lanczos4, in an 8x8 neighbourhood (slow)",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = DEWOBBLE_INTERPOLATION_LANCZOS4 },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "interpolation",
+ },
+
+ /* Border fill algorithms */
+ {
+ "constant",
+ "constant color (default black)",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = DEWOBBLE_BORDER_CONSTANT },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "border_type",
+ },
+ {
+ "reflect",
+ "reflection of the input about the edge",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = DEWOBBLE_BORDER_REFLECT },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "border_type",
+ },
+ {
+ "reflect101",
+ "reflection of the input about the middle of the pixel on the edge",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = DEWOBBLE_BORDER_REFLECT_101 },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "border_type",
+ },
+ {
+ "replicate",
+ "replicate the pixel on the edge",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = DEWOBBLE_BORDER_REPLICATE },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "border_type",
+ },
+ {
+ "wrap",
+ "wrap around to the opposite side of the source image",
+ 0,
+ AV_OPT_TYPE_CONST,
+ { .i64 = DEWOBBLE_BORDER_WRAP },
+ INT_MIN,
+ INT_MAX,
+ FLAGS,
+ "border_type",
+ },
+ { NULL },
+};
+
+AVFILTER_DEFINE_CLASS(libdewobble);
+
+static const AVFilterPad inputs[] = {
+ {
+ .name = "default",
+ .type = AVMEDIA_TYPE_VIDEO,
+ .config_props = &libdewobble_config_input,
+ },
+};
+
+static const AVFilterPad outputs[] = {
+ {
+ .name = "default",
+ .type = AVMEDIA_TYPE_VIDEO,
+ .config_props = &ff_opencl_filter_config_output,
+ },
+};
+
+const AVFilter ff_vf_libdewobble = {
+ .name = "libdewobble",
+ .description = NULL_IF_CONFIG_SMALL(
+ "Apply motion stabilization with awareness of camera projection "
+ "and/or change camera projection"),
+ .priv_size = sizeof(LibDewobbleOpenCLContext),
+ .priv_class = &libdewobble_class,
+ .init = &libdewobble_init,
+ .uninit = &libdewobble_uninit,
+ .query_formats = &ff_opencl_filter_query_formats,
+ FILTER_INPUTS(inputs),
+ FILTER_OUTPUTS(outputs),
+ .activate = activate,
+ .flags_internal = FF_FILTER_FLAG_HWFRAME_AWARE,
+};
--
2.33.0
More information about the ffmpeg-devel
mailing list