[FFmpeg-devel] [PATCH] lavfi: add xbr filter
Michael Niedermayer
michaelni at gmx.at
Sat Nov 1 01:11:25 CET 2014
On Sat, Nov 01, 2014 at 04:27:24AM +0530, arwa arif wrote:
> On Fri, Oct 31, 2014 at 6:31 PM, Stefano Sabatini <stefasab at gmail.com>
> wrote:
>
> > On date Friday 2014-10-31 01:36:46 +0530, arwa arif encoded:
> > [...]
> >
> > > From b9b56b594f856fef8b113b283df2d2045e8357f7 Mon Sep 17 00:00:00 2001
> > > From: Arwa Arif <arwaarif1994 at gmail.com>
> > > Date: Thu, 30 Oct 2014 22:06:20 +0530
> > > Subject: [PATCH] [PATCH]lvafi: add xbr filter
> >
> > typo: lavfi:
> >
> > >
> > > ---
> > > doc/filters.texi | 5 +
> > > libavfilter/Makefile | 1 +
> > > libavfilter/allfilters.c | 1 +
> > > libavfilter/vf_xbr.c | 319
> > ++++++++++++++++++++++++++++++++++++++++++++++
> > > 4 files changed, 326 insertions(+)
> > > create mode 100644 libavfilter/vf_xbr.c
> > >
> > > diff --git a/doc/filters.texi b/doc/filters.texi
> > > index 7be29de..2905e5d 100644
> > > --- a/doc/filters.texi
> > > +++ b/doc/filters.texi
> > > @@ -9163,6 +9163,11 @@ Only deinterlace frames marked as interlaced.
> > > Default value is @samp{all}.
> > > @end table
> > >
> > > + at section xbr
> > > +Apply high-quality magnification filter which is designed for pixel
> > art. It follows a set
> > > +of edge-detection rules @url{
> > http://www.libretro.com/forums/viewtopic.php?f=6&t=134}.
> > > +This filter was originally created by Hyllian.
> > > +
> > > @anchor{yadif}
> > > @section yadif
> > >
> > > diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> > > index 6d868e7..2c56e38 100644
> > > --- a/libavfilter/Makefile
> > > +++ b/libavfilter/Makefile
> > > @@ -198,6 +198,7 @@ OBJS-$(CONFIG_VIDSTABDETECT_FILTER) +=
> > vidstabutils.o vf_vidstabdetect.
> > > OBJS-$(CONFIG_VIDSTABTRANSFORM_FILTER) += vidstabutils.o
> > vf_vidstabtransform.o
> > > OBJS-$(CONFIG_VIGNETTE_FILTER) += vf_vignette.o
> > > OBJS-$(CONFIG_W3FDIF_FILTER) += vf_w3fdif.o
> > > +OBJS-$(CONFIG_XBR_FILTER) += vf_xbr.o
> > > OBJS-$(CONFIG_YADIF_FILTER) += vf_yadif.o
> > > OBJS-$(CONFIG_ZMQ_FILTER) += f_zmq.o
> > > OBJS-$(CONFIG_ZOOMPAN_FILTER) += vf_zoompan.o
> > > diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> > > index d88a9ad..2352d44 100644
> > > --- a/libavfilter/allfilters.c
> > > +++ b/libavfilter/allfilters.c
> > > @@ -213,6 +213,7 @@ void avfilter_register_all(void)
> > > REGISTER_FILTER(VIDSTABTRANSFORM, vidstabtransform, vf);
> > > REGISTER_FILTER(VIGNETTE, vignette, vf);
> > > REGISTER_FILTER(W3FDIF, w3fdif, vf);
> > > + REGISTER_FILTER(XBR, xbr, vf);
> > > REGISTER_FILTER(YADIF, yadif, vf);
> > > REGISTER_FILTER(ZMQ, zmq, vf);
> > > REGISTER_FILTER(ZOOMPAN, zoompan, vf);
> > > diff --git a/libavfilter/vf_xbr.c b/libavfilter/vf_xbr.c
> > > new file mode 100644
> > > index 0000000..1a828d8
> > > --- /dev/null
> > > +++ b/libavfilter/vf_xbr.c
> > > @@ -0,0 +1,319 @@
> > > +/*
> > > + * This file is part of FFmpeg.
> > > + *
> > > + * Copyright (c) 2014 Arwa Arif <arwaarif1994 at gmail.com>
> > > + *
> > > + * FFmpeg is free software; you can redistribute it and/or
> > > + * modify it under the terms of the GNU Lesser General Public
> > > + * License as published by the Free Software Foundation; either
> > > + * version 2.1 of the License, or (at your option) any later version.
> > > + *
> > > + * FFmpeg is distributed in the hope that it will be useful,
> > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> > > + * Lesser General Public License for more details.
> > > + *
> > > + * You should have received a copy of the GNU Lesser General Public
> > > + * License along with FFmpeg; if not, write to the Free Software
> > > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
> > 02110-1301 USA
> > > + */
> > > +
> > > +/**
> > > + * @file
> > > + * XBR Filter is used for depixelization of image.
> > > + * This is based on Hyllian's 2xBR shader.
> > > + * 2xBR Filter v0.2.5
> > > + * @see : http://www.libretro.com/forums/viewtopic.php?f=6&t=134
> > > + * Future work : To implement x3 and x4 scale, and threading.
> > > + */
> > > +
> > > +#include "libavutil/opt.h"
> > > +#include "libavutil/avassert.h"
> > > +#include "libavutil/pixdesc.h"
> > > +#include "internal.h"
> > > +
> > > +typedef struct {
> > > + uint32_t rgbtoyuv[1<<24];
> > > +} xBRContext;
> > > +
> > > +/**
> > > +* Calculates the weight of difference of the pixels, by transforming
> > these
> > > +* pixels into their Y'UV parts. It then uses the threshold used by HQx
> > filters:
> > > +* 48*Y + 7*U + 6*V, to give it those smooth looking edges.
> > > +**/
> > > +static int d(AVFrame *in, int x1, int y1, int x2, int y2, const
> > uint32_t *r2y)
> >
> > nit: use a more significant name, for example compute_diff() or diff()
> >
> > > +{
> > > +
> > > +#define YMASK 0xff0000
> > > +#define UMASK 0x00ff00
> > > +#define VMASK 0x0000ff
> > > +
> > > + int r1 = *(in->data[0] + y1 * in->linesize[0] + x1*3);
> > > + int g1 = *(in->data[0] + y1 * in->linesize[0] + x1*3 + 1);
> > > + int b1 = *(in->data[0] + y1 * in->linesize[0] + x1*3 + 2);
> > > +
> > > + int r2 = *(in->data[0] + y2 * in->linesize[0] + x2*3);
> > > + int g2 = *(in->data[0] + y2 * in->linesize[0] + x2*3 + 1);
> > > + int b2 = *(in->data[0] + y2 * in->linesize[0] + x2*3 + 2);
> > > +
> > > + uint32_t c1 = r1 | g1<<8 | b1<<16 ;
> > > + uint32_t c2 = r2 | g2<<8 | b2<<16 ;
> > > +
> >
> > > + uint32_t yuv1 = r2y[c1 & 0xffffff];
> > > + uint32_t yuv2 = r2y[c2 & 0xffffff];
> >
> > is the 0xffffff mask required?
> >
> > > +
> > > + return abs((yuv1 & YMASK) - (yuv2 & YMASK)) > (48 << 16) ||
> > > + abs((yuv1 & UMASK) - (yuv2 & UMASK)) > ( 7 << 8) ||
> > > + abs((yuv1 & VMASK) - (yuv2 & VMASK)) > ( 6 << 0);
> > > +}
> > > +
> > > +/**
> > > +* Mixes a pixel A, with pixel B, with B's transparency set to 'a'
> > > +* In other words, A is a solid color (bottom) and B is a transparent
> > color (top)
> > > +**/
> > > +static int mix(AVFrame *in, int x1, int y1, int x2, int y2, int a, int
> > color)
> > > +{
> > > + int col1,col2;
> > > + col1 = *(in->data[0] + y1 * in->linesize[0] + x1*3 + color);
> > > + col2 = *(in->data[0] + y2 * in->linesize[0] + x2*3 + color);
> > > +
> > > + return (a*col2 + (2-a)*col1)/2;
> > > +};
> > > +
> > > +/**
> > > +* Fills the output matrix
> > > +**/
> > > +static void fill(AVFrame *in, AVFrame *out, int u, int v, int x, int y,
> > int mode, int new_x, int new_y)
> > > +{
> > > + int r,g,b;
> > > + /*mix colors if they are not on boundary*/
> > > + if (mode!=0 && u>=0 && v>=0 && u<in->width && v<in->height) {
> > > +
> > > + r = mix(in, u, v, x, y, 1, 0);
> > > + g = mix(in, u, v, x, y, 1, 1);
> > > + b = mix(in, u, v, x, y, 1, 2);
> > > +
> > > + } else {
> > > +
> > > + r = *(in->data[0] + y*in->linesize[0] + x*3);
> > > + g = *(in->data[0] + y*in->linesize[0] + x*3 + 1);
> > > + b = *(in->data[0] + y*in->linesize[0] + x*3 + 2);
> > > + }
> > > +
> >
> > > + /*Insert blended color into scaledImageData*/
> >
> > scaledImageData => output image
> >
> > > + *(out->data[0] + (new_y)*out->linesize[0] + (new_x)*3) = r;
> > > + *(out->data[0] + (new_y)*out->linesize[0] + (new_x)*3 + 1) = g;
> > > + *(out->data[0] + (new_y)*out->linesize[0] + (new_x)*3 + 2) = b;
> >
> > nit++: probably unrelevant, but you can factorize like this:
> > uint8_t *outp = out->data[0] + new_y*out->linesize[0] + new_x*3;
> > outp[0] = r;
> > outp[1] = g;
> > outp[2] = b;
> >
> > same above
> >
> > > + return;
> > > +}
> > > +
> > > +/**
> > > +* Applies the xBR filter rules.
> > > +**/
> > > +static void apply_edge_detection_rules(AVFrame *in, AVFrame *out, int
> > x, int y, const uint32_t *r2y)
> > > +{
> > > + /* Matrix: (E is 0,0 i.e: current pixel)
> > > + -2 | -1| 0| +1| +2 (x)
> > > + ______________________________
> > > + -2 | [A1][B1][C1]
> > > + -1 | [A0][ A][ B][ C][C4]
> > > + 0 | [D0][ D][ E][ F][F4]
> > > + +1 | [G0][ G][ H][ I][I4]
> > > + +2 | [G5][H5][I5]
> > > + |(y)|
> > > + */
> > > +
> > > + /*Cached Pixel Weight Difference*/
> > > + int d_E_D = d(in, x, y, x-1, y, r2y);
> > > + int d_E_B = d(in, x, y, x, y-1, r2y);
> > > + int d_E_F = d(in, x, y, x+1, y, r2y);
> > > + int d_E_H = d(in, x, y, x, y+1, r2y);
> > > + int d_E_G = d(in, x, y, x-1, y+1, r2y);
> > > + int d_E_C = d(in, x, y, x+1, y-1, r2y);
> > > + int d_A_D0 = d(in, x-1, y-1, x-2, y, r2y);
> > > + int d_A_B1 = d(in, x-1, y-1, x, y-2, r2y);
> > > + int d_D_B = d(in, x-1, y, x, y-1, r2y);
> > > + int d_D_H = d(in, x-1, y, x, y+1, r2y);
> > > + int d_D_A0 = d(in, x-1, y, x-2, y-1, r2y);
> > > + int d_B_F = d(in, x, y-1, x+1, y, r2y);
> > > + int d_B_A1 = d(in, x, y-1, x-1, y-2, r2y);
> > > + int d_E_A = d(in, x, y, x-1, y-1, r2y);
> > > + int d_E_I = d(in, x, y, x+1, y+1, r2y);
> > > + int d_C_F4 = d(in, x+1, y-1, x+2, y, r2y);
> > > + int d_C_B1 = d(in, x+1, y-1, x, y-2, r2y);
> > > + int d_F_H = d(in, x+1, y, x, y+1, r2y);
> > > + int d_F_C4 = d(in, x+1, y, x+2, y-1, r2y);
> > > + int d_B_C1 = d(in, x, y-1, x+1, y-2, r2y);
> > > + int d_G_D0 = d(in, x-1, y+1, x-2, y, r2y);
> > > + int d_G_H5 = d(in, x-1, y+1, x, y+2, r2y);
> > > + int d_H_G5 = d(in, x, y+1, x-1, y+2, r2y);
> > > + int d_D_G0 = d(in, x-1, y, x-2, y+1, r2y);
> > > + int d_I_F4 = d(in, x+1, y+1, x+2, y, r2y);
> > > + int d_I_H5 = d(in, x+1, y+1, x, y+2, r2y);
> > > + int d_H_I5 = d(in, x, y+1, x+1, y+2, r2y);
> > > + int d_H_I4 = d(in, x, y+1, x+2, y+1, r2y);
> >
> > This is accessing out-of-image pixels at the border. You need probably
> > to normalize the values with something like this:
> >
> > int d_E_D = d(in, x, y, FFMAX(x-1, 0), y, r2y);
> > int d_E_B = d(in, x, y, x, FFMAX(y-1, 0), r2y);
> > int d_E_F = d(in, x, y, FFMIN(x+1, w-1), y, r2y);
> > ...
> >
> >
> > > +
> > > + /**
> > > + * Note: On reading edge detection rules
> > > + *
> >
> > > + * Each edge rule is an if..else statement, everytime on else, the
> > > + * current pixel color pointed to by matrix[0] is used to color it's
> > edge.
> >
> > its edge
> >
> > > + *
> > > + * Each if statement checks wether the sum of weight difference on
> > the left is
> > > + * lesser than that of the right weight differece.
> >
> > difference
> >
> > (please never disregard my previous comments, I already pointed out
> > those typos and having to repeat myself is annoying)
> >
> > > + */
> > > +
> >
> > > + /**
> > > + * Top Left Edge Detection Rule
> > > + **/
> > > + if ((d_E_G+d_E_C+d_A_D0+d_A_B1+(4*d_D_B)) <
> > (d_D_H+d_D_A0+d_B_F+d_B_A1+(4*d_E_A))) {
> > > + // Figure what color to blend with current pixel -->E
> >
> > please remove these comments as they are confusing (or improve them, I
> > cannot figure out what -->E means)
> >
> > > + if (d_E_D <= d_E_B)
> > > + fill(in, out, x-1, y, x, y, 1, x*2, y*2);
> > > + else
> > > + fill(in, out, x, y-1, x, y, 1, x*2, y*2);
> > > + } else {
> > > + /*Insert current pixel color into output frame*/
> > > + fill(in, out, x, y, x, y, 0, x*2, y*2);
> > > + }
> > > + /**
> > > + * Top Right Edge Detection Rule
> > > + **/
> > > + if ((d_E_I+d_E_A+d_C_F4+d_C_B1+(4*d_B_F)) <
> > (d_F_H+d_F_C4+d_D_B+d_B_C1+(4*d_E_C))) {
> > > + // Figure what color to blend with current pixel --> E
> > > + if (d_E_B <= d_E_F)
> > > + fill(in, out, x, y-1, x, y, 1, (x*2)+1, y*2);
> > > + else
> > > + fill(in, out, x+1, y, x, y, 1, (x*2)+1, y*2);
> > > + } else {
> > > + /*Insert current pixel color into output frame*/
> > > + fill(in, out, x, y, x, y, 0, (x*2)+1, y*2);
> > > + }
> > > +
> > > + /**
> > > + * Bottom Left Edge Detection Rule
> > > + **/
> > > + if ((d_E_A+d_E_I+d_G_D0+d_G_H5+(4*d_D_H)) <
> > (d_D_B+d_D_G0+d_F_H+d_H_G5+(4*d_E_G))) {
> > > + // Figure what color to blend with current pixel --> E
> > > + if (d_E_D <= d_E_H)
> > > + fill(in, out, x-1, y, x, y, 1, x*2, (y*2)+1);
> > > + else
> > > + fill(in, out, x, y+1, x, y, 1, x*2, (y*2)+1);
> > > +
> > > + } else {
> > > + /*Insert current pixel color into output frame*/
> > > + fill(in, out, x, y, x, y, 0, x*2, (y*2)+1);
> > > + }
> > > +
> > > + /**
> > > + * Bottom Right Edge Detection Rule
> > > + **/
> > > + if ((d_E_C+d_E_G+d_I_F4+d_I_H5+(4*d_F_H)) <
> > (d_D_H+d_H_I5+d_H_I4+d_B_F+(4*d_E_I))) {
> > > + // Figure what color to blend with current pixel --> E
> > > + if (d_E_F <= d_E_H)
> > > + fill(in, out, x+1, y, x, y, 1, (x*2)+1, (y*2)+1);
> > > + else
> > > + fill(in, out, x, y+1, x, y, 1, (x*2)+1, (y*2)+1);
> > > +
> > > + } else {
> > > + /*Insert current pixel color into output frame*/
> > > + fill(in, out, x, y, x, y, 0, (x*2)+1, (y*2)+1);
> > > + }
> >
> > this could be factorized through a macro. Feel free to discard this
> > comment though.
> >
> > > +}
> > > +
> > > +static int config_output(AVFilterLink *outlink)
> > > +{
> > > + AVFilterContext *ctx = outlink->src;
> > > + AVFilterLink *inlink = ctx->inputs[0];
> > > +
> >
> > > + outlink->w = inlink->w * 2 ;
> > > + outlink->h = inlink->h * 2 ;
> >
> > nit:
> > outlink->h = inlink->h * 2;
> >
> >
> > > + return 0;
> > > +}
> > > +
> > > +static int query_formats(AVFilterContext *ctx)
> > > +{
> > > + static const enum AVPixelFormat pix_fmts[] = {
> > > + AV_PIX_FMT_RGB24, AV_PIX_FMT_BGR24,AV_PIX_FMT_NONE,
> > > + };
> > > +
> > > + ff_set_common_formats(ctx, ff_make_format_list(pix_fmts));
> > > + return 0;
> > > +}
> > > +
> > > +static int filter_frame(AVFilterLink *inlink, AVFrame *in)
> > > +{
> > > + AVFilterContext *ctx = inlink->dst;
> > > + AVFilterLink *outlink = ctx->outputs[0];
> > > + int i,j;
> > > + xBRContext *xBR = ctx->priv;
> > > + const uint32_t *r2y = xBR->rgbtoyuv;
> > > + AVFrame *out = ff_get_video_buffer(outlink, outlink->w, outlink->h);
> > > + if (!out) {
> > > + av_frame_free(&in);
> > > + return AVERROR(ENOMEM);
> > > + }
> > > +
> > > + av_frame_copy_props(out, in);
> >
> > > + for (i=0; i<inlink->w; i++)
> >
> > spaces increase readability. The usual FFmpeg style is:
> >
> > for (i = 0; i < inlink->w; i++)
> >
> > > + for (j=0; j<inlink->h; j++)
> > > + apply_edge_detection_rules(in, out, i, j, r2y);
> >
> > tab
> >
> > > +
> > > + out->width = outlink->w;
> > > + out->height = outlink->h;
> > > +
> > > + av_frame_free(&in);
> > > + return ff_filter_frame(outlink, out);
> > > +}
> > > +
> >
> > > +static int init(AVFilterContext *ctx)
> >
> > static av_cold int init(AVFilterContext *ctx)
> >
> > > +{
> > > + xBRContext *xbr = ctx->priv;
> > > + uint32_t c;
> > > + int bg, rg, g;
> > > +
> > > + for (bg=-255; bg<256; bg++) {
> > > + for (rg=-255; rg<256; rg++) {
> > > + const uint32_t u = (uint32_t)((-169*rg + 500*bg)/1000) +
> > 128;
> > > + const uint32_t v = (uint32_t)(( 500*rg - 81*bg)/1000) +
> > 128;
> > > + int startg = FFMAX3(-bg, -rg, 0);
> > > + int endg = FFMIN3(255-bg, 255-rg, 255);
> > > + uint32_t y = (uint32_t)(( 299*rg + 1000*startg +
> > 114*bg)/1000);
> > > + c = bg + (rg<<16) + 0x010101 * startg;
> > > + for (g = startg; g <= endg; g++) {
> > > + xbr->rgbtoyuv[c] = ((y++) << 16) + (u << 8) + v;
> > > + c+= 0x010101;
> > > + }
> > > + }
> > > + }
> > > + return 0;
> > > +}
> > > +
> > > +static const AVFilterPad xbr_inputs[] = {
> > > + {
> > > + .name = "default",
> > > + .type = AVMEDIA_TYPE_VIDEO,
> > > + .filter_frame = filter_frame,
> > > + },
> > > + { NULL }
> > > +};
> > > +
> > > +static const AVFilterPad xbr_outputs[] = {
> > > + {
> > > + .name = "default",
> > > + .type = AVMEDIA_TYPE_VIDEO,
> > > + .config_props = config_output,
> > > + },
> > > + { NULL }
> > > +};
> > > +
> > > +AVFilter ff_vf_xbr = {
> > > + .name = "xbr",
> > > + .description = NULL_IF_CONFIG_SMALL("Scale the input by 2 using
> > xbr algorithm."),
> > > + .priv_size = sizeof(xBRContext),
> > > + .inputs = xbr_inputs,
> > > + .outputs = xbr_outputs,
> > > + .query_formats = query_formats,
> > > + .init = init,
> > > +};
> >
> > I'd say the patch looks good otherwise.
> > --
> > FFmpeg = Frenzy Fundamentalist Multipurpose Portentous Elastic Goblin
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >
> doc/filters.texi | 5
> libavfilter/Makefile | 1
> libavfilter/allfilters.c | 1
> libavfilter/vf_xbr.c | 331 +++++++++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 338 insertions(+)
> 9a2a1bb515f4ede61706bf0f3fec31c1d6225ca0 0001-PATCH-lvafi-add-xbr-filter.patch
> From 9b6b1d546247cf61b2f696835fb8a5e782d353ea Mon Sep 17 00:00:00 2001
> From: Arwa Arif <arwaarif1994 at gmail.com>
> Date: Thu, 30 Oct 2014 22:06:20 +0530
> Subject: [PATCH] [PATCH]lvafi: add xbr filter
i think the code looks quite good but iam not sure its output is
correct
with the 2 test samples from fate
fate-suite/filter/pixelart0.png
fate-suite/filter/pixelart1.png
that is i see some of the diagonals which are closeer to vertical or
horizontal than 45° have quite a few "staircase" artifacts
while looking around on the net i found someone filtered these
http://ngemu.com/threads/xbrz-high-quality-image-scaler-for-epsxe.156188/#post-2102528
and they dont have anything similar
also the bomb is quite far away from round
through they are xbr x4, yes its sadly not x2, if someone has
reference images for x2 that would be very usefull (that is original+
filtered through thr reference xbr 2x code)
thanks
[...]
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
If a bugfix only changes things apparently unrelated to the bug with no
further explanation, that is a good sign that the bugfix is wrong.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20141101/31402318/attachment.asc>
More information about the ffmpeg-devel
mailing list