[FFmpeg-devel] [PATCH] libavfilter image transformations

[Daniel G. Taylor]

Hey,

Fixed everything mentioned by Bobby (it is indeed significantly faster 
to not do the floor/ceil all those times and I added interpolation of 
edge pixels as suggested), replying to rest here inline.

On 04/20/2010 10:26 PM, Alexander Strange wrote:
> On Apr 20, 2010, at 5:03 PM, Daniel G. Taylor wrote:
>
>> Hey,
>>
>> Attached is a patch to add matrix transformation utility methods to libavfilter. These are useful for translation, rotation, scaling, etc of input pictures and can be easily used from any filter.
>>
>> This code was originally part of my work on the deshake/stabilize filter and the work of Georg Martius from his transcode transform filter.
>>
>> Example usage:
>>
>> #include "transform.h"
>>
>> ...
>> double matrix[9];
>>
>> avfilter_get_matrix(shift_x, shift_y, radians, scale, (double *)&matrix);
>> avfilter_transform(src, dst, src_stride, dst_stride, width, height,&matrix, INTERPOLATE_BILINEAR, FILL_MIRROR);
>> ...
>>
>> The code supports creating simple matrices from translation, rotation, and scaling parameters, adding/subtracting matrices, multiplying a matrix by a scalar, and doing the actual picture transform. It supports several types of interpolation and edge fill methods.
>>
>> This is useful for any filter that might want to transform video frames in any way in the future. I have at least one filter that will use this but is waiting on motion estimation stuff (deshake/stabilize video).
>
> First, are these transforms separable? I'm not familiar with affine rotations to know how to split them into 1D.

I don't know to be honest.

> Second, can they be done in fixed-point?

I believe so, and given some extra time I will try to port it to be e.g. 
16.16 fixed point arithmetic.

> Third, can it be merged with swscale? Though that looks hard, filters should be using it for scaling and transformation if possible.

I'm not at all familiar with swscale and the code looks a bit daunting. 
I'd rather keep it simple and in libavfilter if possible unless someone 
is willing to step up and make it all work with swscale!

>> +#define INTERPOLATE_METHOD(name) static uint8_t name(float x, float y, const uint8_t *src, int width, int height, int stride, uint8_t def)
>> +
>> +#define PIXEL(img, x, y, w, h, stride, def) ((x)<  0 || (y)<  0) ? (def) : (((x)>= (w) || (y)>= (h)) ? (def) : img[(x) + (y) * (stride)])
>
> Break these lines after the macro name.

Fixed.

>> +        return PIXEL(src, (int)round(x), (int)round(y), width, height, stride, def);
>
> Should be roundf(), and then it should be lrintf(). Unless (int)(x + .5f) is faster.
> (Of course, ignore all of this if you can do it in fixed-point)

Using (int)(x + 0.5f) now and it is faster.

>> +/**
>> + * Bilinear interpolation
>> + */
>
>
> This won't work for downscaling because it won't sample any more than 4 source pixels. Same for the other methods.

I'm not sure how I can fix this - ideas?

>> +        return (uint8_t) s;
>
> The cast doesn't do anything.
>
>> +/**
>> + * Biquadratic interpolation
>> + */
>
> If you're going to provide something better than linear, at least make it bicubic or slower than that. swscale has several good kernels.

Ah, this was part of Georg's code that I had borrowed (and now properly 
credited him). I'll try and see if I can figure out how to do a bicubic 
interpolation and code that up.

>> +        x_c = (int)ceilf(x);
>> +        x_f = (int)floorf(x);
>> +        y_c = (int)ceilf(y);
>> +        y_f = (int)floorf(y);
>
> These casts are unnecessary too.

Fixed.

>> +void avfilter_get_matrix(double x_shift, double y_shift, double angle, double zoom, double *matrix) {
>> +    matrix[0] = zoom * cos(angle);
>> +    matrix[1] = -sin(angle);
>> +    matrix[2] = x_shift;
>> +    matrix[3] = -matrix[1];
>> +    matrix[4] = matrix[0];
>> +    matrix[5] = y_shift;
>> +    matrix[6] = 0;
>> +    matrix[7] = 0;
>> +    matrix[8] = 1;
>> +}
>
> Use float instead of double unless it's really unstable.

Fixed.

> Just use a loop for add/sub/mul.

Fixed.

>> +void avfilter_transform(const uint8_t *src, uint8_t *dst, int src_stride, int dst_stride, int width, int height, const double *matrix, enum InterpolateMethod interpolate, enum FillMethod fill)
>
> This line is too long.

Fixed.

>> +                case FILL_EXTRUDE:
>
> Call it CLAMP, maybe?

Fixed.

>> +                    def = src[(int)((y_s<  0) ? -y_s : (y_s>= height) ? (height + height - y_s) : y_s) * src_stride + (int)((x_s<  0) ? -x_s : (x_s>= width) ? (width + width - x_s) : x_s)];
>
> Split this into multiple statements so the lines are shorter.

Fixed.

Take care,
-- 
Daniel G. Taylor
http://programmer-art.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: transform.diff
Type: text/x-diff
Size: 11135 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100422/f4f09278/attachment.diff>