[FFmpeg-user] Blending two inputs with custom expression

[Gyan]

On Tue, May 9, 2017 at 7:47 AM, SviMik <svimik at gmail.com> wrote:

> Let's assume A is a video, and B is a png image with alpha channel. I need
> to do the following blending (assuming the format is rgba and B_alpha is in
> 0...1 range):
>
> red = (A_red - B_red * B_alpha) / (1 - B_alpha);
> green = (A_green - B_green * B_alpha) / (1 - B_alpha);
> blue = (A_blue - B_blue * B_alpha) / (1 - B_alpha);
>

You can achieve this using a sequence of filters, where the above
expression is realized piecemeal. I haven't tested this, but it should work.

I assume the first input to ffmpeg is the video, and the (unlooped) image
second. And both are 8-bit RGBA.

-filter_complex "[1]geq=r='p(X,Y)*alpha(X,Y)/255':g='p(X,Y)*alpha(X,Y)/255':
b='p(X,Y)*alpha(X,Y)/255'[imgpremult]; [imgpremult][0]blend=all_expr=
B-A:c3_expr=A,lutrgb=a=maxval-val,geq=r='255*p(X,Y)/alpha(X,
Y)':g='255*p(X,Y)/alpha(X,Y)':b='255*p(X,Y)/alpha(X,Y)'"

The video output will have an alpha plane populated during the blend
operation. If you need to preserve the original alpha, insert
[0]blend=all_expr=B:c3_expr=A at the end. Only one input is specified, the
2nd is the unconnected output from the last geq filter.

There's no attempt to clip or validate the values from any of the
expressions e.g. A_red - B_red * B_alpha = -0.5 if A_red = 0.5, B_red=1,
B_alpha=1. Which is invalid as it's out of range [0,1] and will remain so
even after division by 1 - B_alpha. Not to mention how you wish to handle
B_alpha = 1.