[FFmpeg-devel] [PATCH] Revert "avfilter/vf_palette(gen|use): support palettes with alpha"

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Clément Bœsch
> Sent: Monday, October 31, 2022 1:30 AM
> To: FFmpeg development discussions and patches <ffmpeg-
> devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] Revert
> "avfilter/vf_palette(gen|use): support palettes with alpha"
> 
> On Sun, Oct 30, 2022 at 10:55:31PM +0000, Soft Works wrote:
> [...]
> > > I understand why. I know that it's not perfect. But it's the best
> > > what's achievable within the way the filter is working.
> > >
> > > But I wouldn't go that far as saying it would be "broken". I
> think
> > > the result is quite acceptable with regards to the method being
> > > using.
> 
> It's broken because the alpha channel in the output is really
> completely
> random. If we blend the output of that "O" PNG somewhere, it's going
> to be
> a real mess.
> 
> Here is a more concrete example: if your input has some red fully
> transparent (00ff0000), and some red fully opaque (ffff0000) which
> end up
> in the same box, they will be averaged to a red with an alpha of
> 0x80, and
> I'm not even accounting for the weight and other colors with
> different
> transparency. That non-opaque average alpha might end up being used
> in
> area that are expected to be opaque, and in some area where it's
> supposed
> to be transparent. That's exactly what I showed with the "O" png.
> 
> In addition to that problem, since you're also accounting the alpha
> as a
> weight to determine the proximity of 2 colors, you're going to select
> the
> wrong colors. For example if we want to find the closest color to an
> opaque green (ff00ff00), and the palette has a slightly transparent
> green
> (fa00ff00) and an opaque blue (ff0000ff), then now the algorithm will
> prefer the blue over the green.
> 
> That explains why in addition to the alpha being random in the "O"
> png,
> the colors are also messed up.
> 
> > > The patch I had submitted doesn't change the previous behavior
> > > without the use_alpha parameter.
> 
> Yes I noticed, but unfortunately I'm reworking the color distance to
> work
> in perceptual color space, and the way that alpha is mixed up in the
> equation just doesn't make any sense at all and prevents me from
> doing
> these changes. 

If you want to implement a new color distance algorithm, it should 
be either a new filter or a new (switchable) mode for the existing 
filter. Photoshop has these different modes as well and it would 
surely be useful, but I don't think it should be replacing the
existing behavior.

When it turns out that the use_alpha implementation doesn't fit
with your new color distance calculation and you add it as 
an additional mode, then it would be fine IMO when the filter
errors in case it would be attempted to use that mode in 
combination with use_alpha.


> > Do you think it might make sense to put more weight on the
> > alpha value by tripling it? So it would be weighted equally to the
> > RGB value?
> 
> You cannot mix alpha with colors at all, they are separate domains
> and you
> need to treat them as such.

What's interesting is that I've followed the same (simplified)
way for adding a use_alpha option to vf_elbg and it provides excellent
results without treating alpha separately.


> From paletteuse perspective what you need to do is first choose the
> colors
> in the palette that match exactly the alpha (or at least the closest
> if
> and only there is no exact match). Then within that set, and only
> within
> that one, you'd pick the closest color.
> 
> From palettegen perspective, you need to split the colors in
> different
> transparency domain (a first dimensional quantization), then quantize
> the
> colors in each quantized alpha dimension. And when you have all your
> quantized palettes for each level of alpha, you find an algorithm to
> reduce the number of transparency dimensions or the number of colors
> per
> dimension to make it fit inside a single palette. But you can't just
> do
> the alpha and the colors at the same time, it cannot work, whatever
> weights you choose.

I would be curious to see how well that would work, especially
in cases when the target palettes have just a few number of colors.


But to return to the proposal of removal: If everything from ffmpeg
would be removed which is not perfect, then it would be lacking
quite a number of features I suppose :-)

In the same way, one could say that palettegen/-use should be removed
because its results are wrong and colors are randomly mixed and 
misplaced while the vf_elbg filter does it right.

When you look at the result under the heading

"Paletteuse/gen Regular (to 8-bit non-alpha palette; only single 
transparent color)"
https://gist.github.com/softworkz/deef5c2a43d3d629c3e17f9e21544a8f?permalink_comment_id=3905155#gistcomment-3905155

Even without the alpha, many color pixels appear to be wrong and
random like for example the light purple pixels on the darker purple
at the bottom of the "O". That's not much different from irregularities
in the alpha channel you've shown (https://imgur.com/a/50YyRGV).

So, I agree to that it's not perfect, but the whole filter is
not perfect (vf_elbg is close-to). Do we remove the filter because
it's not perfect?

As mentioned above, if you want to add an additional mode for 
calculating the color distance, it's fine when it doesn't 
work with use_alpha IMO.

Thanks,
softworkz