[FFmpeg-devel] [PATCH 01/12] avutil/frame: add AVFrame.alpha_mode

[Nicolas George]

Niklas Haas (HE12025-02-19):
> FFmpeg currently handles alpha in a quasi-arbitrary way. Some filters/codecs
> assume alpha is premultiplied, others assume it is independent. If there is
> to be any hope for order in this chaos, we need to start by defining an enum
> for the possible range of values.

Please, not like that.

To start with: in libavfilter, if a filter expects full range and
receives premultiplied, then automatic conversion needs to happen. That
means integrating the new flag into the negotiation process. And given
how fragile and uncovered by FATE the negotiation process is, good luck
with that without breaking anything.¹

But more importantly: “We have a bug that makes your output subtly wrong
in some corner cases. To fix that, we introduce this new flag. You have
to take it into account because otherwise any of your output might be
wrong.” This is a terrible API change for applications.

IIRC, we use full range alpha everywhere except in a few specialized
filters. I remember thinking adding these specialized filters was a
terrible idea at the time. It is possible that more such cases have been
added while I was not paying attention.

So my main suggestion is to keep it that way: decide that FFmpeg uses
full range alpha, period. Make it clear in the documentation that the
few filters that use premultiplied are a specialized case for experts
only, with the responsibility of checking the format and converting
resting squarely on the shoulders of these expert users.

Another option would be to treat premultiplied alpha as different pixels
formats: we have YUVA420P for full range alpha, add YUVM420P for
premultiplied alpha.

Last option, the worst one in my opinion: Like you did, but every
component must explicitly declare if it supports premultiplied alpha; if
a premultiplied frame arrives to a component that does not support them,
return an error. The guiding principle is that it is better to fail than
to silently propagate wrong output.


1: On the other hand, if you have energy to spare, adding FATE coverage
to the libavfilter negotiation process would be immensely useful. You
can see an attempt there:
https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2022-August/299593.html
It requires a detailed graph output option, which you are working on, so
that is good.

The process to develop these tests is: find a line in the code that
looks like it needs coverage; understand what it does; conceive a filter
graph where it has a consequence and add it as a test; disable that line
and check the test fails.

Regards,

-- 
  Nicolas George