[FFmpeg-devel] [RFC] Exporting the alpha mode from decoders

[wm4]

On Fri, 6 Feb 2015 12:45:43 +0100
Clément Bœsch <u at pkh.me> wrote:

> On Fri, Feb 06, 2015 at 12:32:14PM +0100, wm4 wrote:
> > This is a proposal for an API extension.
> > 
> > Currently, some pixel formats support alpha, but whether the alpha
> > component contains something useful or just garbage is not part of the
> > pixel format definition. This applies at least to packed RGB formats,
> > where the 4th component is either alpha or garbage depending on the
> > context.
> > 
> > This means that if a decoder outputs such a packed RGB format, an API
> > user has no idea whether it has alpha or not. E.g. take PNG and
> > Camtasia (tscc.c). PNG with alpha outputs AV_PIX_FMT_RGBA, and Camtasia
> > can output AV_PIX_FMT_RGB32 (which is ARGB or BGRA depending on
> > endian). Camtasia supports no alpha, and the alpha component literally
> > contains garbage (AFAIK and judging by the single sample I've seen). An
> > application decoding both of these can't know whether the output frame
> > has alpha or not, unless every codec is special-cased.
> > 
> > One possible solution is duplicating all these pixel formats, so you'd
> > have e.g. AV_PIX_FMT_RGBA and AV_PIX_FMT_RGBX. But I think we all agree
> > that we don't want more pixel formats.
> > 
> > The other solution, which I want to advocate here, is adding a field to
> > AVFrame that indicates the alpha mode. Something like:
> > 
> > typedef enum AVAlphaMode {
> >     // if an alpha component is present, its function is unknown, and
> >     // it may be garbage
> >     AV_ALPHA_UNKNOWN,
> >     // the alpha component contains premultiplied alpha
> >     // (not sure if any file format actually uses this)
> >     AV_ALPHA_PREMULTIPLIED,
> >     // the alpha component contains straight alpha
> >     // (e.g. PNG)
> >     AV_ALPHA_STRAIGHT,
> > } AVAlphaMode;
> > 
> > typedef struct AVFrame {
> >     ...
> >     AVAlphaMode alpha_mode;
> > } AVFrame;
> > 
> > Thoughts?
> 
> What's your opinion on the existing RGB0?

It's not consistently done, doesn't account for the garbage case (RGB0
means the alpha component is always 0, right?), and just contributes to
the combinatorial explosion of pixel formats.

> 
> Another pixel format is useful for at least... libavfilter: some filters
> work with alpha, and some filters do not. RGB0 pixel format makes querying
> such constraint simply and you can confirm the whole input will be
> sanitized.

I consider this an internal problem with libavfilter's format
negotiation, which should have no influence on the cleanliness of the
overall ffmpeg API.