[FFmpeg-devel] [PATCH 1/2] pixdesc: mark AV_PIX_FMT_PAL8 as having alpha

[wm4]

On Tue, 10 Feb 2015 12:47:04 +0100
Michael Niedermayer <michaelni at gmx.at> wrote:

> On Tue, Feb 10, 2015 at 12:20:01PM +0100, wm4 wrote:
> > On Mon, 9 Feb 2015 23:49:00 +0100
> > Michael Niedermayer <michaelni at gmx.at> wrote:
> > 
> > > On Mon, Feb 09, 2015 at 10:33:53PM +0100, wm4 wrote:
> > > > Otherwise, you could lose the alpha when handling pixel formats in an
> > > > opaque manner (i.e. when you don't special-case PAL8).
> > > > 
> > > > The special case for av_get_pix_fmt_loss()/av_find_best_pix_fmt_of_2()
> > > > is now redundant and can be removed.
> > > > ---
> > > > Fate still passes.
> > > 
> > > fate fails here
> > > make V=2 fate-ac3-2.0
> > > TEST    ac3-2.0
> > > [...]
> > > Assertion (d->nb_components==4 || d->nb_components==2) == !!(d->flags & (1 << 7)) failed at libavutil/pixdesc.c:2097
> > > Aborted (core dumped)
> > > make: *** [fate-ac3-2.0] Error 134
> > > 
> > > [...]
> > 
> > It doesn't fail here, even with the same command.
> 
> indeed, re reading the code the call is under
> #if defined(ASSERT_LEVEL) && ASSERT_LEVEL > 0
> 
> 
> > 
> > Anyway, patch dropped, too many problems and inconsistencies. (There's
> > probably a lot of code that checks for the number of components
> > explicitly. Although the AVPixFmtDescriptor.comp doxygen is completely
> > inconsistent with PAL8. Well, whatever.)
> 
> I think as is, the best would be to check if any of the 256 entries
> has a non 0xFF alpha to find out if theres alpha
> 
> If thats insufficient, we could add a 2nd PAL8 pixel format with set
> alpha flag to distingish pal8 from transparent pal8
> 
> this additional information could be usefull in selecting the
> pixel formats in convertion like if RGB24 or RGBA is the optimal
> format for a PAL8 source when PAL8 itself cannot be used
> 

That's not the problem. The problem is knowing whether the palette
entry's 4th component contains garbage, or alpha. If PAL8 is not marked
as having alpha, you could interpret it such that the 4th component
is never written/interpreted by anything. 

It should just be made clear SOMEWHERE, how else is the API user
supposed to know? What's the problem with making it explicit?