[FFmpeg-devel] [PATCH] Make decoding alpha optional for somecodecs.

[Reimar Döffinger]

Don Moir <donmoir at comcast.net> wrote:
>
>----- Original Message ----- 
>From: "Reimar Döffinger" <Reimar.Doeffinger at gmx.de>
>To: "FFmpeg development discussions and patches"
><ffmpeg-devel at ffmpeg.org>
>Sent: Friday, September 20, 2013 1:54 AM
>Subject: Re: [FFmpeg-devel] [PATCH] Make decoding alpha optional for
>somecodecs.
>
>
>> On 20.09.2013, at 00:50, "Don Moir" <donmoir at comcast.net> wrote:
>>
>>> ----- Original Message ----- From: "compn" <tempn at twmi.rr.com>
>>> To: <ffmpeg-devel at ffmpeg.org>
>>> Sent: Thursday, September 19, 2013 6:26 PM
>>> Subject: Re: [FFmpeg-devel] [PATCH] Make decoding alpha optional for
>some codecs.
>>>
>>>
>>>> On Thu, 19 Sep 2013 18:05:21 -0400, Don Moir wrote:
>>>>> You could get lucky and some might look ok, but then alpha can
>give a smooth anti-aliasing effect and that would be lost. So 
>>>>> effort
>>>>> is add clutter to save some CPU cycles at the possible expense of
>a bad display.
>>>>
>>>> since ffplay doesnt handle alpha now, do your videos look bad in
>ffplay?
>>>> i mean, not decoding alpha vs decoding alpha, does it make any
>>>> difference when you dont display alpha anyways?
>>>
>>> I pretty much don't use ffplay for anything. In my own app, alpha
>makes a great deal of difference, used as overlays and even 
>>> when video is over black, the smoothness with alpha is very
>apparent. It's like take a alpha png and remove the alpha. Looks bad. 
>>> stair stepped
>>
>> I just realize there is a possible issue with the whole argument, and
>if you mostly look at intermediate data you might get the 
>> wrong impression.
>> You assume that the color values need to be multiplied with alpha.
>> However except for intermediates that makes no sense.
>> The color values should be pre-multiplied with alpha after
>processing, since it makes the features more natural and avoid sharp 
>> edges, which improves visual quality. It also means you will not
>waste bandwidth storing data that will be blended away anyway, 
>> and it saves you one multiplication when displaying.
>> In this case, just discarding alpha will actually display an image
>that looks exactly like against black background.
>> Now the interesting question would be whether flashplayer does it
>like that, which would explain why I could never actually see 
>> these issues...
>
>Yeah I was thinking that... But I don't think you could actually know
>on a general basis. All depends on how it might be encoded 
>given various formats.

Yes, and it looks like it might be quite reliable to guess based on codec. At least for vp6a.

> It could be that it is pretty much assumed some
>would not display the alpha and so maybe pre-multiplied 

That however is exactly not the reason for flash. They know quite sure alpha will be used. And I suspect the saved multiplication isn't that important either.
The real reason is that since video codecs try to emulate human vision and alpha is processed completely separately it is critical the encoder gets data as close to the final image as possible.
Otherwise ringing effects of a hard black to write border inside a transparent region would cause it to have quite visible effects on a nearby opaque area, as just one example of the possible issues.
And as I mentioned you effectively get partial HDR support for free for the colour part (HDR meaning colour values > 1, which for pre-multiplied alpha just translates to colour values between alpha and 1 which can be encoded without issues).