[FFmpeg-soc] [PATCH] updated! vf_overlay alpha patch and watermarking using PNG with alpha

[Baptiste Coudurier]

Hi,

On 05/12/2010 02:33 PM, Michael Niedermayer wrote:
> On Wed, May 12, 2010 at 09:00:46AM +0200, Vitor Sessak wrote:
>> Baptiste Coudurier wrote:
>>> On 12/06/2009 08:12 AM, Vitor Sessak wrote:
>>>> Hi and sorry for the delay.
>>>>
>>>> Artur Bodera wrote:
>>>>> On Tue, Dec 1, 2009 at 12:50 AM, Vitor Sessak<vitor1001 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> While I normally oppose making non-committed code more complex, I think
>>>>>> this feature is so often requested that it is worth the extra work in
>>>>>> the
>>>>>> future. Stefano, Michael, any strong opinion about this?
>>>>>>
>>>>>
>>>>> I think the vf_overlay should be modified altogether. Although
>>>>> mathematically alpha-blending is more expensive than opaque pixel
>>>>> replacement, I think that it should be automatically decided by
>>>>> analyzing
>>>>> the overlay format.
>>>>>
>>>>> So the alpha-blending should be a "built-in" functionality (not a
>>>>> switchable
>>>>> parameter) and should be implicitly functional with any overlay
>>>>> stream/image
>>>>> that has alpha channel (i.e. rgba). If there is no alpha channel, then
>>>>> pixel
>>>>> overriding would be used. This makes much more sense.
>>>>
>>>> I agree that this would be nice, but there is no way to make it work
>>>> with the current format negotiation in libavfilter. For example, there
>>>> is no way to have a filter that accepts either "input: rgb, output rgba"
>>>> or "input: yuv, output: yuva", so I suggest you just do as your present
>>>> patch for the time been.
>>> How much harm does doing yuv ->  yuva or rgb ->  rgba in all cases ?
>>> To my knowledge it would only be a matter of adding one component and
>>> memset it to 0xff.
>>
>> This wouldn't do much harm, but there is no way of asking for such kind of
>> inputs in lavfi (i.e., either "rgb,rgba" or "yuv,yuva"). And changing lavfi
>> to accept such constraints would probably make colospace negotiation a NP
>> problem.
>
> finding a solution with a minimum of convertion could become NP, finding any
> solution wont
>
> heres a quick option (i do not know how good / bad this suggestion is,
> if someone like loren could look at this before its implemented this
> would probably be a good idea)
>
> 1. we extend query_format to query_format(int alternative) where alternative
> selects which of several (small number normally and for most filters just 1)
> alternative colorspace lists each link supports.
> Euch such alternative is like the current case that is 2 links either must
> have the same list of possible pixelformats/colorspaces or they are
> independant.
>
> 2. we run the current avfiltergraph.c query_formats() over the graph skiping
> all filters that have alternatives.
>
> 3. we go over all filters that have alternatives
>     for each such filter we find the 2 best alterantives, best based on
>     score= (A<<32) + B;
>     A= the number of scale filters the specific alterantive would require
>     B= the number of degrees of freedom lost (aka fewer colorspaces)
>     we then calculate a score2 that is the difference of the scores of
>     the best 2 alterantives.
>     And last lock in filter alterantives iteratively based on best score2
>
> The idea here is that we lock in the filter with the most clearly supperior
> alterantive first. That is if we have a filter that has 2 alterantives and
> neither on its own would require an additional convertion filter and another
> filter that also has 2 alternatives and one of these would require 1
> convertion and the other case would require 2 convertions then the 1
> convertion alternative would be locked in and we would restart from the begin
>
> This is not optimal, not even for a simple linear filter chain with 2 filters.
> But it might be an acceptable heuristic for real filter graphs.
> For a single linear chain its possible to find the optimal case with
> viterbi
>
> Another method would be to only use avfiltergraph.c query_formats() and
> consider all the alternative values inputs to it and the number of convertions
> and degrees of freedom left over the whole graph its score. And then use some
> generic optimization technique that minimizes this score.
>
> btw, off topic a little but simply favoring the first colorspace in each
> list like its done in svn is not a good idea, it likely would make more
> sense to favor a colorspace that preserves all information that is there
> and is simple. that is if either upstream or downstream is greyscale we
> shouldnt select yuv similar issues exist with 16/8 bit and rgb/yuv and alpha
>

Question, why not using a pull paradigm from the output filter ?
That certainly gives an hint on what the user want at the end.
This would allow the overlay filter to request RGBA as input when output 
is RGB, same for YUVA/YUV

-- 
Baptiste COUDURIER
Key fingerprint                 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA
FFmpeg maintainer                                  http://www.ffmpeg.org