[FFmpeg-devel] [PATCH]Support 64bit RGB input

[Carl Eugen Hoyos]

Michael Niedermayer <michaelni <at> gmx.at> writes:

> On Sat, Mar 03, 2012 at 11:22:13PM +0000, Carl Eugen Hoyos wrote:
> > Michael Niedermayer <michaelni <at> gmx.at> writes:
> > 
> > > > > > Attached patch allows to convert 64bit RGB input.
> > > > > 
> > > > > LGTM

Applied since I assume the remaining problems are not directly 
64bit related and conversion to yuva420 works fine.

> > > > Conversion to RGBA produces (nearly) transparent files 
> > > > because A1 and A2 overflow in yuv2rgb_1_c_template if 
> > > > input is 0xFF, patch follows inlined:
> > > 
> > > why do they overflow ? is there some incorrect scaling applied
> > > somewhere ?
> > 
> > rgba64ToA_c() in input.c reads 65535 (=completely opaque), not 
> > 0xFF as written above, as alpha value for (the lower part of) 
> > the sample from ticket #503, hyscale() in swscale.c calls 
> > c->hyScale() which changes the value to 32767, 
> > yuv2rgb_1_c_template() in output.c calculates A1 and A2 
> > as 0x100 ((32767+64)/2^7) which is then written as "0" 
> > (=completely transparent) if the output is RGBA.
> 
> something like
> *255 + 16384 >> 15
> 
> should work better

I can confirm that the following inlined patch fixes the problem 
with RGBA64 -> RGBA conversion:

diff --git a/libswscale/output.c b/libswscale/output.c
index cae2c31..de41f77 100644
--- a/libswscale/output.c
+++ b/libswscale/output.c
@@ -1040,7 +1040,7 @@ yuv2rgb_1_c_template
                        *b =  c->table_bU[U + YUVRGB_TABLE_HEADROOM];

             if (hasAlpha) {
-                A1 = (abuf0[i * 2    ] + 64) >> 7;
-                A2 = (abuf0[i * 2 + 1] + 64) >> 7;
+                A1 = abuf0[i * 2    ] * 255 + 16384 >> 15;
+                A2 = abuf0[i * 2 + 1] * 255 + 16384 >> 15;
             }



Thank you, Carl Eugen