[MPlayer-dev-eng] altivec_yuv2packedX (again. found the problem)

[Michael Niedermayer]

Hi

On Wednesday 13 July 2005 18:59, Andrea Palmatè wrote:
> Hi dear!
>
> I have another problem with damned altivec. I have configured the
> output video driver to scale the image. Mplayer use an optimized
> altivec scale function. Now. when i use this function i receive a
> great GR in an altivec function with a "alignment" crash type. I'm
> not too expert of altivec and alignment but i have compiled mplayer
> with debug and i have found the line that crash. The function is:
>
> void
> altivec_yuv2packedX (SwsContext *c,
>                         int16_t *lumFilter, int16_t **lumSrc, int
> lumFilterSize,
>                         int16_t *chrFilter, int16_t **chrSrc, int
> chrFilterSize,
>                         uint8_t *dest, int dstW, int dstY)
> {
> [...]snip[...]
>
>      cvtyuvtoRGB (c, Y0,U0,V0,&R0,&G0,&B0);
>      cvtyuvtoRGB (c, Y1,U1,V1,&R1,&G1,&B1);
>
>      R  = vec_packclp (R0,R1);
>      G  = vec_packclp (G0,G1);
>      B  = vec_packclp (B0,B1);
>
>      out_rgba (R,G,B,out);
>      ^^^^^^^^^
> and it crash here. Most probably for the wrong "out" parameter.
>
> the dump is:
> >R:804CC5C0 G:00004140 B:00000018 out:00004190
>
> so it seems that R and out are swapped??

have these been obtained with printf or gdb? i wouldnt trust gdb, not to 
mention R,G and B being altivec vectors not pointers

a few general questions ...
which gcc version
does it happen just with your VO or does it also happen if the same 
conversation is done by vf_scale?
does compiling with -O0 or -O1 help?
ahh and the arrays are aligned correctly right? they need to be ptr%16==0 and 
stride %16==0

[...]
-- 
Michael