[FFmpeg-devel] [PATCH] VC-1 MMX DSP functions

[Zuxy Meng]

Hi,

2007/7/7, Christophe GISQUET <christophe.gisquet at free.fr>:
> infernix a ?crit :
> > Zuxy Meng wrote:
> >> I wonder if it's caused by "m"(sstr) and things like that. Maybe it
> >> can be corrected by using "g"((intptr_t)sstr).
> >
> > IANAC, but i applied:
> >
> > sed -i 's|\([SDrmRg]" *((\)long)|\1intptr_t)|g' libavcodec/i386/*.[ch]
> >
> > and changed libavcodec/i386/vc1dsp_mmx.c line 252-253 to:
> >
> >          : "g"((intptr_t)sstr), "g"((intptr_t)dstr),
> > "r"((intptr_t)offset), "r"((intptr_t)3*offset), \
> >            "g"((intptr_t)rnd)
>
> The crash occurs in vc1_put_shift2_mmx, so line
>        : "m"(src_stride), "m"(dst_stride),
> becomes:
>        : "m"((intptr_t)src_stride), "m"((intptr_t)dst_stride),

"m" can't be used here since (intptr_t)src_stride is no longer
directly addressable.

> But anyway, it still sound strange:
> rax is 3*offset and rcx is offset, harmless
>
> In the backtrace, only the value of rdx seems to be able to cause a
> crash. And its value is that of src, that gdb warns as:
> src=0x2ab2aaefdfee <Address 0x2ab2aaefdfee out of bounds>
>
> Same for dst, that we can trace back to vc1_decode_p_blocks. I'm not
> sure how to interpret this, but that does sound bad.
>
> I have to admit I'm clumsy with gas contraints (though the
> put_no_rnd_h264 was a particularly bad example), so I would rather
> suspect that (src register being kept across function calls) causing the
> problem.

-- 
Zuxy
Beauty is truth,
While truth is beauty.
PGP KeyID: E8555ED6