[Ffmpeg-devel] SSE2 Crashing

[Marco Manfredini]

On Thursday 17 August 2006 05:13, Brian Brice wrote:
> Hi!
> I'm not sure if this is the right place to bring this up, but maybe
> others have experience with this issue.  I've compiled ffmpeg with
> gcc-3.4.5/ld-2.16-91 in mingw.  It seems to work correctly in the
> ffmpeg program, but when I use the libraries with something else,
> then I experience crashes involving SSE2 instructions.  The crashes
> were occurring in lavc/i386/fdct_mmx.c, fdct_row_sse2.  I do have
> SSE and the memalign hack enabled.  The function that calls
> fdct_row_sse2 is ff_fdct_sse2 allocates a buffer that is aligned
> to to an 8 byte boundary.  I've watched the allocation of this
> buffer and it seems to not actually be aligned correctly.  Is this
> a problem with my version of gcc?  Anyone else experience this and
> figure out a way around it?  Thanks.

This happened with Mactel here on various occasions too. I found that, in my 
cases, ff_fdct_sse2 got blocks from locals that were attributed 
DECLARE_ALIGNED_8(t,v). For some reason DECLARE_ALIGNED_8 got defined in 
dsputil.h as "__attribute__ ((aligned (8)))" and not "__attribute__ ((aligned 
(16)))" as it should (I failed to understand the logic behind 
DECLARE_ALIGN_{8,16} though :-) ).

I helped myself by replacing 16 for 8 in the first definition of 
DECLARE_ALIGNED_8 and STRIDE_ALIGN in dsputil.h

Marco