[FFmpeg-devel] PATCH Fix SSE core-dumps when decoding AAC on optimized ffmpeg with shared libraries compiled under Mingw (win32)

[Art Clarke]

Hi folks,

Attached is a patch that fixes a bug where ACC decoding fails with the
new decoder.  To reproduce, build ffmpeg in SHARED library mode on
Mingw with the latest runtime and win32 library, with all
optimizations turned on, and then try to decode an aac file on a
machine with the SSE2 instruction set available (how's that for
esoteric).

ffmpeg -i aacfile.aac -acodec pcm_mulaw -y out.wav

The problem is that the GCC compiler on mingw doesn't correctly align
uninitialized local variables in mingw.  This causes some of the SSE
optimizations under the new AAC decoder to work on non-64-bit aligned
local variables and badness occurs.

The fix is to pass "-fno-common" to the GCC compiler under mingw if
mmx2 is enabled.  This allows assembly that tries to align
uninitialized local variables on 64 bit boundaries at the cost of not
allowing you to define an uninitialized variable with extern linkage
more than once (without specifying at all but one instances as extern
-- something ffmpeg already enforces)*.

Major major kudos to Gianluigi Tiesi for figuring this out with the
GCC team.  He's been working on getting mplayer to compile on Mingw.

See here for the details on the GCC bug and "-fno-common" workaround:
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=37216

- Art

* it also moves where the uninitialized variables live from the common
section to the .bss section, and gcc can remember alignment of
non-common sections, so that's the nature of the fix.
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: diff.txt
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20081022/8c700715/attachment.txt>