[FFmpeg-devel] [PATCH] fix h264_deblock_sse2.asm segfaults on clang/x86-32

Ronald S. Bultje rsbultje
Wed Sep 1 01:06:09 CEST 2010


on fate/clang/freebsd/x86-32 [1], all h264 tests fail with a sigbus.
There's some alignment issues in the sse2 asm. The asm appears to
assume that the caller function guarantees a certain alignment, which
is (IMO) not necessarily true. If the function is called directly from
C, stack can have any alignment, depending on the compiler used, so
the functions shouldn't assume any specific alignment imo.

Attached patch tries to make it work with any alignment. x86-64
h_luma_intra wasn't fixed because it doesn't show any obvious issues
and any attempt of mine to fix it bombed out, probably somewhere in
the macros which do way much. This code is scary witchcraft, almost as
crazy as inline asm. With this patch, most sigbus issues are gone and
"make fate-h264" goes well until about halfway
(h264-conformance-caba2_sony_e), where it crashes somewhere in
decode_cabac_mb_mvd() which I haven't quite traced down yet.

(Related, how do I run make fate-xyz without it quitting as soon as a
single test fails? I'd like output as on fate.ffmpeg.org so I can see
how many failures I fixed with this patch.)


[1] http://fate.ffmpeg.org/x86_32-freebsd-clang/20100831203328
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix_clang_x86_32_fbsd.patch
Type: application/octet-stream
Size: 6943 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100831/7fe28fa3/attachment.obj>

More information about the ffmpeg-devel mailing list