[MPlayer-dev-eng] [PATCH] demux_mkv.c: fix crash with gcc 4.4 and optimizations

[Corey Hickey]

Hello,

The MKV demuxer segfaults when compiled with gcc 4.4.

Steps to reproduce:

$ ./configure --cc=gcc-4.4
$ make
$ wget http://samples.mplayerhq.hu/Matroska/theora.mkv
$ ./mplayer theora.mkv

This has been reported before; the most recent discussion seems to be:
http://bugs.gentoo.org/269975

The segfault goes away when certain optimizations are disabled (such as
when compiling with './configure --enable-debug').

I narrowed down the set of necessary optimization flags and asked on the
gcc-help mailing list.

http://gcc.gnu.org/ml/gcc-help/2009-09/msg00277.html

Andrew Haley pointed out that the crash was probably due to an aliasing
bug, and I was able to find it with -Wstrict-aliasing=2.

http://gcc.gnu.org/ml/gcc-help/2009-09/msg00299.html

Ian Lance Taylor explained the problem nicely:

http://gcc.gnu.org/ml/gcc-help/2009-09/msg00301.html


The attached patch fixes the segfault for me. I'm not absolutely certain
that I've found the bug aside from Ian's indication that it looked
right. Also, there's probably a better way to fix the bug, without code
duplication, but I don't know what else to do. Therefore I submit the
patch mostly as an optimistic proof-of-concept, and I don't threaten to
apply it (if my svn account is even still active).

Thanks,
Corey
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: no_pointer_casting.diff
URL: <http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/attachments/20090929/6a4d3833/attachment.asc>