[MPlayer-dev-eng] [patch] backports/fixes from uoti demux_mkv.c for comp_algo==3

[Reimar Döffinger]

On Sat, Aug 14, 2010 at 03:38:24AM +0400, Yuriy Kaminskiy wrote:
> PS By the way, I quickly looked at lavf matroska demuxer, EXACTLY SAME integer
> overflow problems present there...

I don't see that. I think there is one, but at least
1) It is not so extremely obvious as the one in the first
   patch
2) It can only happen if a malloc of > 1.3 GB succeeds before

This I think should fix it:
Index: libavformat/matroskadec.c
===================================================================
--- libavformat/matroskadec.c   (revision 24788)
+++ libavformat/matroskadec.c   (working copy)
@@ -901,6 +901,9 @@
     int result = 0;
     int olen;
 
+    if (pkt_size >= 10000000)
+        return -1;
+
     switch (encodings[0].compression.algo) {
     case MATROSKA_TRACK_ENCODING_COMP_HEADERSTRIP:
         return encodings[0].compression.settings.size;