[FFmpeg-devel] A patch to fix buffer overflow when decoding h264

[Michael Niedermayer]

On Fri, May 28, 2010 at 03:10:12PM +0300, Antti Nietosvaara wrote:
> Michael Niedermayer wrote:
>> On Wed, May 26, 2010 at 03:34:38PM +0300, Antti Nietosvaara wrote:
>>   
>>> I was experiencing crashes when decoding certain h264 videos 
>>> (unfortunately it is quite hard to extract the problematic stream for 
>>> replication, since its in proprietary DVR format).
>>> It seems that s->mb_height can change in decode_slice_header after 
>>> alloc_tables has been called for the current context, which causes 
>>> overflows later. Hopefully this behaviour can be confirmed without a 
>>> sample stream.
>>> I have attached a patch that reallocates the tables if mb_width or 
>>> mb_height change.
>>>     
>>
>> what is changing mb_height without changing height?
>>
>>
>> [...]
>>   ------------------------------------------------------------------------
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at mplayerhq.hu
>> https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-devel
> I dug a little deeper and I may have found a reason for the crash on our 
> software. Before decompressing the frame I set AVCodecContext's width and 
> height to values that the frame should have been compressed to. This seems 
> to end up crashing the program later on.
> I suppose altering AVCodecContext::width and height outside libavcodec is 
> not using the library as intended, and as such, this patch is probably 
> useless.

yes, if this can only occur if a application messes with width/height after
init/open() then its an application bug.


> If you are interested in replicating the crash anyway, I could slap 
> together a small C program that does just that.

not needed


[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In a rich man's house there is no place to spit but his face.
-- Diogenes of Sinope
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100528/94c02243/attachment.pgp>