[FFmpeg-devel] H263 decoding crash, [BUG] : reading memory past the end of the buffer.

[Pavel Pavlov]

> Hello everybody,
> I have some random crashes while decoding h263.
> 
> I use icl build of ffmpeg and can debug with visual studio. 
> The problem is that I only get crashes in release build, and 
> only if I'm not attached with the debugger.
> I always get it in the same function after running a video 
> call for a while.
> Once I attach with the debugger after it crashed some of the 
> variables aren't showing correctly, so I can't really see 
> what's the problem.
> 
> For example, it crashes inside h263.c on line 4190
> 
> Here's the stack trace:
> 
>  	avcodec-52.dll!ff_h263_decode_mb(MpegEncContext * s=0x01068c10,
> short [64]* block=0x06ce48c0)  Line 4190 + 0x11 bytes	C
>  	avcodec-52.dll!decode_slice(MpegEncContext * s=0x00000000)  Line
> 221 + 0x7 bytes	C
>  	avcodec-52.dll!ff_h263_decode_frame(AVCodecContext * 
> avctx=0x01068280, void * data=0x01068620, int * data_size=0x015af8d8,
> AVPacket * avpkt=0x015af830)  Line 653	C
>  	avcodec-52.dll!avcodec_decode_video(AVCodecContext * 
> avctx=0x00000000, AVFrame * picture=0x015af8f8, int * 
> got_picture_ptr=0x015afcc4, const unsigned char * buf=0x005ac98a, int
> buf_size=17203840)  Line 563 + 0x6c bytes	C
> >	EC-09.exe!H263DecoderStruct::Transform(void * pData=0x07aa5954,
> int nSize=1706, SAL::MediaSample & Sample={...}, 
> SAL::SmartPointer<SAL::MediaHandler> p={...})  Line 459 + 0x20 bytes
> C++
> 
> 
> stack trace doesn't make much sence, as my code calls 
> avcodec_decode_video(0x01068620, 0x01068620, &nResult, 
> 0x07aa5954, 1706); but on the next line it shows 
> AVCodecContext * avctx=0x00000000
> 
> Just before ff_h263_decode_mb is called, in file 
> h263dec.c:221 inside function decode_slice there is a 
> commented out printf:
> //            s->mb_skipped = 0;
> //printf("%d %d %06X\n", ret, get_bits_count(&s->gb), 
> show_bits(&s->gb, 24)); Was someone getting some sort of 
> problems and that's why there is this printf to trace?


Before you start reading below, I start with my conclusion: 
there is code that always reads past the end of the buffer when
decoding h263.


I was able to get more info

show_bits (h263.c:4190) is called with GetBitsContext that contains
these values:
buffer : 0x017dbf4c     (25018188)
buffer_end : 0x017dcffd (25022461)
index : 34184
size_in_bits : 34184

And the exceptions I get is:
Unhandled exception at 0x10180e06 (avcodec-52.dll) in EC-09.exe: 
0xC0000005: Access violation reading location 0x017dd000.

Location 0x017dd000 (25022464) is just 3 bytes outside of 
passed GetBitsContext.buffer_end

The place of crash has this asm code:
        int v= show_bits(&s->gb, 16);
10180DF5  mov         ebx,dword ptr [esp+0B4h] 
10180DFC  mov         edx,dword ptr [ebx+8] 
10180DFF  mov         eax,dword ptr [ebx] 
10180E01  mov         edi,edx 
10180E03  sar         edi,3 
10180E06  mov         eax,dword ptr [edi+eax]    
10180E09  bswap       eax  
10180E0B  mov         ecx,edx 
10180E0D  and         ecx,7 
10180E10  shl         eax,cl 
10180E12  shr         eax,0F0h 

And the crash is at 10180e06



The values passed to the show_bits as buffer and buffer_end is the
pointer 
to the h263 video data that I pass to the decoder:
avcodec_decode_video(m_pContext, m_pFrame, &nResult, (uint8_t*)pData,
nSize);
where pData is 0x017dbf4c and nSize is 4273 (which is 34184/8)

At the point of crash EDI is 000010B1 and eax is : 017DBF4C which means
that edi cotnains 4273 and eax is the passed buffer, so on line 10180E06

it tries to read past the buffer.


On the encoding side I use the same version of ffmpeg and the same
client.
The question I have - is it a bug that it tries to read past the end of
the buffer
or it's my fault that I don't pass some sort of multiple of 8 bytes or
something 
like that?



On top of that, I can confirm that this read happens 100% all the time.
I addded hardware breakpoint at the end the the passed h263 data and it 
always asserts. Basically, I tell cpu to int 1 if any instruction tries
to access 
memory after the end of the buffer and now I get assertion on the first
frame and 
it's not some random error. Moreover, since it's not a fatal error I get
assertion 
for reading past the end of the buffer in many places:

get_bits.h:628 

h263.c:3949
h263.c:3962
h263.c:3978
h263.c:4190
h263.c:4639
h263.c:4661

And hardware breakpoint on i386 is done through setting dr0-3 with the
address of the memory and setting flags for data read or write in dr7
http://en.wikipedia.org/wiki/Debug_register