[FFmpeg-devel] [PATCH] prevent buffer overflow with large a/mulaw frames

[Peter Ross]

On Sat, Jul 25, 2009 at 11:55:09PM -0700, Baptiste Coudurier wrote:
> On 07/25/2009 11:45 PM, Peter Ross wrote:
>> On Sun, Jul 26, 2009 at 03:32:59PM +1000, Peter Ross wrote:
>>> On Sat, Jul 25, 2009 at 09:42:52PM -0700, Baptiste Coudurier wrote:
>>>> Hi Peter,
>>>>
>>>> On 07/25/2009 09:19 PM, Peter Ross wrote:
>>>>> Hi,
>>>>>
>>>>> This patch prevents alaw/mulaw decoders from writing beyond the output buffer.
>>>>>

(snip)

>> +        audio_out_size = FFMAX(size, 4*MAX_AUDIO_PACKET_SIZE);
>> +        audio_out = av_realloc(audio_out, audio_out_size);
>> +    }
>>       if (!audio_buf || !audio_out)
>>           return;               /* Should signal an error ! */
>>
>> @@ -596,9 +599,11 @@
>>   #define MAKE_SFMT_PAIR(a,b) ((a)+SAMPLE_FMT_NB*(b))
>>       if (!ost->audio_resample&&  dec->sample_fmt!=enc->sample_fmt&&
>>           MAKE_SFMT_PAIR(enc->sample_fmt,dec->sample_fmt)!=ost->reformat_pair) {
>> +        if (!audio_out2 || size>  audio_out2_size) {
>> +            audio_out2_size = FFMAX(size, 4*MAX_AUDIO_PACKET_SIZE);
>> +            audio_out2 = av_realloc(audio_out2, audio_out2_size);
>> +        }
>
> av_fast_realloc may look cleaner in this situation.

Revised patch enclosed. I have overhualed the audio_{buf,out,out2} allocations,
to prevent overflows for each phase of do_audio_out().

-- Peter
(A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ffmpeg-audiooutsize-r2.diff
Type: text/x-diff
Size: 3880 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20090808/06a645f8/attachment.diff>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20090808/06a645f8/attachment.pgp>