[FFmpeg-devel] [PATCH] *alloc(type)

[Yuriy Kaminskiy]

Reimar D?ffinger wrote:
> On Sat, Nov 20, 2010 at 04:37:30AM +0100, Michael Niedermayer wrote:
>> patchset below fixes the type used in malloc and co
>> The sense behind this patch is that feeding things that dont fit in unsigned
>> int into *alloc() can lead to successfull allocation of too small arrays which
>> is pretty bad.
>> There are probably more functions that should be changed like av_new_packet()
>> but i had to start somewhere and will look into the others too if noone else
>> does.
>> Note, i will apply this in a few days if there are no objections
> 
> This has some side-effects I do not like.
> For example, allocating more than 4 GB now becomes possible, even
> though such an allocation is almost certain to be a bug.
No. A bit more context:
=== cut ===
void *av_malloc(unsigned int size)
{
    void *ptr = NULL;
#if CONFIG_MEMALIGN_HACK
    long diff;
#endif

    /* let's disallow possible ambiguous cases */
    if(size > (INT_MAX-16) )
        return NULL;
=== cut ===
And this check actually protect your ass in some cases, where otherwise would be
possible exploitable integer overflow (e.g. in matroska demuxer there are few
places that only *implicitly* protected by this check and *cast from int to
unsigned* when calling av_malloc).