[Ffmpeg-devel] [PATCH] ported SGI decoder to the new API

[Xiaohui Sun]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Michael Niedermayer wrote:
> Hi
> 
> On Wed, Apr 04, 2007 at 09:05:31AM +0800, Xiaohui Sun wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Michael Niedermayer wrote:
>>> Hi
>>>
>>>>> [...]
>>>>>> +            start_offset = bytestream_get_be32(&start_table);
>>>>>> +            if(start_offset > end_buf - in_buf) {
>>>>>> +                return AVERROR_INVALIDDATA;
>>>>>> +            }
>>>>> excelent, finally the check looks good
>>>>>
>>>>>
>>>>>> +            if (expand_rle_row(in_buf + start_offset, end_buf, 
>>>>> dest_row,
>>>>>> +                dest_row + s->linesize, z, s->depth) != s->width)
>>>>>> +                return AVERROR_INVALIDDATA;
>>>>> linesize can be negative
>>>>>
>>>> I modified, but not sure.
>>> it should have been dest_row + width*pixelsize or dest_row + FFABS(s->linesize)
>> I found width*pixelsize is not equal to(actually less than)
>> FFABS(s->linesize). because I found in avcodec_default_get_buffer it
>> will add some extra EDGE_WIDTH to it, is that right?
> 
> yes
It seems if we simply add a FFABS(s->linesize) to the s->linesize
without adjust pointer, then the end_buf is always less than the in_buf,
and the above check will cause an overflow:
...
    /* table size */
    unsigned int len = s->height * s->depth * 4;
...

    if(len * 2 > end_buf - in_buf) {
	return AVERROR_INVALIDDATA;
    }
That is the same when we decode uncompressed images.


[...]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGE4ZY+BwsLu3sVWwRApAMAJ9vb0dFQKT6h3Ola0+M2CpE5wVKkQCgpP2e
pBmx29Q3fQCp/UHGgSr2toY=
=mk8l
-----END PGP SIGNATURE-----