[FFmpeg-devel] [PATCH 5/5] avcodec/flacdsp: Fix: runtime error: signed integer overflow: -1027555328 + -1226681270 cannot be represented in type 'int'

[Paul B Mahol]

On 2/25/17, Michael Niedermayer <michael at niedermayer.cc> wrote:
> On Sat, Feb 25, 2017 at 10:00:36PM +0100, Paul B Mahol wrote:
>> On 2/25/17, Michael Niedermayer <michael at niedermayer.cc> wrote:
>> > Fixes: 673/clusterfuzz-testcase-5948736536576000
>> >
>> > Found-by: continuous fuzzing process
>> > https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
>> > Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
>> > ---
>> >  libavcodec/flacdsp.c | 2 +-
>> >  1 file changed, 1 insertion(+), 1 deletion(-)
>> >
>> > diff --git a/libavcodec/flacdsp.c b/libavcodec/flacdsp.c
>> > index 560091f73a..bc9a5dbed9 100644
>> > --- a/libavcodec/flacdsp.c
>> > +++ b/libavcodec/flacdsp.c
>> > @@ -67,7 +67,7 @@ static void flac_lpc_16_c(int32_t *decoded, const int
>> > coeffs[32],
>> >          int sum = 0;
>> >          for (j = 0; j < pred_order; j++)
>> >              sum += coeffs[j] * (SUINT)decoded[j];
>> > -        decoded[j] += sum >> qlevel;
>> > +        decoded[j] = decoded[j] + (unsigned)(sum >> qlevel);
>> >      }
>> >  }
>> >
>>
>> What about making sum unsigned instead?
>
> in "sum >> qlevel" sum needs to be signed to get the correct
> result
>
> I can also add a check for the overflow and return an error but thats
> more code

no need, this is ok