[FFmpeg-devel] [PATCH] aac_fixed: fix overflow in sbr_sum_square_c

[Andreas Cadhalpun]

On 08.11.2015 20:17, Michael Niedermayer wrote:
> On Sun, Nov 08, 2015 at 05:14:10PM +0100, Andreas Cadhalpun wrote:
>> If accu overflows, a negative value can be returned.
>>
>> Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com>
>> ---
>>  libavcodec/sbrdsp_fixed.c | 28 ++++++++++++++++++++++------
>>  1 file changed, 22 insertions(+), 6 deletions(-)
> 
> CCing the authors of this, they are probably interrested in
> commenting

Thanks.

> but the patch does not look like an optimal solution

It's certainly not pretty, but it fixes the crashes/assertion failures.

> also does anyone known if values large enough to cause overflows
> are alowed in valid AAC ? (didnt investigate yet, just asking as
> someone might know ...)

I don't know either, but it would be strange if that's invalid,
as e.g. the float aac decoder handles this just fine.

> also an additional bit of precission can be use by making the
> variable unsigned

One bit is probably not sufficient, as dozens of arbitrary integers
are squared and added together.
Making the variable unsigned would effectively hide the overflows,
though.

Best regards,
Andreas