[FFmpeg-devel] [RFC v2 3/3] daaladec: Implement a native Daala decoder

[Ronald S. Bultje]

Hi,

On Tue, Dec 29, 2015 at 10:55 AM, Andreas Cadhalpun <
andreas.cadhalpun at googlemail.com> wrote:

> > +static av_always_inline void idct_1D_8(pixel *x, int xstride, const
> pixel y[16])
> > +{
> > +    int t1h, t4h, t6h, t0 = y[0], t1 = y[1], t2 = y[2], t3 = y[3], t4 =
> y[4];
> > +    int t5 = y[5], t6 = y[6], t7 = y[7];
> > +    t5 -= (t3*2485 + 4096) >> 13;
> > +    t3 += (t5*18205 + 16384) >> 15;
> > +    t5 -= (t3*2485 + 4096) >> 13;
> > +    t7 -= (t1*3227 + 16384) >> 15;
> > +    t1 += (t7*6393 + 16384) >> 15;
> > +    t7 -= (t1*3227 + 16384) >> 15;
> > +    t1 += t3;
>
> These seven lines can overflow.


Why do you believe they can overflow? Look at range constraints for the
input values.

Ronald