[FFmpeg-devel] [PATCH 2/4] avcodec/apedec: properly calculate and store absolute value

[Paul B Mahol]

On Tue, Oct 06, 2020 at 09:53:44AM +0200, Anton Khirnov wrote:
> Quoting Paul B Mahol (2020-10-06 02:17:12)
> > Signed-off-by: Paul B Mahol <onemda at gmail.com>
> > ---
> >  libavcodec/apedec.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
> > index 273abe2490..aa4d8fa524 100644
> > --- a/libavcodec/apedec.c
> > +++ b/libavcodec/apedec.c
> > @@ -1311,7 +1311,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f,
> >                              int32_t *data, int count, int order, int fracbits)
> >  {
> >      int res;
> > -    int absres;
> > +    unsigned absres;
> 
> Does anything other than the type change?

absres value should change in single case when -INT32_MIN is stored again in int.
Reference implementation use int64_t type instead.