[FFmpeg-devel] [PATCH 1/4] avcodec/dirac_dwt_template: Fix signedness regression in interleave()

[James Darnley]

On 2018-07-13 19:26, Michael Niedermayer wrote:
> Found-by: <jdarnley>
> 
> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> ---
>  libavcodec/dirac_dwt_template.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/dirac_dwt_template.c b/libavcodec/dirac_dwt_template.c
> index 528fc7e9e7..2369c8d15b 100644
> --- a/libavcodec/dirac_dwt_template.c
> +++ b/libavcodec/dirac_dwt_template.c
> @@ -57,8 +57,8 @@ static av_always_inline void RENAME(interleave)(TYPE *dst, TYPE *src0, TYPE *src
>  {
>      int i;
>      for (i = 0; i < w2; i++) {
> -        dst[2*i  ] = (src0[i] + (unsigned)add) >> shift;
> -        dst[2*i+1] = (src1[i] + (unsigned)add) >> shift;
> +        dst[2*i  ] = ((int)(src0[i] + (unsigned)add)) >> shift;
> +        dst[2*i+1] = ((int)(src1[i] + (unsigned)add)) >> shift;
>      }
>  }
>  
> 

This does fix the failure I saw.  I will submit a patch for more fate
tests which happen to cover this piece of code.