[FFmpeg-devel] [PATCH 2/2] avcodec/pixlet: Fixes: undefined shift in av_mod_uintp2()

[Paul B Mahol]

On 8/18/17, Michael Niedermayer <michael at niedermayer.cc> wrote:
> Fixes: runtime error: shift exponent 4294967289 is too large for 32-bit type
> 'int'
> Fixes: 3030/clusterfuzz-testcase-minimized-4649809254285312
>
> Found-by: continuous fuzzing process
> https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> ---
>  libavcodec/pixlet.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavcodec/pixlet.c b/libavcodec/pixlet.c
> index 088226bdda..a9cfe085c9 100644
> --- a/libavcodec/pixlet.c
> +++ b/libavcodec/pixlet.c
> @@ -262,7 +262,7 @@ static int read_high_coeffs(AVCodecContext *avctx,
> uint8_t *src, int16_t *dst, i
>
>          flag = 0;
>
> -        if (state * 4ULL > 0xFF || i >= size)
> +        if ((uint64_t)state > 0xFF / 4 || i >= size)

This is not exact same behaviour.

>              continue;
>
>          pfx = ((state + 8) >> 5) + (state ? ff_clz(state): 32) - 24;
> --
> 2.14.1
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>