[FFmpeg-devel] [PATCH 1/7] avcodec/sunrast: Use ptrdiff_t for stride

[Andreas Rheinhardt]

Andreas Rheinhardt:
> Fixes segfaults with negative linesizes; in particular,
> this affected the sunraster-(1|8|24)bit-(raw|rle) and
> sunraster-8bit_gray-raw FATE tests.
> 
> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at outlook.com>
> ---
>  libavcodec/sunrast.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/sunrast.c b/libavcodec/sunrast.c
> index 77feef06e1..45b29e4d72 100644
> --- a/libavcodec/sunrast.c
> +++ b/libavcodec/sunrast.c
> @@ -31,7 +31,8 @@ static int sunrast_decode_frame(AVCodecContext *avctx, AVFrame *p,
>  {
>      const uint8_t *buf       = avpkt->data;
>      const uint8_t *buf_end   = avpkt->data + avpkt->size;
> -    unsigned int w, h, depth, type, maptype, maplength, stride, x, y, len, alen;
> +    unsigned int w, h, depth, type, maptype, maplength, x, y, len, alen;
> +    ptrdiff_t stride;
>      uint8_t *ptr, *ptr2 = NULL;
>      const uint8_t *bufstart = buf;
>      int ret;
> @@ -141,7 +142,7 @@ static int sunrast_decode_frame(AVCodecContext *avctx, AVFrame *p,
>  
>      if (type == RT_BYTE_ENCODED) {
>          int value, run;
> -        uint8_t *end = ptr + h * stride;
> +        uint8_t *end = ptr + (ptrdiff_t)h * stride;
>  
>          x = 0;
>          while (ptr != end && buf < buf_end) {

Will apply this patchset tonight unless there are objections.

- Andreas