[FFmpeg-devel] [PATCH 1/2] avcodec/pictordec: Optimize more odd cases for picmemset()

[Peter Ross]

On Tue, Aug 31, 2021 at 08:33:41PM +0200, Michael Niedermayer wrote:
> Fixes: Timeout
> Fixes: 36875/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PICTOR_fuzzer-4842960888922112
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> ---
>  libavcodec/pictordec.c | 5 ++---
>  1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/pictordec.c b/libavcodec/pictordec.c
> index c37c5e7106..c7b3a335d3 100644
> --- a/libavcodec/pictordec.c
> +++ b/libavcodec/pictordec.c
> @@ -89,8 +89,7 @@ static void picmemset(PicContext *s, AVFrame *frame, unsigned value, int run,
>                  d = frame->data[0] + yl * frame->linesize[0];
>                  if (s->nb_planes == 1 &&
>                      run*pixels_per_value >= s->width &&
> -                    pixels_per_value < s->width &&
> -                    s->width % pixels_per_value == 0
> +                    pixels_per_value < (s->width / pixels_per_value * pixels_per_value)
>                      ) {
>                      for (; xl < pixels_per_value; xl ++) {
>                          j = (j < bits_per_plane ? 8 : j) - bits_per_plane;
> @@ -98,7 +97,7 @@ static void picmemset(PicContext *s, AVFrame *frame, unsigned value, int run,
>                      }
>                      av_memcpy_backptr(d+xl, pixels_per_value, s->width - xl);
>                      run -= s->width / pixels_per_value;
> -                    xl = s->width;
> +                    xl = s->width / pixels_per_value * pixels_per_value;
>                  }
>              }
>          }
> -- 
> 2.17.1

i have tested this against variety of samples. please apply.

-- Peter
(A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20210901/8d6372db/attachment.sig>