[FFmpeg-devel] [PATCH] avutil/imgutils: use INT64_MAX for check stride and stride*(uint64_t)(h+128)

[Andreas Rheinhardt]

Steven Liu:
> look into the Picture size condition using INT_MAX is smaller,
> so make it to INT64_MAX maybe large enough for Picture.
> before patch:
> [mjpeg @ 0x619000004b80] [IMGUTILS @ 0x7ffee7e71210] Picture size 15748x23622 is invalid
> [image2 @ 0x616000000680] decoding for stream 0 failed
> [image2 @ 0x616000000680] Could not find codec parameters for stream 0 (Video: mjpeg (Baseline), none(bt470bg/unknown/unknown)): unspecified size
> Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
> Input #0, image2, from '/Users/liuqi/kuaishou/Test/docs/image.jpeg':
>   Duration: 00:00:00.04, start: 0.000000, bitrate: 8995854 kb/s
> 
> after patch:
> Input #0, image2, from '/Users/liuqi/kuaishou/Test/docs/image.jpeg':
>   Duration: 00:00:00.04, start: 0.000000, bitrate: 8995854 kb/s
>   Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 15748x23622 [SAR 200:200 DAR 2:3], 25 fps, 25 tbr, 25 tbn
> 
> Signed-off-by: Steven Liu <lq at chinaffmpeg.org>
> ---
>  libavutil/imgutils.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavutil/imgutils.c b/libavutil/imgutils.c
> index 9ab5757cf6..03abf1cece 100644
> --- a/libavutil/imgutils.c
> +++ b/libavutil/imgutils.c
> @@ -298,7 +298,7 @@ int av_image_check_size2(unsigned int w, unsigned int h, int64_t max_pixels, enu
>          stride = 8LL*w;
>      stride += 128*8;
>  
> -    if ((int)w<=0 || (int)h<=0 || stride >= INT_MAX || stride*(uint64_t)(h+128) >= INT_MAX) {
> +    if ((int)w<=0 || (int)h<=0 || stride >= INT64_MAX || stride*(uint64_t)(h+128) >= INT64_MAX) {
>          av_log(&imgutils, AV_LOG_ERROR, "Picture size %ux%u is invalid\n", w, h);
>          return AVERROR(EINVAL);
>      }
> 

The documentation of av_image_check_size2() contains this:
"Check if the given dimension of an image is valid, meaning that all
bytes of a plane of an image with the specified pix_fmt can be addressed
with a signed int."
So using INT_MAX here is intended.
(In the medium run, we will have to get rid of this 32bit stuff and
allow bigger buffers (and check whether the buffer fits into ptrdiff_t).
But this will require more work than just modifying a single line.)

- Andreas