[FFmpeg-devel] [PATCH] avfilter/scale: use int64_t for height/width calculation to address overflow

[Michael Niedermayer]

On Sat, Feb 04, 2017 at 10:04:36AM -0800, Aman Gupta wrote:
> From: Aman Gupta <aman at tmm1.net>
> 
> ---
>  libavfilter/scale.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/libavfilter/scale.c b/libavfilter/scale.c
> index 50cd442..9725f1f 100644
> --- a/libavfilter/scale.c
> +++ b/libavfilter/scale.c
> @@ -68,7 +68,7 @@ int ff_scale_eval_dimensions(void *log_ctx,
>      const AVPixFmtDescriptor *desc = av_pix_fmt_desc_get(inlink->format);
>      const AVPixFmtDescriptor *out_desc = av_pix_fmt_desc_get(outlink->format);
>      const char *expr;
> -    int w, h;
> +    int64_t w, h;
>      int factor_w, factor_h;
>      int eval_w, eval_h;
>      int ret;
> @@ -138,8 +138,8 @@ int ff_scale_eval_dimensions(void *log_ctx,
>      if (h < 0)
>          h = av_rescale(w, inlink->h, inlink->w * factor_h) * factor_h;
>  
> -    *ret_w = w;
> -    *ret_h = h;
> +    *ret_w = (int)w;
> +    *ret_h = (int)h;

I think the INT_MAX checks should be before casting to int

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The real ebay dictionary, page 1
"Used only once"    - "Some unspecified defect prevented a second use"
"In good condition" - "Can be repaird by experienced expert"
"As is" - "You wouldnt want it even if you were payed for it, if you knew ..."
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20170205/aebc8f9d/attachment.sig>