[FFmpeg-devel] [PATCH] avutil/video_enc_param: fix warning

[lance.lmwang at gmail.com]

On Thu, May 21, 2020 at 04:34:57PM +0200, Carl Eugen Hoyos wrote:
> Am Do., 21. Mai 2020 um 16:21 Uhr schrieb <lance.lmwang at gmail.com>:
> >
> > From: Limin Wang <lance.lmwang at gmail.com>
> >
> > warning: comparison is always false due to limited range of data type [-Wtype-limits]
> > Also nb_blocks is unsigned int, so nb_blocks * sizeof(AVVideoBlockParams) may overflow,
> > so force to size_t
> >
> > Signed-off-by: Limin Wang <lance.lmwang at gmail.com>
> > ---
> >  libavutil/video_enc_params.c | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/libavutil/video_enc_params.c b/libavutil/video_enc_params.c
> > index c46c0f1..4a4c85f 100644
> > --- a/libavutil/video_enc_params.c
> > +++ b/libavutil/video_enc_params.c
> > @@ -33,8 +33,8 @@ AVVideoEncParams *av_video_enc_params_alloc(enum AVVideoEncParamsType type,
> >      size_t size;
> >
> >      size = sizeof(*par);
> > -    if (nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams) ||
> > -        nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size)
> > +    if (nb_blocks > UINT_MAX / sizeof(AVVideoBlockParams) ||
> > +        (size_t)nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size)
> 
> I don't think this patch is ok.
> 
> Shouldn't this be an assert() about sizeof(AVVideoBlockParams) instead?

Sorry, I try to fix the warning message, so go ahead if anyone have a proper fix.

> 
> Carl Eugen
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".

-- 
Thanks,
Limin Wang