[FFmpeg-devel] [PATCH v2] avutils/video_enc_params: fix type-limits compile warning on 64 bits build system

[Carl Eugen Hoyos]

Am So., 14. Feb. 2021 um 17:57 Uhr schrieb Nuo Mi <nuomi2021 at gmail.com>:
>
> On Mon, Feb 15, 2021 at 12:41 AM Nuo Mi <nuomi2021 at gmail.com> wrote:
>
> > This will fix following compile warning:
> >
> >     libavutil/video_enc_params.c: In function ‘av_video_enc_params_alloc:
> >
> >
> >            libavutil/video_enc_params.c:36:19: warning: comparison is
> > always false due to limited range of data type [-Wtype-limits]
> >
> >                              36 |     if (nb_blocks > SIZE_MAX /
> > sizeof(AVVideoBlockParams) ||
> >
> >                                                      |                   ^
> > ---
> >  libavutil/video_enc_params.c | 6 +++++-
> >  1 file changed, 5 insertions(+), 1 deletion(-)
> >
> > diff --git a/libavutil/video_enc_params.c b/libavutil/video_enc_params.c
> > index c46c0f1dc6..2606b5589a 100644
> > --- a/libavutil/video_enc_params.c
> > +++ b/libavutil/video_enc_params.c
> > @@ -33,7 +33,11 @@ AVVideoEncParams *av_video_enc_params_alloc(enum
> > AVVideoEncParamsType type,
> >      size_t size;
> >
> >      size = sizeof(*par);
> > -    if (nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams) ||
> > +    if (
> > +#if SIZE_MAX <= UINT_MAX
> > +        //check the overflow
> > +        nb_blocks > SIZE_MAX / sizeof(AVVideoBlockParams) ||
> > +#endif
> >          nb_blocks * sizeof(AVVideoBlockParams) > SIZE_MAX - size)
> >          return NULL;
> >      size += sizeof(AVVideoBlockParams) * nb_blocks;

Apart from the ugliness (that may be unavoidable, I get the warning
here for both "if (0 && condition)" and "if (0) if (condition)"), the
comment is unnecessary.

Not sure if the condition is guaranteed to be sufficient, a sizeof() trick
might be necessary.

Carl Eugen