[FFmpeg-devel] [PATCH] avcodec/jpeg2000dec: Explicitly cast quad_width and quad_height to size_t

[Pierre-Anthony Lemieux]

On Tue, Apr 25, 2023 at 11:25 AM <etemesicaleb at gmail.com> wrote:
>
> From: caleb <etemesicaleb at gmail.com>
>
> Prevents un-necessary casts when calculating buf_size

This is to address Coverity CID 1528149: """Suspicious implicit sign
extension: "quad_height" with type "uint16_t const" (16 bits,
unsigned) is promoted in "4 * quad_width * quad_height" to type "int"
(32 bits, signed), then sign-extended to type "unsigned long" (64
bits, unsigned).  If "4 * quad_width * quad_height" is greater than
0x7FFFFFFF, the upper bits of the result will all be 1."""

> ---
>  libavcodec/jpeg2000htdec.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/libavcodec/jpeg2000htdec.c b/libavcodec/jpeg2000htdec.c
> index 51cd96e0f1..8907820806 100644
> --- a/libavcodec/jpeg2000htdec.c
> +++ b/libavcodec/jpeg2000htdec.c
> @@ -592,8 +592,8 @@ static int jpeg2000_decode_ht_cleanup_segment(const Jpeg2000DecoderContext *s,
>      const uint16_t is_border_x = width % 2;
>      const uint16_t is_border_y = height % 2;
>
> -    const uint16_t quad_width  = ff_jpeg2000_ceildivpow2(width, 1);
> -    const uint16_t quad_height = ff_jpeg2000_ceildivpow2(height, 1);
> +    const size_t quad_width  = ff_jpeg2000_ceildivpow2(width, 1);
> +    const size_t quad_height = ff_jpeg2000_ceildivpow2(height, 1);

Shouldn't quad_width and quad_height be kept as uint16_t, and instead
be explicitly cast to (size_t) when computing buf_size?

>
>      size_t buf_size = 4 * quad_width * quad_height;
>
> --
> 2.39.2
>