[FFmpeg-devel] [PATCH 1/2] lavc/vaapi_encode: fix the caculation overflow

Mark Thompson sw at jkqxz.net
Tue Mar 13 00:54:12 EET 2018


On 12/03/18 05:38, Pengfei Qu wrote:
> this fix the overflow during the caculation before value assignment.
> 
> Signed-off-by: Pengfei Qu <Pengfei.Qu at intel.com>
> ---
>  libavcodec/vaapi_encode.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libavcodec/vaapi_encode.c b/libavcodec/vaapi_encode.c
> index 36c85a3..78347d4 100644
> --- a/libavcodec/vaapi_encode.c
> +++ b/libavcodec/vaapi_encode.c
> @@ -1168,9 +1168,9 @@ static av_cold int vaapi_encode_init_rate_control(AVCodecContext *avctx)
>              rc_target_percentage = 100;
>          } else {
>              rc_bits_per_second   = avctx->rc_max_rate;
> -            rc_target_percentage = (avctx->bit_rate * 100) / rc_bits_per_second;
> +            rc_target_percentage = (unsigned long)(avctx->bit_rate * 100) / rc_bits_per_second;

What is the problem here?  All this will do is truncate the 64-bit value on platforms with 32-bit long.

>          }
> -        rc_window_size = (hrd_buffer_size * 1000) / avctx->bit_rate;
> +        rc_window_size = (unsigned long)(hrd_buffer_size * 1000) / avctx->bit_rate;

Perhaps you want to ensure that the multiplication doesn't overflow rather than truncating the result after it has overflowed.

>      }
>  
>      ctx->rc_params.misc.type = VAEncMiscParameterTypeRateControl;
> 


More information about the ffmpeg-devel mailing list