[FFmpeg-devel] [PATCH] vaapi_encode_h26[45]: Reject bitrate targets higher than 2^31

[Mark Thompson]

On 02/06/16 22:00, Matthieu Bouron wrote:
> On Thu, Jun 02, 2016 at 07:13:39PM +0100, Mark Thompson wrote:
>> ---
>> ... something like this.
>>
>>  libavcodec/vaapi_encode_h264.c | 6 ++++++
>>  libavcodec/vaapi_encode_h265.c | 6 ++++++
>>  2 files changed, 12 insertions(+)
>>
>> diff --git a/libavcodec/vaapi_encode_h264.c b/libavcodec/vaapi_encode_h264.c
>> index 0a99bb1..019ed1f 100644
>> --- a/libavcodec/vaapi_encode_h264.c
>> +++ b/libavcodec/vaapi_encode_h264.c
>> @@ -731,6 +731,12 @@ static av_cold int vaapi_encode_h264_init_constant_bitrate(AVCodecContext *avctx
>>      int hrd_buffer_size;
>>      int hrd_initial_buffer_fullness;
>>
>> +    if (avctx->bit_rate >= 1u << 31) {
> 
> Wouldn't INT32_MAX be more aproriate ?

Hmm.  No preference - I went for 1u << 31 to match the 2^31 in the error message, but maybe INT32_MAX makes the code constraint slightly clearer.

>> +        av_log(avctx, AV_LOG_ERROR, "Target bitrate of 2^31 bps or "
>> +               "higher is not supported.\n");
>> +        return AVERROR(EINVAL);
>> +    }
>> +
>>      if (avctx->rc_buffer_size)
>>          hrd_buffer_size = avctx->rc_buffer_size;
>>      else
>> diff --git a/libavcodec/vaapi_encode_h265.c b/libavcodec/vaapi_encode_h265.c
>> index 05d3aa4..060c7b7 100644
>> --- a/libavcodec/vaapi_encode_h265.c
>> +++ b/libavcodec/vaapi_encode_h265.c
>> @@ -1158,6 +1158,12 @@ static av_cold int vaapi_encode_h265_init_constant_bitrate(AVCodecContext *avctx
>>      int hrd_buffer_size;
>>      int hrd_initial_buffer_fullness;
>>
>> +    if (avctx->bit_rate >= 1u << 31) {a
> 
> Same comment as above.

Whichever answer you prefer, they should be the same.

>> +        av_log(avctx, AV_LOG_ERROR, "Target bitrate of 2^31 bps or "
>> +               "higher is not supported.\n");
>> +        return AVERROR(EINVAL);
>> +    }
>> +
>>      if (avctx->rc_buffer_size)
>>          hrd_buffer_size = avctx->rc_buffer_size;
>>      else
>> -- 
>> 2.8.1
>>