[FFmpeg-devel] [PATCH v2] avcodec/nvenc: add option to skip padding OBUs

[Timo Rothenpieler]

On 29.03.2025 23:28, Cameron Gutman wrote:
> On Sat, Mar 29, 2025 at 4:42 PM Timo Rothenpieler <timo at rothenpieler.org> wrote:
>>
>> On 29.03.2025 22:37, Cameron Gutman wrote:
>>> On Sat, Mar 29, 2025 at 4:26 PM Timo Rothenpieler <timo at rothenpieler.org> wrote:
>>>>
>>>> On 29.03.2025 22:17, Cameron Gutman wrote:
>>>>> On Sat, Mar 29, 2025 at 4:02 PM Timo Rothenpieler <timo at rothenpieler.org> wrote:
>>>>>>
>>>>>> On 29.03.2025 21:58, Cameron Gutman wrote:
>>>>>>> Some scenarios (such as game streaming or videoconferencing) may use CBR
>>>>>>> to strictly cap the maximum encoded bitrate, but they don't mind the
>>>>>>> bitrate falling below the target if the encoder doesn't need the
>>>>>>> additional headroom.
>>>>>>
>>>>>> But why aren't they using vbr with a maxrate then?
>>>>>> I didn't add this conditionally since when in CBR mode, you actually
>>>>>> really want a constant bitrate.
>>>>>> If you're okay with it dropping lower, why not use another mode with a
>>>>>> maxrate set?
>>>>>>
>>>>>
>>>>> As I understand it, the rate control behavior of CBR without filler is
>>>>> different from
>>>>> just using VBR, particularly for low latency applications. Nvidia recommends use
>>>>> of CBR [0] (with or without filler depending on application needs [1])
>>>>> rather than
>>>>> using VBR for low latency applications.
>>>>>
>>>>> Also FWIW, we don't set enableFillerDataInsertion for CBR on H.264 and HEVC,
>>>>> so those will dip below the target bitrate in CBR mode while AV1 will not.
>>>>
>>>> We probably should really. In the past outputBufferingPeriodSEI
>>>> controlled this in some weird way.
>>>>
>>>> The expectation of CBR by default should be to be actually constant no
>>>> matter what.
>>>>
>>>> I just don't know what actual role the buffering period SEI plays this
>>>> day, with the new option now existing.
>>>>
>>>> If you like you can add the same logic for that flag as well.
>>>>
>>>> I'd also prefer this to be called strict_cbr or cbr_padding or
>>>> something, to make it obvious what it relates to.
>>>
>>> To ensure I understand, you want the following:
>>> 1. Rename the filler_data option in this patch to strict_cbr (keeping
>>> the default value of 1)
>>
>> Yes, definitely keep it enabled by default.
>> I'm not too picky about the exact name, as long as it contains cbr.
>>
>> Might even slightly prefer cbr_padding, since it's more obvious to the
>> common user what it does.
>>
> 
> Sounds good to me.
> 
>>> 2. Change H.264 and HEVC to set enableFillerDataInsertion for CBR by
>>> default and add strict_cbr option to those too
>>
>> Yes, or whatever the option ends up named.
>>
>>> 3. Don't set outputBufferingPeriodSEI = 1 unless filler data is also enabled
>>
>> That's what I'm not sure about.
>> To my knowledge, and given that people stream to Twitch and other
>> services with strict CBR requirements, h264 and hevc already produce
>> strict CBR.
>>
>> I'd want the new option to toggle the strict CBR behaviour, and would
>> need to investigate first what that implies.
>>
>> Might just be that filler data insertion is enabled by default there,
>> but not for AV1.
>> Or it's actually somehow related to the buffering period SEI. Since it's
>> the only original user of the IS_CBR check.
>>
> 
> When looking through the older headers, I saw that
> outputBufferingPeriodSEI used to have the following comment:
> 
> "When set for following rateControlMode : NV_ENC_PARAMS_RC_CBR,
> NV_ENC_PARAMS_RC_CBR_LOWDELAY_HQ, NV_ENC_PARAMS_RC_CBR_HQ, filler data
> is inserted if needed to achieve HRD bitrate"
> 
> However, this was removed starting in SDK 12.0. Since it's hard to say
> what each driver version between SDK 9.1 and SDK 12.0 will do with a
> mismatch between outputBufferingPeriodSEI and
> enableFillerDataInsertion, I think it's probably safest to tie the
> options together for now.

Sounds good to me

>> So for now, if you want to add the option for the other codecs to, just
>> make it toggle the new flag.
>>
>> (And keep in mind that the oldest SDK still supported does not have that
>> flag, so it'll need a feature guard)
> 
> Thanks for the heads up. Looks like it was added in SDK 9.1.
>