[FFmpeg-devel] [PATCH v2] avcodec/nvenc: add option to skip padding OBUs
Timo Rothenpieler
timo at rothenpieler.org
Sat Mar 29 23:42:44 EET 2025
On 29.03.2025 22:37, Cameron Gutman wrote:
> On Sat, Mar 29, 2025 at 4:26 PM Timo Rothenpieler <timo at rothenpieler.org> wrote:
>>
>> On 29.03.2025 22:17, Cameron Gutman wrote:
>>> On Sat, Mar 29, 2025 at 4:02 PM Timo Rothenpieler <timo at rothenpieler.org> wrote:
>>>>
>>>> On 29.03.2025 21:58, Cameron Gutman wrote:
>>>>> Some scenarios (such as game streaming or videoconferencing) may use CBR
>>>>> to strictly cap the maximum encoded bitrate, but they don't mind the
>>>>> bitrate falling below the target if the encoder doesn't need the
>>>>> additional headroom.
>>>>
>>>> But why aren't they using vbr with a maxrate then?
>>>> I didn't add this conditionally since when in CBR mode, you actually
>>>> really want a constant bitrate.
>>>> If you're okay with it dropping lower, why not use another mode with a
>>>> maxrate set?
>>>>
>>>
>>> As I understand it, the rate control behavior of CBR without filler is
>>> different from
>>> just using VBR, particularly for low latency applications. Nvidia recommends use
>>> of CBR [0] (with or without filler depending on application needs [1])
>>> rather than
>>> using VBR for low latency applications.
>>>
>>> Also FWIW, we don't set enableFillerDataInsertion for CBR on H.264 and HEVC,
>>> so those will dip below the target bitrate in CBR mode while AV1 will not.
>>
>> We probably should really. In the past outputBufferingPeriodSEI
>> controlled this in some weird way.
>>
>> The expectation of CBR by default should be to be actually constant no
>> matter what.
>>
>> I just don't know what actual role the buffering period SEI plays this
>> day, with the new option now existing.
>>
>> If you like you can add the same logic for that flag as well.
>>
>> I'd also prefer this to be called strict_cbr or cbr_padding or
>> something, to make it obvious what it relates to.
>
> To ensure I understand, you want the following:
> 1. Rename the filler_data option in this patch to strict_cbr (keeping
> the default value of 1)
Yes, definitely keep it enabled by default.
I'm not too picky about the exact name, as long as it contains cbr.
Might even slightly prefer cbr_padding, since it's more obvious to the
common user what it does.
> 2. Change H.264 and HEVC to set enableFillerDataInsertion for CBR by
> default and add strict_cbr option to those too
Yes, or whatever the option ends up named.
> 3. Don't set outputBufferingPeriodSEI = 1 unless filler data is also enabled
That's what I'm not sure about.
To my knowledge, and given that people stream to Twitch and other
services with strict CBR requirements, h264 and hevc already produce
strict CBR.
I'd want the new option to toggle the strict CBR behaviour, and would
need to investigate first what that implies.
Might just be that filler data insertion is enabled by default there,
but not for AV1.
Or it's actually somehow related to the buffering period SEI. Since it's
the only original user of the IS_CBR check.
So for now, if you want to add the option for the other codecs to, just
make it toggle the new flag.
(And keep in mind that the oldest SDK still supported does not have that
flag, so it'll need a feature guard)
> Is that correct?
>
>>
>>> [0]: https://docs.nvidia.com/video-technologies/video-codec-sdk/12.1/nvenc-video-encoder-api-prog-guide/index.html#recommended-nvenc-settings__table-nvenc-settings
>>> [1]: https://docs.nvidia.com/video-technologies/video-codec-sdk/12.1/nvenc-video-encoder-api-prog-guide/index.html#rate-control
>>>
>>>>> Allow users to opt-out of filler data in CBR mode for those usecases
>>>>> where it is unwanted.
>>>>>
>>>>> Signed-off-by: Cameron Gutman <aicommander at gmail.com>
>>>>> ---
>>>>> v2: Rebased to resolve conflicts against master
>>>>> ---
>>>>> libavcodec/nvenc.c | 2 +-
>>>>> libavcodec/nvenc.h | 1 +
>>>>> libavcodec/nvenc_av1.c | 2 ++
>>>>> 3 files changed, 4 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
>>>>> index 0f5e772b3e..0f2336b0ec 100644
>>>>> --- a/libavcodec/nvenc.c
>>>>> +++ b/libavcodec/nvenc.c
>>>>> @@ -1624,7 +1624,7 @@ static av_cold int nvenc_setup_av1_config(AVCodecContext *avctx)
>>>>>
>>>>> av1->idrPeriod = cc->gopLength;
>>>>>
>>>>> - if (IS_CBR(cc->rcParams.rateControlMode)) {
>>>>> + if (ctx->filler_data && IS_CBR(cc->rcParams.rateControlMode)) {
>>>>> av1->enableBitstreamPadding = 1;
>>>>> }
>>>>>
>>>>> diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h
>>>>> index e035e123c6..3431457422 100644
>>>>> --- a/libavcodec/nvenc.h
>>>>> +++ b/libavcodec/nvenc.h
>>>>> @@ -309,6 +309,7 @@ typedef struct NvencContext
>>>>> int unidir_b;
>>>>> int split_encode_mode;
>>>>> int mdm, cll;
>>>>> + int filler_data;
>>>>> } NvencContext;
>>>>>
>>>>> int ff_nvenc_encode_init(AVCodecContext *avctx);
>>>>> diff --git a/libavcodec/nvenc_av1.c b/libavcodec/nvenc_av1.c
>>>>> index 01626113ab..c00817af7b 100644
>>>>> --- a/libavcodec/nvenc_av1.c
>>>>> +++ b/libavcodec/nvenc_av1.c
>>>>> @@ -156,6 +156,8 @@ static const AVOption options[] = {
>>>>> OFFSET(extra_sei), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, VE },
>>>>> { "a53cc", "Use A53 Closed Captions (if available)", OFFSET(a53_cc), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, VE },
>>>>> { "s12m_tc", "Use timecode (if available)", OFFSET(s12m_tc), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, VE },
>>>>> + { "filler_data", "Use filler data to ensure CBR bitrate is strictly adhered to",
>>>>> + OFFSET(filler_data), AV_OPT_TYPE_BOOL, { .i64 = 1 }, 0, 1, VE },
>>>>> #ifdef NVENC_HAVE_H264_AND_AV1_TEMPORAL_FILTER
>>>>> { "tf_level", "Specifies the strength of the temporal filtering",
>>>>> OFFSET(tf_level), AV_OPT_TYPE_INT, { .i64 = -1 }, -1, INT_MAX, VE, .unit = "tf_level" },
>>>>
>>>> _______________________________________________
>>>> ffmpeg-devel mailing list
>>>> ffmpeg-devel at ffmpeg.org
>>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>>>
>>>> To unsubscribe, visit link above, or email
>>>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>>
More information about the ffmpeg-devel
mailing list