[FFmpeg-devel] [PATCH v2] avcodec/nvenc: add option to skip padding OBUs

[Timo Rothenpieler]

On 29.03.2025 22:17, Cameron Gutman wrote:
> On Sat, Mar 29, 2025 at 4:02 PM Timo Rothenpieler <timo at rothenpieler.org> wrote:
>>
>> On 29.03.2025 21:58, Cameron Gutman wrote:
>>> Some scenarios (such as game streaming or videoconferencing) may use CBR
>>> to strictly cap the maximum encoded bitrate, but they don't mind the
>>> bitrate falling below the target if the encoder doesn't need the
>>> additional headroom.
>>
>> But why aren't they using vbr with a maxrate then?
>> I didn't add this conditionally since when in CBR mode, you actually
>> really want a constant bitrate.
>> If you're okay with it dropping lower, why not use another mode with a
>> maxrate set?
>>
> 
> As I understand it, the rate control behavior of CBR without filler is
> different from
> just using VBR, particularly for low latency applications. Nvidia recommends use
> of CBR [0] (with or without filler depending on application needs [1])
> rather than
> using VBR for low latency applications.
> 
> Also FWIW, we don't set enableFillerDataInsertion for CBR on H.264 and HEVC,
> so those will dip below the target bitrate in CBR mode while AV1 will not.

We probably should really. In the past outputBufferingPeriodSEI 
controlled this in some weird way.

The expectation of CBR by default should be to be actually constant no 
matter what.

I just don't know what actual role the buffering period SEI plays this 
day, with the new option now existing.

If you like you can add the same logic for that flag as well.

I'd also prefer this to be called strict_cbr or cbr_padding or 
something, to make it obvious what it relates to.

> [0]: https://docs.nvidia.com/video-technologies/video-codec-sdk/12.1/nvenc-video-encoder-api-prog-guide/index.html#recommended-nvenc-settings__table-nvenc-settings
> [1]: https://docs.nvidia.com/video-technologies/video-codec-sdk/12.1/nvenc-video-encoder-api-prog-guide/index.html#rate-control
> 
>>> Allow users to opt-out of filler data in CBR mode for those usecases
>>> where it is unwanted.
>>>
>>> Signed-off-by: Cameron Gutman <aicommander at gmail.com>
>>> ---
>>> v2: Rebased to resolve conflicts against master
>>> ---
>>>    libavcodec/nvenc.c     | 2 +-
>>>    libavcodec/nvenc.h     | 1 +
>>>    libavcodec/nvenc_av1.c | 2 ++
>>>    3 files changed, 4 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
>>> index 0f5e772b3e..0f2336b0ec 100644
>>> --- a/libavcodec/nvenc.c
>>> +++ b/libavcodec/nvenc.c
>>> @@ -1624,7 +1624,7 @@ static av_cold int nvenc_setup_av1_config(AVCodecContext *avctx)
>>>
>>>        av1->idrPeriod = cc->gopLength;
>>>
>>> -    if (IS_CBR(cc->rcParams.rateControlMode)) {
>>> +    if (ctx->filler_data && IS_CBR(cc->rcParams.rateControlMode)) {
>>>            av1->enableBitstreamPadding = 1;
>>>        }
>>>
>>> diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h
>>> index e035e123c6..3431457422 100644
>>> --- a/libavcodec/nvenc.h
>>> +++ b/libavcodec/nvenc.h
>>> @@ -309,6 +309,7 @@ typedef struct NvencContext
>>>        int unidir_b;
>>>        int split_encode_mode;
>>>        int mdm, cll;
>>> +    int filler_data;
>>>    } NvencContext;
>>>
>>>    int ff_nvenc_encode_init(AVCodecContext *avctx);
>>> diff --git a/libavcodec/nvenc_av1.c b/libavcodec/nvenc_av1.c
>>> index 01626113ab..c00817af7b 100644
>>> --- a/libavcodec/nvenc_av1.c
>>> +++ b/libavcodec/nvenc_av1.c
>>> @@ -156,6 +156,8 @@ static const AVOption options[] = {
>>>                                                                OFFSET(extra_sei),    AV_OPT_TYPE_BOOL,  { .i64 = 1 }, 0, 1, VE },
>>>        { "a53cc",        "Use A53 Closed Captions (if available)", OFFSET(a53_cc),   AV_OPT_TYPE_BOOL,  { .i64 = 1 }, 0, 1, VE },
>>>        { "s12m_tc",      "Use timecode (if available)",        OFFSET(s12m_tc),      AV_OPT_TYPE_BOOL,  { .i64 = 1 }, 0, 1, VE },
>>> +    { "filler_data",  "Use filler data to ensure CBR bitrate is strictly adhered to",
>>> +                                                            OFFSET(filler_data),  AV_OPT_TYPE_BOOL,  { .i64 = 1 }, 0, 1, VE },
>>>    #ifdef NVENC_HAVE_H264_AND_AV1_TEMPORAL_FILTER
>>>        { "tf_level",     "Specifies the strength of the temporal filtering",
>>>                                                                OFFSET(tf_level),     AV_OPT_TYPE_INT,   { .i64 = -1 }, -1, INT_MAX, VE, .unit = "tf_level" },
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".