[FFmpeg-devel] [PATCH v2] avcodec/nvenc: add option to skip padding OBUs

[Cameron Gutman]

On Sat, Mar 29, 2025 at 4:26 PM Timo Rothenpieler <timo at rothenpieler.org> wrote:
>
> On 29.03.2025 22:17, Cameron Gutman wrote:
> > On Sat, Mar 29, 2025 at 4:02 PM Timo Rothenpieler <timo at rothenpieler.org> wrote:
> >>
> >> On 29.03.2025 21:58, Cameron Gutman wrote:
> >>> Some scenarios (such as game streaming or videoconferencing) may use CBR
> >>> to strictly cap the maximum encoded bitrate, but they don't mind the
> >>> bitrate falling below the target if the encoder doesn't need the
> >>> additional headroom.
> >>
> >> But why aren't they using vbr with a maxrate then?
> >> I didn't add this conditionally since when in CBR mode, you actually
> >> really want a constant bitrate.
> >> If you're okay with it dropping lower, why not use another mode with a
> >> maxrate set?
> >>
> >
> > As I understand it, the rate control behavior of CBR without filler is
> > different from
> > just using VBR, particularly for low latency applications. Nvidia recommends use
> > of CBR [0] (with or without filler depending on application needs [1])
> > rather than
> > using VBR for low latency applications.
> >
> > Also FWIW, we don't set enableFillerDataInsertion for CBR on H.264 and HEVC,
> > so those will dip below the target bitrate in CBR mode while AV1 will not.
>
> We probably should really. In the past outputBufferingPeriodSEI
> controlled this in some weird way.
>
> The expectation of CBR by default should be to be actually constant no
> matter what.
>
> I just don't know what actual role the buffering period SEI plays this
> day, with the new option now existing.
>
> If you like you can add the same logic for that flag as well.
>
> I'd also prefer this to be called strict_cbr or cbr_padding or
> something, to make it obvious what it relates to.

To ensure I understand, you want the following:
1. Rename the filler_data option in this patch to strict_cbr (keeping
the default value of 1)
2. Change H.264 and HEVC to set enableFillerDataInsertion for CBR by
default and add strict_cbr option to those too
3. Don't set outputBufferingPeriodSEI = 1 unless filler data is also enabled

Is that correct?

>
> > [0]: https://docs.nvidia.com/video-technologies/video-codec-sdk/12.1/nvenc-video-encoder-api-prog-guide/index.html#recommended-nvenc-settings__table-nvenc-settings
> > [1]: https://docs.nvidia.com/video-technologies/video-codec-sdk/12.1/nvenc-video-encoder-api-prog-guide/index.html#rate-control
> >
> >>> Allow users to opt-out of filler data in CBR mode for those usecases
> >>> where it is unwanted.
> >>>
> >>> Signed-off-by: Cameron Gutman <aicommander at gmail.com>
> >>> ---
> >>> v2: Rebased to resolve conflicts against master
> >>> ---
> >>>    libavcodec/nvenc.c     | 2 +-
> >>>    libavcodec/nvenc.h     | 1 +
> >>>    libavcodec/nvenc_av1.c | 2 ++
> >>>    3 files changed, 4 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
> >>> index 0f5e772b3e..0f2336b0ec 100644
> >>> --- a/libavcodec/nvenc.c
> >>> +++ b/libavcodec/nvenc.c
> >>> @@ -1624,7 +1624,7 @@ static av_cold int nvenc_setup_av1_config(AVCodecContext *avctx)
> >>>
> >>>        av1->idrPeriod = cc->gopLength;
> >>>
> >>> -    if (IS_CBR(cc->rcParams.rateControlMode)) {
> >>> +    if (ctx->filler_data && IS_CBR(cc->rcParams.rateControlMode)) {
> >>>            av1->enableBitstreamPadding = 1;
> >>>        }
> >>>
> >>> diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h
> >>> index e035e123c6..3431457422 100644
> >>> --- a/libavcodec/nvenc.h
> >>> +++ b/libavcodec/nvenc.h
> >>> @@ -309,6 +309,7 @@ typedef struct NvencContext
> >>>        int unidir_b;
> >>>        int split_encode_mode;
> >>>        int mdm, cll;
> >>> +    int filler_data;
> >>>    } NvencContext;
> >>>
> >>>    int ff_nvenc_encode_init(AVCodecContext *avctx);
> >>> diff --git a/libavcodec/nvenc_av1.c b/libavcodec/nvenc_av1.c
> >>> index 01626113ab..c00817af7b 100644
> >>> --- a/libavcodec/nvenc_av1.c
> >>> +++ b/libavcodec/nvenc_av1.c
> >>> @@ -156,6 +156,8 @@ static const AVOption options[] = {
> >>>                                                                OFFSET(extra_sei),    AV_OPT_TYPE_BOOL,  { .i64 = 1 }, 0, 1, VE },
> >>>        { "a53cc",        "Use A53 Closed Captions (if available)", OFFSET(a53_cc),   AV_OPT_TYPE_BOOL,  { .i64 = 1 }, 0, 1, VE },
> >>>        { "s12m_tc",      "Use timecode (if available)",        OFFSET(s12m_tc),      AV_OPT_TYPE_BOOL,  { .i64 = 1 }, 0, 1, VE },
> >>> +    { "filler_data",  "Use filler data to ensure CBR bitrate is strictly adhered to",
> >>> +                                                            OFFSET(filler_data),  AV_OPT_TYPE_BOOL,  { .i64 = 1 }, 0, 1, VE },
> >>>    #ifdef NVENC_HAVE_H264_AND_AV1_TEMPORAL_FILTER
> >>>        { "tf_level",     "Specifies the strength of the temporal filtering",
> >>>                                                                OFFSET(tf_level),     AV_OPT_TYPE_INT,   { .i64 = -1 }, -1, INT_MAX, VE, .unit = "tf_level" },
> >>
> >> _______________________________________________
> >> ffmpeg-devel mailing list
> >> ffmpeg-devel at ffmpeg.org
> >> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> >>
> >> To unsubscribe, visit link above, or email
> >> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>