[FFmpeg-devel] [PATCH] libavcodec/nvenc.c: copy incoming hwaccel frames instead of ref count increase
Timo Rothenpieler
timo at rothenpieler.org
Tue Sep 28 22:11:15 EEST 2021
On 28.09.2021 19:58, Timo Rothenpieler wrote:
> Hmm, so far my approach to deal with this was to inject a
> scale_cuda=passthrough=0 into the filter chain, which pretty much does
> exactly this, but only controllable by the user.
>
> But I do agree that this is a bit of a clutch and not all that user
> friendly.
>
> My main concern with this approach is that it will inevitably increase
> VRAM usage, depending on bframe count and resolution even quite
> significantly.
> And it's surprisingly common that users show up that are highly pressed
> for memory. When bframes were switched on by default, several people
> showed up who where suddenly running out of VRAM.
>
> I do like this approach though, since it will for the average user make
> using a full hw chain a lot less bothersome.
>
> So what I'd propose is:
>
> - Add an option to retain the old behaviour of just holding a reference
> to the input frame no matter what.
> - Instead of explicitly copying the frame like you do right now, call
> av_frame_make_writable() on the frame, right after where you right now
> are replacing av_frame_ref with av_hwframe_transfer_data.
> That is for one very easy to disable conditionally, and does not require
> you to guard all the unref calls.
> Plus, it will only actually copy the frame if needed (i.e. it won't do
> anything if it comes out of a filterchain and has nothing else holding a
> ref)
>
>
> Timo
See attached patch for that approach.
I just encoded a 5 minute sample using it, and I do see a marginal
decrease in performance (it drops by literally x0.01 speed, so pretty
much within margin of error, but it did show that consistently) and
increase in VRAM usage as expected.
However, given that your patch does seem to work just fine, somehow, it
would be interesting to know if re-using a frame/CUDA buffer after
registering it with nvenc is safe?
Given that the logic right now never unregisters buffers unless it runs
out of free slots, it would seem weird to me if that was the case. What
if a buffer actually does get re-used, as is common with
non-nvdec-frames allocated from a hwframes ctx?
-------------- next part --------------
From e8e099dac72b1962aba7c8a4ba87f4035eb0a29d Mon Sep 17 00:00:00 2001
From: Timo Rothenpieler <timo at rothenpieler.org>
Date: Tue, 28 Sep 2021 20:45:26 +0200
Subject: [PATCH] avcodec/nvenc: copy incoming hwframes when applicable
---
libavcodec/nvenc.c | 21 +++++++++++++++------
libavcodec/nvenc.h | 1 +
libavcodec/nvenc_h264.c | 2 ++
libavcodec/nvenc_hevc.c | 2 ++
4 files changed, 20 insertions(+), 6 deletions(-)
diff --git a/libavcodec/nvenc.c b/libavcodec/nvenc.c
index edc46ed33a..e3cd67fbc1 100644
--- a/libavcodec/nvenc.c
+++ b/libavcodec/nvenc.c
@@ -1910,20 +1910,29 @@ static int nvenc_upload_frame(AVCodecContext *avctx, const AVFrame *frame,
NvencDynLoadFunctions *dl_fn = &ctx->nvenc_dload_funcs;
NV_ENCODE_API_FUNCTION_LIST *p_nvenc = &dl_fn->nvenc_funcs;
- int res;
+ int res, reg_idx;
NVENCSTATUS nv_status;
if (avctx->pix_fmt == AV_PIX_FMT_CUDA || avctx->pix_fmt == AV_PIX_FMT_D3D11) {
- int reg_idx = nvenc_register_frame(avctx, frame);
+ res = av_frame_ref(nvenc_frame->in_ref, frame);
+ if (res < 0)
+ return res;
+
+ if (avctx->pix_fmt == AV_PIX_FMT_CUDA && !ctx->lowmem) {
+ res = av_frame_make_writable(nvenc_frame->in_ref);
+ if (res < 0) {
+ av_frame_unref(nvenc_frame->in_ref);
+ return res;
+ }
+ }
+
+ reg_idx = nvenc_register_frame(avctx, nvenc_frame->in_ref);
if (reg_idx < 0) {
av_log(avctx, AV_LOG_ERROR, "Could not register an input HW frame\n");
+ av_frame_unref(nvenc_frame->in_ref);
return reg_idx;
}
- res = av_frame_ref(nvenc_frame->in_ref, frame);
- if (res < 0)
- return res;
-
if (!ctx->registered_frames[reg_idx].mapped) {
ctx->registered_frames[reg_idx].in_map.version = NV_ENC_MAP_INPUT_RESOURCE_VER;
ctx->registered_frames[reg_idx].in_map.registeredResource = ctx->registered_frames[reg_idx].regptr;
diff --git a/libavcodec/nvenc.h b/libavcodec/nvenc.h
index 08531e1be3..927f7aa8f3 100644
--- a/libavcodec/nvenc.h
+++ b/libavcodec/nvenc.h
@@ -235,6 +235,7 @@ typedef struct NvencContext
int intra_refresh;
int single_slice_intra_refresh;
int constrained_encoding;
+ int lowmem;
} NvencContext;
int ff_nvenc_encode_init(AVCodecContext *avctx);
diff --git a/libavcodec/nvenc_h264.c b/libavcodec/nvenc_h264.c
index 7d78aa0d87..c96a26902a 100644
--- a/libavcodec/nvenc_h264.c
+++ b/libavcodec/nvenc_h264.c
@@ -194,6 +194,8 @@ static const AVOption options[] = {
OFFSET(single_slice_intra_refresh), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE },
{ "constrained-encoding", "Enable constrainedFrame encoding where each slice in the constrained picture is independent of other slices",
OFFSET(constrained_encoding), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE },
+ { "lowmem", "Avoids DtoD copies of frames, reducing memory footprint and potentially increasing performance at the cost of stability",
+ OFFSET(lowmem), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE },
{ NULL }
};
diff --git a/libavcodec/nvenc_hevc.c b/libavcodec/nvenc_hevc.c
index a13ac5a395..f46c4862ce 100644
--- a/libavcodec/nvenc_hevc.c
+++ b/libavcodec/nvenc_hevc.c
@@ -175,6 +175,8 @@ static const AVOption options[] = {
OFFSET(single_slice_intra_refresh), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE },
{ "constrained-encoding", "Enable constrainedFrame encoding where each slice in the constrained picture is independent of other slices",
OFFSET(constrained_encoding), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE },
+ { "lowmem", "Avoids DtoD copies of frames, reducing memory footprint and potentially increasing performance at the cost of stability",
+ OFFSET(lowmem), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VE },
{ NULL }
};
--
2.25.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4494 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20210928/45bc6e87/attachment.bin>
More information about the ffmpeg-devel
mailing list