[FFmpeg-devel] [PATCH] libavcodec/nvenc.c: copy incoming hwaccel frames instead of ref count increase

[Timo Rothenpieler]

On 28.09.2021 19:58, Timo Rothenpieler wrote:
> On 28.09.2021 18:22, Roman Arzumanyan wrote:
>> Hello,
>>
>> This patch makes nvenc copy incoming hwaccel frames instead of ref 
>> count increase.
>> It fixes the bug which may happen when on-GPU transcoding is done and 
>> encoder is set to use B frames.
>>
>> How to reproduce the bug:
>> ./ffmpeg \
>>    -hwaccel cuda -hwaccel_output_format cuda \
>>    -i input.mkv \
>>    -c:v h264_nvenc -preset p4 -tune hq -bf 3 \
>>    -y output.mkv
>>
>> Expected output:
>> [h264 @ 0x55b14da4b4c0] No decoder surfaces left
>> [h264 @ 0x55b14da682c0] No decoder surfaces left
>> [h264 @ 0x55b14da850c0] No decoder surfaces left
>> [h264 @ 0x55b14daa1ec0] No decoder surfaces left
>> Error while decoding stream #0:0: Invalid data found when processing 
>> input
>> [h264 @ 0x55b14da2e6c0] No decoder surfaces left
>> Error while decoding stream #0:0: Invalid data found when processing 
>> input
>>      Last message repeated 1 times
>>
>>
>> Although fix adds extra CUDA DtoD memcopy, our internal testing 
>> results didn't show any noticeable difference in transcoding performance.
>>

Something else I just realized:

The nvenc_frame->in_ref you are copying to is never actually used anywhere.
It is purely held as a reference, so the frame is not freed before nvenc 
is done with it.

So effectively what your patch is doing right now is making a copy of 
the frame, but then never do anything with that copy.
nvEncRegisterResource() is still called on the original frame, and that 
registered resource is then used for nvenc.

I'm surprised that actually works? Is that okay to do, and then let 
nvdec continue on?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4494 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20210928/6d22d431/attachment.bin>