[FFmpeg-user] 4K RGB->YUV realtime conversion.

[Peter White]

On Mon, Mar 08, 2021 at 03:19:46PM +0100, Anders Degerberg wrote:
> Hi,
> 
> I've come across a problem I really can't figure out when trying to stream
> realtime a RGB 10-bit Decklink input, as YUV444 10-bit HEVC in 25fps.
>  In a 1920x1080 resolution on the Decklink input, it all works very well,
> using hevc_nvenc codec on a Nvidia GPU. Auto_scaler kicks in converting the
> correctly identified 'gbrp10le' input format to 'yuv44416le' for hevc_nvenc.
>
> But doing the same in 4K resolution causes input buffer overruns, only
> capable of about 20 fps. When watching CPU usage, there's only real load on
> one of the 32 threads, and that load is maxing out at 99%. CPU frequency is
> close to 4GHz on the AMD CPU.
> If I then switch the Decklink input to YUV instead, 4K works fine, so the
> hevc encoding on the gpu or streaming protocol is not the bottleneck.

I am not sure, which pixel formats (-pix_fmt) the hevc_nvenc supports
but YUV seems a reasonable guess. That is why it runs fine with that
format. But for the conversion of RGB to YUV the software scaler seems
to kick in, which explains the high processor load.

Try:

ffmpeg -i input_file -sws_flags show_info ... output_file

and send the output.

>  Any suggestions how I can do this 4K RGB 10-bit->YUV444 10-bit colorspace
> conversion in ffmpeg in realtime?

Do you really have to use RGB on the Decklink? Seems pointless to me, if
it has to be converted anyway. Why not simply keep it in YUV-mode?

> I've also tried to build with a patch for multi-thread swscale found here:
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/C1125BD979E9AE41908DC4913CBFACBB67A31216@IRSMSX103.ger.corp.intel.com/
> But this patch doesn't seem to make a difference for the rgb2yuv colorspace
> conversions, it's probably only affective for image up/downscaling?

Maybe it is because of the massive amount of data that needs to be
shifted around. Not sure, but 4K is four times as much data as 1080p.