[FFmpeg-user] Understanding the performance of getting decoded video out of ffmpeg

[Ferdi Scholten]

On 24-01-2022 22:41, Eric Gross wrote:
> Hi all,
>
> I'd love to understand better some of the performance differences I'm seeing in various situations (and hopefully use that contribute some patches upstream!). My use case I'm exploring is using ffmpeg to decode video to pass to another process. I see dramatically different performance in a few situations that I'd like to better understand.
>
> My test setup is on a 5950x CPU (Windows), with hardware offload off just to keep things simple. I'm decoding a 4k video stream (h264 (Main), yuv420p(progressive), 3848x2168, 30 fps, 30 tbr, 1200k tbn),
>
> Here are some quick benchmarks:
> Null renderer ("-f null -") - 643 fps
> Yuv420p rawvideo ("-pix_fmt yuv420p -f rawvideo - > nul") - 290 fps
> Rgb8 rawvideo ("-pix_fmt rgb8 -f rawvideo - > nul") - 276 fps
> Yuyv422 rawvideo ("-pix_fmt yuyv422 -f rawvideo - > nul) - 173 fps
> Rgb24 rawvideo ("-pix_fmt rgb24 -f rawvideo - > nul") - 128 fps
> Rgba rawvideo ("-pix_fmt rgba -f rawvideo - > nul") - 102 fps
>
> This brought up some a few questions:
> - What does the null renderer skip that costs so much once we want to get video out? What is the native format out of x264?
> - For the RGB formats, it almost seems proportional to the output data size. I'd imagine the main calculation to generate RGB from YUV is similar, and just the layout/bits are different. But RGB8 is 2.5x faster than RGBA, even though one would think the extra bit shifting might add some overhead.
> - RGB formats seem more expensive than YUV formats, but more than I'd expect. Does ffmpeg use any vectorized implementations to convert between the two?
>
> I also noticed some other things like ffmpeg being very sensitive to the latency of the thing on the other end of the pipe. Is it possible to increase the buffering that ffmpeg does on the output side?
>
> Let me know if I can provide any more information. At some point I'll get out the profiler myself, but hoped to see if I could get some insight into anything obvious first.
>
> Thanks!
> Eric Gross
>
> INTERNAL - NI CONFIDENTIAL
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
As far as the documentation goes, the NULL renderer does not generate 
output. So it only has to read (and probably decode) the input.

For all the other formats, some amount of recoding has to be done after 
decoding the input stream depending on the format of the h264 stream. 
Read more about that here:
https://en.wikipedia.org/wiki/YUV#Y.E2.80.B2UV420p_.28and_Y.E2.80.B2V12_or_YV12.29_to_RGB888_conversion