[FFmpeg-user] Understanding the performance of getting decoded video out of ffmpeg

[Eric Gross]

Hi all,

I'd love to understand better some of the performance differences I'm seeing in various situations (and hopefully use that contribute some patches upstream!). My use case I'm exploring is using ffmpeg to decode video to pass to another process. I see dramatically different performance in a few situations that I'd like to better understand.

My test setup is on a 5950x CPU (Windows), with hardware offload off just to keep things simple. I'm decoding a 4k video stream (h264 (Main), yuv420p(progressive), 3848x2168, 30 fps, 30 tbr, 1200k tbn),

Here are some quick benchmarks:
Null renderer ("-f null -") - 643 fps
Yuv420p rawvideo ("-pix_fmt yuv420p -f rawvideo - > nul") - 290 fps
Rgb8 rawvideo ("-pix_fmt rgb8 -f rawvideo - > nul") - 276 fps
Yuyv422 rawvideo ("-pix_fmt yuyv422 -f rawvideo - > nul) - 173 fps
Rgb24 rawvideo ("-pix_fmt rgb24 -f rawvideo - > nul") - 128 fps
Rgba rawvideo ("-pix_fmt rgba -f rawvideo - > nul") - 102 fps

This brought up some a few questions:
- What does the null renderer skip that costs so much once we want to get video out? What is the native format out of x264?
- For the RGB formats, it almost seems proportional to the output data size. I'd imagine the main calculation to generate RGB from YUV is similar, and just the layout/bits are different. But RGB8 is 2.5x faster than RGBA, even though one would think the extra bit shifting might add some overhead.
- RGB formats seem more expensive than YUV formats, but more than I'd expect. Does ffmpeg use any vectorized implementations to convert between the two?

I also noticed some other things like ffmpeg being very sensitive to the latency of the thing on the other end of the pipe. Is it possible to increase the buffering that ffmpeg does on the output side?

Let me know if I can provide any more information. At some point I'll get out the profiler myself, but hoped to see if I could get some insight into anything obvious first.

Thanks!
Eric Gross

INTERNAL - NI CONFIDENTIAL