[FFmpeg-user] Lavfi video generator slow performance

[Carl Zwanzig]

Permit me to be skeptical, I've spent a lot of time working on storage systems.

On 4/5/2018 6:40 PM, Elliott Balsley wrote:
> Yes, this RAID can write at 3000MBps.
That doesn't tell us too much. Is that a raw sequential workload or onto a 
file system (and which one). (Have you benchmarked the system with a 
representative workload?)

What interface (SAS, iSCSI, 10g ether, FC) and protocol? A 12g (bits per 
second) SAS connection can only move 1500 MB/sec before you remove overhead, 
so as a guess it's probably only good for maybe 800MB/s of data. Then 
there's file-system overhead and other users. NFS over 10g ethernet will 
come in slower, of course. And block size matters, too.

In short, I'd be looking very closely at the storage system and connections. 
fio is a decent benchmarking tool, there are others.


And after I wrote all that above... does ffmpeg do synchronous writes? Even 
with file system buffering, they're -slow-; most benchmarks use queued I/O 
with long-ish queues (64/128). Also, what is ffmpeg's write buffer size? If 
it's writing 512 byte blocks, it'll run a lot slower than if it's writing 
64k blocks (or 128k). (I'm not diving into the code to check myself.)

Later,

z!