Turns out this has been discussed before and we're both right: http://lists.ffmpeg.org/pipermail/ffmpeg-user/2013-August/016861.html One video per core is the fastest way to encode, and it can become I/O bound as a result.