[FFmpeg-user] Multiple outputs worse performance than multiple instances

[Carl Lindqvist]

2013/10/22 Henk D. Schoneveld <belcampo at zonnet.nl>:
>
> On Oct 22, 2013, at 9:33 AM, littlebat <dashing.meng at gmail.com> wrote:
>
>> On Mon, 21 Oct 2013 14:36:47 +0200
>> Carl Lindqvist <lulebo at gmail.com> wrote:
>>
>>> Sorry for the previous top post. Gmail is giving me a headache..
>>>
>>> I did a small typo/brain fart, what I meant to say was:
>>>
>>> Everything works perfectly, but cpu usage on the very fast server (16
>>> cores) is quite bad. I can set -preset slow and get the same fps
>>> encoding speed, but with higher cpu usage.
> If you do a simple test on 1 file, just change threads 1, threads 2 …. threads 0, and note what each added core adds, you'll discover that every additional core adds LESS.
> Your original multiple instances AFAIK will be the optimum you'll get out your CPU-cores.
> Breaking up the source file to get 1 single core to encode 1 piece and then catting them together will be maximum encoding speed you'll get.
> Henk
>>>
>> Don't know more about multiple threads decoding or encoding, But I
>> think you need understand some things:
>>
>> 1, CPU usage is low, if there are some other bottle neck? e.g., disk IO?
>>
>> 2, What kinds of codecs support multiple threads decoding or encoding?
>>
>> 3, If a codecs support multiple threads decoding or encoding, if these
>> threads can occupy all the CPUs' usage?
>>
>> Just a clue. If we can't issue enough commands at same time to occupy
>> all the CPU cores, and how to use the multiple decoding and encoding
>> ability of ffmpeg to achieve this goal, I am also interest in this
>> question :)
>> _______________________________________________
>> ffmpeg-user mailing list
>> ffmpeg-user at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user

I will do some more tests today. One of the reasons I don't want to do
encode with multiple instances of ffmpeg is that I don't want to tax
the network. The source files are high bitrate mxf broadcast files,
and the file system is shared, so 5 ffmpeg instances each reading the
same file might impact other parts of the network. I would then have
to copy the file locally before transcoding, which would negate some
of the speed advantages. (Network speed is not a bottleneck;
especially in my tests, as I have the file locally on hard drive when
testing)

As I understand it from the console output, ffmpeg spawns 5 seperate
x264 encoders each with their own threads. I don't see why it should
be slower than starting five instances of ffmpeg. I should be doing
less work this way (only decode/deinterlace once, instead of five
times), so I expected the speed to go up, not down. Maybe I am doing
something weird in my filter graph that is inefficient?

I will report if I find anything interesting.

/Carl