[FFmpeg-user] Multiple outputs worse performance than multiple instances

[Henk D. Schoneveld]

On 23 Oct 2013, at 10:55, Carl Lindqvist <lulebo at gmail.com> wrote:

> 2013/10/22 Henk D. Schoneveld <belcampo at zonnet.nl>:
>> 
>> On Oct 22, 2013, at 9:33 AM, littlebat <dashing.meng at gmail.com> wrote:
>> 
>>> On Mon, 21 Oct 2013 14:36:47 +0200
>>> Carl Lindqvist <lulebo at gmail.com> wrote:
>>> 
>>>> Sorry for the previous top post. Gmail is giving me a headache..
>>>> 
>>>> I did a small typo/brain fart, what I meant to say was:
>>>> 
>>>> Everything works perfectly, but cpu usage on the very fast server (16
>>>> cores) is quite bad. I can set -preset slow and get the same fps
>>>> encoding speed, but with higher cpu usage.
>> If you do a simple test on 1 file, just change threads 1, threads 2 …. threads 0, and note what each added core adds, you'll discover that every additional core adds LESS.
>> Your original multiple instances AFAIK will be the optimum you'll get out your CPU-cores.
>> Breaking up the source file to get 1 single core to encode 1 piece and then catting them together will be maximum encoding speed you'll get.
>> Henk
>>>> 
>>> Don't know more about multiple threads decoding or encoding, But I
>>> think you need understand some things:
>>> 
>>> 1, CPU usage is low, if there are some other bottle neck? e.g., disk IO?
>>> 
>>> 2, What kinds of codecs support multiple threads decoding or encoding?
>>> 
>>> 3, If a codecs support multiple threads decoding or encoding, if these
>>> threads can occupy all the CPUs' usage?
>>> 
>>> Just a clue. If we can't issue enough commands at same time to occupy
>>> all the CPU cores, and how to use the multiple decoding and encoding
>>> ability of ffmpeg to achieve this goal, I am also interest in this
>>> question :)
>>> _______________________________________________
>>> ffmpeg-user mailing list
>>> ffmpeg-user at ffmpeg.org
>>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
>> 
>> _______________________________________________
>> ffmpeg-user mailing list
>> ffmpeg-user at ffmpeg.org
>> http://ffmpeg.org/mailman/listinfo/ffmpeg-user
> 
> I will do some more tests today. One of the reasons I don't want to do
> encode with multiple instances of ffmpeg is that I don't want to tax
> the network. The source files are high bitrate mxf broadcast files,
> and the file system is shared, so 5 ffmpeg instances each reading the
> same file might impact other parts of the network. I would then have
> to copy the file locally before transcoding, which would negate some
> of the speed advantages. (Network speed is not a bottleneck;
> especially in my tests, as I have the file locally on hard drive when
> testing)
5 instances of highbitrate source file encode @ xfps how much % is that of your network bandwidth ?
Probably nothing to be concerned about.
> 
> As I understand it from the console output, ffmpeg spawns 5 seperate
> x264 encoders each with their own threads. I don't see why it should
> be slower than starting five instances of ffmpeg.
As mentioned in my previous post, multithreading DOESN’T scale well. Every additional core produces relatively less.
> I should be doing
> less work this way (only decode/deinterlace once, instead of five
> times), so I expected the speed to go up, not down. Maybe I am doing
> something weird in my filter graph that is inefficient?
I don’t think so, that’s why I suggested ‘simple encoding’ sample with only difference changing threads 1, threads 2 …. threads 0 to see what I mean.
> 
> 
> I will report if I find anything interesting.
> 
> /Carl
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-user