[FFmpeg-devel] FFmpeg Issue - looking for help!

Mon Dec 10 22:20:50 CET 2012

Just to update everyone, the hardware added Sunday morning appears to have
addressed the issue.  We had another busy ffmpeg day today and the web
server has been performing great.

Thanks,
Ethan

On Mon, Dec 10, 2012 at 1:23 AM, Ethan Harlow <eharlow at omnisg.com> wrote:

> Hi Michael,
>
> OMNI SG has a .NET web application that facilitates file uploads by end
> users.  The app accepts files can be of all types, but most popular uploads
> are docs (.pdf, .doc, .docx), video (.mov, .wmv, .mpg, .mp4), audio (.mp3),
> and images (.jpg, .jpeg, .png, .gif).
>
> When a user uploads a supported type of image or video, the application
> invokes FFMPEG to create two additional files:  i) a thumbnail ii) a 'web'
> version of the original; so for video, we create an .flv.
>
> *Environment*:
>
> We host at Rackspace, run Windows, and the production environment
> comprises:
>
> 1 web server (IIS7) - 4 GB RAM, 2 CPU
> 1 database server (SQL Server 2008 standard) - 32 GB RAM, 8 CPU
> 1 firewall (Cisco 1 GB)
> cloud file storage for the uploaded files (original, thumb, and web
> versions).
>
>
> FFMPEG version:
>
> N-34549-g13b7781, built on Nov 6, 2011 with gcc 4.6.1
>
>
> Web server specs:
>
> Microsoft Windows Server 2008 R2 Enterprise
> Quad-Core AMD Opteron™ processor 2374 HE ~2300 Mhz (2 cores)
> RAM: 4Gb
>
>
> *Issue*:  Over the past 1 week (Dec 3 to 7) our web server performance
> degraded for extended periods - to the end users the application appeared
> unresponsive.  During the unresponsive periods, we saw ffmpeg.exe processes
> running on our web server collectively using 50% to 100% of the web
> server's CPU.  We saw up to 3 separate ffmpeg.exe processing
> simultaneously, but usually not more than 2 simultaneously.
>
> *Analysis*:  We have a real time application monitoring tool (New Relic).
>  See the images below for an example. We can correlate the high CPU from
> ffmpeg periods with periods of slow/unresponsive web application.  During
> unresponsive times, the web server's CPU is driven up to 100% and stays
> there for several minutes.  Once ffmpeg.exe processes completed, the server
> would start responding again.
>
> The amount of file processing activity last week was far above anything
> we've seen since we started monitoring with New Relic.  In the first 5
> business days of December, we had 43 GB of source video or image files be
> processed by FFMPEG.  Each day, activity is typically concentrated over
> about 9 hours, so last week FFMPEG, running on the web server, almost 1 GB
> of video/images was processed every hour from 9 AM EST to 6 PM EST.
>
> *Current Theory*: In general, the web server we were running last week is
> kind of old and undersized for doing lots of web serving and lots of file
> processing simultaneously.  An active FFMPEG should not run on an active
> web/application server because FFMPEG is CPU intensive.  When the web
> application and FFMPEG are active, they will battle for CPU.  At a certain
> threshold volume (of file processing), end users will suffer with slow page
> responses.  I think we found and went beyond that threshold last week.
>
> *Action Taken*:  Yesterday morning we increased the # of CPUs on our web
> server from 2 to 4 and increased our the RAM from 4 to 8 GB.  Since we host
> at Rackspace, this was pretty easy and we pay by the minute for those
> resources and can go up or down without too much hassle.
>
> *Action Planned*:  Move file processing (creating the FLVs and JPGs) to a
> separate box (from the web server) and decouple file processing from web
> application.
>
> *Questions*:
>
> 1. We were advised FFMPEG can run multi-threaded.  We're not sure i) if we
> are running a version of FFMPEG that can be run multi-threaded, ii) how to
> enable/configure it to be iii) does application code that invokes FFMPEG
> needs to know/use multi-threading techniques, or iv) can we limit it to run
> across some CPUs but not all (so if it does consume all the CPU it can, it
> will still leave a couple for the web server's other responsibilities).
>
> 2. If anything above jumps out as wrong or if you have advice or
> recommendations, we're all ears.
>
>
>
> [image: Inline image 2]
>
> [image: Inline image 1]
>
> On Sun, Dec 9, 2012 at 9:45 AM, Michael Niedermayer <michaelni at gmx.at>wrote:
>
>> Hi
>>
>> On Sat, Dec 08, 2012 at 08:02:24AM -0500, Pat Seery wrote:
>> > Michael -
>> > Got your info from http://ffmpeg.org/consulting.html.
>> > We have an web application using FFmpeg.  We host our dedicated servers
>> > with RackSpace - 1 web & 1 SQL.
>> > Ethan can provide additional information on our issue but at the moment
>> we
>> > need immediate advise on how to best correct this critical problem.
>>
>> what critical problem ?
>>
>>
>> [...]
>> --
>> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>>
>> Rewriting code that is poorly written but fully understood is good.
>> Rewriting code that one doesnt understand is a sign that one is less smart
>> then the original author, trying to rewrite it will not make it better.
>>
>
>
>
> --
> *Ethan Harlow  |  *Client Support Services
>
> office: (301) 869-6890 x 114
> mobile: (571) 216-3014
> Skype: ethan.harlow
> eharlow at omnisg.com
>
> *OMNI Solutions Group, Inc.*
> www.omnisg.com
>
> *If you have a technology need or problem...*
> *                                 ... we have an OMNI solution.*
>
>

-- 
*Ethan Harlow  |  *Client Support Services

office: (301) 869-6890 x 114
mobile: (571) 216-3014
Skype: ethan.harlow
eharlow at omnisg.com

*OMNI Solutions Group, Inc.*
www.omnisg.com

*If you have a technology need or problem...*
*                                 ... we have an OMNI solution.*
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 249195 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20121210/4df9012f/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 265302 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20121210/4df9012f/attachment-0001.png>