[FFmpeg-devel] 32bit transcoding app running out of memory

[Simone Donadini]

Hi Ronald,

actually the problem was occurring transcoding quicktime files with ProRes settings and not with h264.
Changing the number of threads would not solve the problem.
I noticed that memory usage would keep growing as long i reached a certain point at which i would get a message like "Delay between the first packet and last packet in the muxing queue is 10004900 > 10000000: forcing output".  When transcoding 8k files i would not get to this point as at 4GB the app would crash.
The solution at the moment is to set the output format context's "max_interleave_delta" at a value lower than 10000000 after the output context is allocated and before avformat_write_header(). Doing this i am forcing the encoder to output frames, memory usage is kept steady, and i managed to transcode 8k files with no errors.
But is this a correct workaround? Or should i set some flag too?
I tried also to set the output context flag to AVFMT_FLAG_SHORTEST but i am not getting the same result. 

Thank you,
bye,
Simone.


________________________________________
From: Simone Donadini
Sent: Friday, March 15, 2019 11:42 AM
To: FFmpeg development discussions and patches
Subject: RE: [FFmpeg-devel] 32bit transcoding app running out of memory

Hi Ronald,
yes, we are using our own codec wrapped inside FFmpeg.
Thank you for your suggestion, i will try with limiting the number of threads launched by FFmpeg.
Simone.

________________________________________
From: ffmpeg-devel [ffmpeg-devel-bounces at ffmpeg.org] on behalf of Ronald S. Bultje [rsbultje at gmail.com]
Sent: Friday, March 15, 2019 11:24 AM
To: FFmpeg development discussions and patches
Subject: Re: [FFmpeg-devel] 32bit transcoding app running out of memory

Hi,

On Thu, Mar 14, 2019 at 7:52 AM Simone Donadini <
Simone.Donadini at avolites.com> wrote:

> > 2019-03-14 11:28 GMT+01:00, Simone Donadini <
> Simone.Donadini at avolites.com>:
> > > While transcoding video files with larger resolution (8K) the app,
> > > which is 32bit, will run out of memory (>4GB).
> > > It does not look like a memory leak, as memory usage will grow as long
> as
> > > transcoding a certain amount of frames and then it will stay steady. It
> > > looks like the amount of memory usage depends on the resolution of the
> >
> > > frame, but even with a 8K frame using >4GB is not expected.
> >
> > Why do you think so?
>
> With our codec one encoded frame 7680x4320 should be ~100MB.
>

Am I right in interpreting this as that you're not actually using an
existing FFMpeg encoder, but rather your own codec's encoder wrapped into
FFmpeg by yourself?

> h264 8K @60fps

I think several dozen frame allocations in the decoder (b/c of frame
threading) can happen, depending on the machine. 8*4*1.5*2=96MB/frame
(assuming 10bits 8K, you didn't mention), so for 3 dozen (assuming
n_threads=20-30), we're talking 3.6GB. To limit, use ffmpeg -thread_type
none -i .. or ffmpeg -threads $n -i .. where $n is a smaller number (e.g. 2
or 4) than the default (which is system-dependent).

Ronald
_______________________________________________
ffmpeg-devel mailing list
ffmpeg-devel at ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-devel