[FFmpeg-devel] Output buffer size in ffmpeg.c

[Kostya]

On Sat, Jun 13, 2009 at 01:12:29PM +0100, M?ns Rullg?rd wrote:
> In ffmpeg.c, line 1926, the size of the output buffer for compressed
> data is calculated as 6*width*height+200.  The subsequent malloc is
> failing on my 64MB systems for some inputs, and is the cause of the
> redcode-demux FATE failure on AVR32.  This particular test passes if I
> enable memory overcommit.  This will not help on blackfin, however,
> since it has no MMU and hence overcommit is impossible.
> 
> Does the output buffer really need to be this big?  The only case I
> can think of that would need it is 48-bit raw.  Perhaps we should use
> a more intelligent estimate taking the codec into account.

IIRC, it was settled after experiments with XViD coding pure noise.
Also we do support 48-bit raw a bit now.

> At the very least, we should print an error message when this
> allocation fails.  It would have saved me some debugging time.  Patch
> for this sent separately.
> 
> -- 
> M?ns Rullg?rd
> mans at mansr.com