[FFmpeg-devel] ffv1enc: question about "Cannot allocate worst case packet size, the encoding could fail"

[Paul B Mahol]

On 10/11/18, Jerome Martinez <jerome at mediaarea.net> wrote:
> Hi,
>
> Testing FFmpeg FFV1 encoder on big frames (more than 4K: 4300x3956), I
> have a warning for each frame encoded (so a lot of warnings!):
> [ffv1 @ 0000024a6bcfe880] Cannot allocate worst case packet size, the
> encoding could fail
>
> Checking avcodec/ffv1enc.c, it is due to the following lines:
>      int64_t maxsize =   AV_INPUT_BUFFER_MIN_SIZE
>                        + avctx->width*avctx->height*37LL*4;
> [...]
>     if (maxsize > INT_MAX - AV_INPUT_BUFFER_PADDING_SIZE - 32) {
>          av_log(avctx, AV_LOG_WARNING, "Cannot allocate worst case
> packet size, the encoding could fail\n");
>          maxsize = INT_MAX - AV_INPUT_BUFFER_PADDING_SIZE - 32;
>      }
>
> The value of maxsize is 2517614784, more than INT_MAX, sure, but I don't
> understand:
> - why does FFmpeg need to allocate 148 times the size of a frame?
> - why is having a number > INT_MAX an issue? modern machines are 64-bit
> and have 8+ GB of RAM, and in practice I currently saw no encoding
> failure on thousands of frames.
>
> Additionally I didn't get why maxsize is reduced ("only" need of 12
> times the size of the frame) in case of FFV1 version > 3 (experimental
> right now):
>      if (f->version > 3)
>          maxsize = AV_INPUT_BUFFER_MIN_SIZE +
> avctx->width*avctx->height*3LL*4;
>
> maxsize is used for calling ff_alloc_packet2(), which accepts 64-bit
> numbers.
>     if ((ret = ff_alloc_packet2(avctx, pkt, maxsize, 0)) < 0)
>          return ret;
>
> Is it possible to reduce this "37x4" multiplier without risk and/or
> remove maxsize reduction in case of sizeof(size_t) == 8 and/or show the
> warning only once?
>
> Issue can be reproduced with:
> ./ffmpeg.exe -f lavfi -i testsrc2=size=4300x3956 -t 0.040 -pix_fmt
> rgba64 rgba64.dpx
> ./ffmpeg.exe -i rgba64.dpx -c:v ffv1 rgba64.mkv

You are not using latest version of FFv1.

In latest version, encoder will not produce packets which are bigger than
input ones -- uncompressed raw. And instead if that happens it will encode
as raw video.

Theoretically output packets for old versions of FFv1 could be several
times bigger than raw input frame.