[FFmpeg-user] frame buffering in split graph

[Adrien F]

Hello,

I am facing a situation where memory grows too much to execute a 'complex_filter'. I found a workaround but I'd like to have your thoughts on a more elegant solution.

The pipeline is basically (see attached file script_no_bench.txt):

- one input stream
- preprocess (eg. eq=saturation=1.1)

- split the input stream in K parts
- trim each part in disjoint temporal segments (eg. frames 0 to 9, 10 to 19, 20 to 29 ...)
- eventually process each segment independently (eg. crop)
- join each processed stream (eg. hstack)
- output

The problem comes from the fact that ffmpeg is processing and *buffering* all frames from the (K-1) parts before being able to output the first one from the join part. And it can grow fast (with a toy example of 10 slices, 100xRGBAxHD frames each, it goes around 7GB).

As far as I understand ffmpeg only has one "read pointer" on the file and process it sequentially, hence the current behavior.

One workaround would be to open the input file in K streams, each shifted by some time with '-ss' option, but this would require to duplicate the (potentially long) preprocessing part, and the '-ss' option is not frame accurate.

The workaround I currently use is to render each splited/trimmed/processed part in a first pass, then open all inputs and join/postprocess them in a second pass. But I am not very satisfied by the intermediate rendering to disk.

On a side note, the preprocess/process/postprocess parts given are only examples, for the sake of illustration. It can potentially be long processes, some of which might happens on GPU VRAM (OpenCL), hence the memory constraints.
So, do you have a more elegant solution to propose, which would avoid memory grow and intermediate rendering to disk?

Adrien F
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: script_no_bench.txt
URL: <https://ffmpeg.org/pipermail/ffmpeg-user/attachments/20220616/5737871f/attachment.txt>