[FFmpeg-devel] [PATCH 0/5] replace scale2ref by scale=rw:rh

[Gyan Doshi]

On 2024-04-24 04:21 pm, Niklas Haas wrote:
> As discussed in my previous series for fixing scale2ref[1], this filter
> is fundamentally broken, and the only real fix would be to switch to
> activate(), or ideally FFFrameSync.
>
> [1] https://ffmpeg.org//pipermail/ffmpeg-devel/2024-March/323382.html
>
> The main thing making this difficult is the fact that scale2ref also
> wants to output ref frames to its secondary output, which FFFrameSync
> does not support, and which is ultimately at least part of the root
> cause of trac #10795.
>
> Since this is in principle completely unnecessary (users can just
> 'split' the ref input and have it be consumed by vf_scale), and to make
> the design of this filter a bit more robust and maintainable, switch to
> an approach where vf_scale itself gains the ability to reference
> a secondary input stream, using the "ref_*" series of variables.
>
> This makes the current [i][ri]scale2ref[o][ro] equivalent to the only
> slightly more verbose [ri]split[t][ro]; [i][t]scale=rw:rh[o]. (And
> conversely, it is no longer necessary to use nullsink to consume an
> unused [ro])

In principle, a good idea, but how does this impact memory use and speed 
in the not-so-uncommon scenario where multiple overlay targets are 
scaled 2 ref and then overlaid
e.g.

in current flow:

[a][base]scale2ref[a][ref];
[b][ref]scale2ref[b][ref[;
[c][ref]scale2ref[c][ref[;
[d][ref]scale2ref[d][ref[;
[ref][a]overlay[ref];
[ref][b]overlay[ref];
[ref][c]overlay[ref];
[ref][d]overlay[ref];

in new flow:

[base]split=5[base][refa][refb][refc][refd];
[a][refa]scale[a];
[b][refb]scale[b];
[c][refc]scale[c];
[d][refd]scale[d];
[base][a]overlay[outa];
[outa][b]overlay[outb];
[outb][c]overlay[outc];
[outc][d]overlay[out];


Regards,
Gyan