[FFmpeg-devel] [PATCH 1/2] ffbuild: compose linker response files in a loop

[Gyan Doshi]

On 2025-03-29 11:52 pm, Martin Storsjö wrote:
> On Sat, 29 Mar 2025, Gyan Doshi wrote:
>
>>> Did you not try to use GNU make's flie function?
>>
>> I just benched this and it ranges from 1m28.093s to 1m29.971s (5% 
>> faster) for the lavc targets.
>> However, this was added in make 4.0. Are we supporting older make?
>
> Yes, we generally do support older GNU make; macOS (even the latest 
> versions) only ships with GNU make 3.81.
>
> Regarding measuring the runtime cost of this change; measuring the 
> whole build time is quite uninteresting, the interesting bit is 
> measuring the time to build e.g. an .a library on its own. So after a 
> full build, I do "rm libavcodec/libavcodec.a; time make 
> libavcodec/libavcodec.a". This change raises that time from ~3.5 
> seconds to ~3.8 seconds. However do note that this is on a quite slow 
> system in itself; without the "rm", it takes make 2.3 seconds just to 
> figure out that nothing needs to be done.
>
> So on that level, the change indeed is mostly tolerable.
>
> However - this is very quick as long as "echo" is a shell builtin. If 
> "echo" turns out to be an external executable instead of the shell 
> builtin (which we can simulate by calling "/usr/bin/echo" instead of 
> "echo"), then this suddenly takes >16 seconds rather than the earlier 
> <4 seconds. And that's quite a steep price to pay.
>
> As noted before, this is only a fix for a potential, hypothetical 
> problem. The fix is inexpensive in the case of a builtin echo, where 
> we don't need the fix anyway. For the case of an external echo, where 
> we potentially could need the fix, the fix is quite expensive though.
>
> But even with the external /usr/bin/echo (on msys2), I still can 
> produce a very long (>32k) .objs file with only one single invocation 
> of /usr/bin/echo. So we don't actually have this problem even in that 
> case.
>
> So given that there are multiple concerns about the performance about 
> this, and the problem that it tries to fix is entirely hypothetical at 
> the moment, I would suggest that we skip this fix for now.
>
> If someone actually manages to hit the problem in some setup and can 
> tell us about it, we could reconsider of course.

Ok, I'll skip the piecewise patch.

But I'll note that just the linking step in isolation is not the 
relevant benchmark here. Most users who are not doing active ffmpeg 
development are building the whole thing. That means thousands of .o 
files. followed by linking external and internal libs.
So what they will see with an echo utility is closer to 3m30s vs 3m42s 
than 4s vs 16s, which is a minimal change for someone not iterating app 
development.

Regards,
Gyan