[FFmpeg-devel] [PATCH 1/2] ffbuild: compose linker response files in a loop

[Christopher Snowhill]

On Sat Mar 29, 2025 at 10:02 PM PDT, Gyan Doshi wrote:
>
>
> On 2025-03-29 11:52 pm, Martin Storsjö wrote:
>> On Sat, 29 Mar 2025, Gyan Doshi wrote:
>>
>>>> Did you not try to use GNU make's flie function?
>>>
>>> I just benched this and it ranges from 1m28.093s to 1m29.971s (5% 
>>> faster) for the lavc targets.
>>> However, this was added in make 4.0. Are we supporting older make?
>>
>> Yes, we generally do support older GNU make; macOS (even the latest 
>> versions) only ships with GNU make 3.81.
>>
>> Regarding measuring the runtime cost of this change; measuring the 
>> whole build time is quite uninteresting, the interesting bit is 
>> measuring the time to build e.g. an .a library on its own. So after a 
>> full build, I do "rm libavcodec/libavcodec.a; time make 
>> libavcodec/libavcodec.a". This change raises that time from ~3.5 
>> seconds to ~3.8 seconds. However do note that this is on a quite slow 
>> system in itself; without the "rm", it takes make 2.3 seconds just to 
>> figure out that nothing needs to be done.
>>
>> So on that level, the change indeed is mostly tolerable.
>>
>> However - this is very quick as long as "echo" is a shell builtin. If 
>> "echo" turns out to be an external executable instead of the shell 
>> builtin (which we can simulate by calling "/usr/bin/echo" instead of 
>> "echo"), then this suddenly takes >16 seconds rather than the earlier 
>> <4 seconds. And that's quite a steep price to pay.
>>
>> As noted before, this is only a fix for a potential, hypothetical 
>> problem. The fix is inexpensive in the case of a builtin echo, where 
>> we don't need the fix anyway. For the case of an external echo, where 
>> we potentially could need the fix, the fix is quite expensive though.
>>
>> But even with the external /usr/bin/echo (on msys2), I still can 
>> produce a very long (>32k) .objs file with only one single invocation 
>> of /usr/bin/echo. So we don't actually have this problem even in that 
>> case.
>>
>> So given that there are multiple concerns about the performance about 
>> this, and the problem that it tries to fix is entirely hypothetical at 
>> the moment, I would suggest that we skip this fix for now.
>>
>> If someone actually manages to hit the problem in some setup and can 
>> tell us about it, we could reconsider of course.
>
> Ok, I'll skip the piecewise patch.
>
> But I'll note that just the linking step in isolation is not the 
> relevant benchmark here. Most users who are not doing active ffmpeg 
> development are building the whole thing. That means thousands of .o 
> files. followed by linking external and internal libs.
> So what they will see with an echo utility is closer to 3m30s vs 3m42s 
> than 4s vs 16s, which is a minimal change for someone not iterating app 
> development.

I'm building on a Mac, so I'll either be using the Make supplied with
Xcode, or I'll have to install a newer gmake with Homebrew. I'm also
using overrides so the binaries will work on anything as old as 10.13.
(or 11.0 for Apple Silicon)

At least 10-15 seconds of each architecture's build process is running
the configure script with no feedback whatsoever until it completes.

>
> Regards,
> Gyan
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".