[FFmpeg-devel] [PATCH 1/2] ffbuild: compose linker response files in a loop

[Ramiro Polla]

On Sun, Mar 30, 2025 at 7:02 AM Gyan Doshi <ffmpeg at gyani.pro> wrote:
> On 2025-03-29 11:52 pm, Martin Storsjö wrote:
> > On Sat, 29 Mar 2025, Gyan Doshi wrote:
> >
> >>> Did you not try to use GNU make's flie function?
> >>
> >> I just benched this and it ranges from 1m28.093s to 1m29.971s (5%
> >> faster) for the lavc targets.
> >> However, this was added in make 4.0. Are we supporting older make?
> >
> > Yes, we generally do support older GNU make; macOS (even the latest
> > versions) only ships with GNU make 3.81.
> >
> > Regarding measuring the runtime cost of this change; measuring the
> > whole build time is quite uninteresting, the interesting bit is
> > measuring the time to build e.g. an .a library on its own. So after a
> > full build, I do "rm libavcodec/libavcodec.a; time make
> > libavcodec/libavcodec.a". This change raises that time from ~3.5
> > seconds to ~3.8 seconds. However do note that this is on a quite slow
> > system in itself; without the "rm", it takes make 2.3 seconds just to
> > figure out that nothing needs to be done.
> >
> > So on that level, the change indeed is mostly tolerable.
> >
> > However - this is very quick as long as "echo" is a shell builtin. If
> > "echo" turns out to be an external executable instead of the shell
> > builtin (which we can simulate by calling "/usr/bin/echo" instead of
> > "echo"), then this suddenly takes >16 seconds rather than the earlier
> > <4 seconds. And that's quite a steep price to pay.
> >
> > As noted before, this is only a fix for a potential, hypothetical
> > problem. The fix is inexpensive in the case of a builtin echo, where
> > we don't need the fix anyway. For the case of an external echo, where
> > we potentially could need the fix, the fix is quite expensive though.
> >
> > But even with the external /usr/bin/echo (on msys2), I still can
> > produce a very long (>32k) .objs file with only one single invocation
> > of /usr/bin/echo. So we don't actually have this problem even in that
> > case.
> >
> > So given that there are multiple concerns about the performance about
> > this, and the problem that it tries to fix is entirely hypothetical at
> > the moment, I would suggest that we skip this fix for now.
> >
> > If someone actually manages to hit the problem in some setup and can
> > tell us about it, we could reconsider of course.

This sounds reasonable.

> Ok, I'll skip the piecewise patch.

Ok, and thank you for digging deeper into the issue.

Ramiro