[FFmpeg-devel] [PATCH 00/13] [RFC] Reduce unnecessary recompilation

[Martin Storsjö]

On Mon, 14 Mar 2022, Michael Niedermayer wrote:

> On Fri, Mar 11, 2022 at 02:17:42PM +0200, Martin Storsjö wrote:
>> On Wed, 23 Feb 2022, Martin Storsjö wrote:
>>
>>> When updating the ffmpeg source, one quite often ends up in a situation
>>> where practically all of the codebase (or all of a library) gets rebuilt,
>>> due to updates to headers that are included in most files.
>>>
>>> In some cases, full rebuilds are warranted of course, but they could also
>>> be avoided in many cases - e.g. such as if the minor/micro version of
>>> a library has been bumped, or if a new component (codec, demuxer, filter
>>> etc) has been enabled (or removed if reconfiguring with an older source
>>> version). Very few source files are affected by exactly what the full
>>> library version is, or by the full list of enabled components.
>>>
>>> To avoid such rebuilds, I've got a proof of concept patchset that
>>> splits headers, so that most source files avoid including the bits that
>>> change often and that shouldn't affect how they are built.
>>>
>>> - The version.h headers are split into a separate version_major.h which
>>>  contains only the major library version, and accompanying FF_API_*
>>>  defines. The main library headers only include version_major.h, and
>>>  files that need the exact version (e.g. LIB<name>_VERSION* or
>>>  LIB<name>_IDENT) can include version.h explicitly. This is a minor
>>>  break of the public API though, as definitions that used to be available
>>>  no longer are.
>>>
>>>  This works mostly fine for most libraries, but there's little point in
>>>  splitting libavutil/version.h, because LIBAVUTIL_VERSION_INT is used
>>>  in every source file that defines an AVClass.
>>>
>>>  By splitting version.h, and update to the minor/micro version numbers
>>>  of all libraries except avutil now would require recompiling 30
>>>  files instead of 1653 before.
>>>
>>>  (This change also should lower the barrier to and reduce the risk of
>>>  forgetting to bump the version numbers, which one otherwise often
>>>  postpones while working on a patch, as it forces rebuilds.)
>>>
>>> - config.h is split into a separate config_components.h that includes the
>>>  list of enabled/disabled components (corresponding to $ALL_COMPONENTS
>>>  in configure). One could consider splitting up config.h even more, but
>>>  that probably gives less benefit compared to the amount of churn.
>>>
>>>  Surprisingly, a nontrivial number of source files do depend on the
>>>  state of specific encoders/decoders/components, so quite a few files
>>>  do end up requiring including config_components.h. (Also this change
>>>  can possibly break compilation of source files that require external
>>>  dependencies that I haven't tested.)
>>>
>>>  In practice, this reduces the number of rebuilt source files from
>>>  1979 to 193, if there's a change to the list of enabled components
>>>  but not to the rest of config.h.
>>>
>>> What do you think - is it worth the slight churn to avoid pointless
>>> rebuilds?
>>
>> Ping - I never got any feedback on the general concept of this patchset; is
>> either of the refactorings worthwhile?
>
> I think its a good idea. Faster rebuilds & tests are always desirable

Pushed now, thanks!

// Martin