[FFmpeg-devel] [PATCH 00/13] [RFC] Reduce unnecessary recompilation
Martin Storsjö
martin at martin.st
Wed Feb 23 16:29:38 EET 2022
Hi,
When updating the ffmpeg source, one quite often ends up in a situation
where practically all of the codebase (or all of a library) gets rebuilt,
due to updates to headers that are included in most files.
In some cases, full rebuilds are warranted of course, but they could also
be avoided in many cases - e.g. such as if the minor/micro version of
a library has been bumped, or if a new component (codec, demuxer, filter
etc) has been enabled (or removed if reconfiguring with an older source
version). Very few source files are affected by exactly what the full
library version is, or by the full list of enabled components.
To avoid such rebuilds, I've got a proof of concept patchset that
splits headers, so that most source files avoid including the bits that
change often and that shouldn't affect how they are built.
- The version.h headers are split into a separate version_major.h which
contains only the major library version, and accompanying FF_API_*
defines. The main library headers only include version_major.h, and
files that need the exact version (e.g. LIB<name>_VERSION* or
LIB<name>_IDENT) can include version.h explicitly. This is a minor
break of the public API though, as definitions that used to be available
no longer are.
This works mostly fine for most libraries, but there's little point in
splitting libavutil/version.h, because LIBAVUTIL_VERSION_INT is used
in every source file that defines an AVClass.
By splitting version.h, and update to the minor/micro version numbers
of all libraries except avutil now would require recompiling 30
files instead of 1653 before.
(This change also should lower the barrier to and reduce the risk of
forgetting to bump the version numbers, which one otherwise often
postpones while working on a patch, as it forces rebuilds.)
- config.h is split into a separate config_components.h that includes the
list of enabled/disabled components (corresponding to $ALL_COMPONENTS
in configure). One could consider splitting up config.h even more, but
that probably gives less benefit compared to the amount of churn.
Surprisingly, a nontrivial number of source files do depend on the
state of specific encoders/decoders/components, so quite a few files
do end up requiring including config_components.h. (Also this change
can possibly break compilation of source files that require external
dependencies that I haven't tested.)
In practice, this reduces the number of rebuilt source files from
1979 to 193, if there's a change to the list of enabled components
but not to the rest of config.h.
What do you think - is it worth the slight churn to avoid pointless
rebuilds?
Martin Storsjö (13):
libavutil: Remove leftover uses of version.h
libavcodec: Remove unnecessary includes of version.h
libavformat: Remove unnecessary includes of version.h
libavdevice: Remove unnecessary includes of version.h
libavcodec: Split version.h
libavformat: Split version.h
libavdevice: Split version.h
libpostproc: Split version.h
libswresample: Split version.h
libswscale: Split version.h
libavfilter: Split version.h
doc: Add an entry to APIchanges about no longer implicitly including
version.h
configure: Use a separate config_components.h header for
$ALL_COMPONENTS
configure | 17 +++++++--
doc/APIchanges | 6 ++++
fftools/cmdutils.c | 7 ++++
fftools/ffmpeg.c | 1 +
fftools/ffplay.c | 1 +
fftools/ffprobe.c | 7 ++++
libavcodec/8svx.c | 1 +
libavcodec/Makefile | 1 +
libavcodec/a64multienc.c | 1 +
libavcodec/aac_ac3_parser.c | 1 +
libavcodec/aacenc.c | 1 +
libavcodec/aactab.c | 1 +
libavcodec/aarch64/h264cmc_neon.S | 1 +
libavcodec/ac3_parser.c | 1 +
libavcodec/ac3dec.c | 1 +
libavcodec/ac3enc.c | 1 +
libavcodec/ac3enc_template.c | 1 +
libavcodec/adpcm.c | 1 +
libavcodec/adpcmenc.c | 1 +
libavcodec/allcodecs.c | 1 +
libavcodec/aptxdec.c | 1 +
libavcodec/aptxenc.c | 1 +
libavcodec/arm/flacdsp_init_arm.c | 1 +
libavcodec/arm/h264cmc_neon.S | 1 +
libavcodec/assdec.c | 1 +
libavcodec/assenc.c | 1 +
libavcodec/asvdec.c | 1 +
libavcodec/asvenc.c | 1 +
libavcodec/audiotoolboxdec.c | 1 +
libavcodec/av1dec.c | 1 +
libavcodec/avcodec.c | 1 +
libavcodec/avcodec.h | 2 +-
libavcodec/binkaudio.c | 1 +
libavcodec/bintext.c | 1 +
libavcodec/bsf.c | 1 +
libavcodec/codec.h | 2 +-
libavcodec/cyuv.c | 1 +
libavcodec/dpxenc.c | 1 +
libavcodec/flashsv.c | 1 +
libavcodec/g726.c | 1 +
libavcodec/gsmdec.c | 1 +
libavcodec/h263dec.c | 1 +
libavcodec/h264_slice.c | 1 +
libavcodec/h264dec.c | 2 ++
libavcodec/hevcdec.c | 1 +
libavcodec/huffyuvdec.c | 1 +
libavcodec/huffyuvenc.c | 1 +
libavcodec/idctdsp.c | 1 +
libavcodec/iff.c | 1 +
libavcodec/imc.c | 1 +
libavcodec/ituh263dec.c | 1 +
libavcodec/ituh263enc.c | 1 +
libavcodec/j2kenc.c | 1 +
libavcodec/lcldec.c | 1 +
libavcodec/libgsmdec.c | 1 +
libavcodec/libgsmenc.c | 1 +
libavcodec/libopencore-amr.c | 1 +
libavcodec/libvorbisenc.c | 1 +
libavcodec/libvpx.c | 1 +
libavcodec/libvpxdec.c | 1 +
libavcodec/libvpxenc.c | 1 +
libavcodec/libx264.c | 1 +
libavcodec/me_cmp.c | 1 +
libavcodec/mediacodec.c | 1 -
libavcodec/mediacodec_wrapper.c | 1 -
libavcodec/metasound_data.c | 1 +
libavcodec/mjpegdec.c | 1 +
libavcodec/mjpegenc.c | 1 +
libavcodec/mjpegenc_common.c | 1 +
libavcodec/mlpdec.c | 1 +
libavcodec/mlpenc.c | 1 +
libavcodec/mpeg12dec.c | 1 +
libavcodec/mpeg12enc.c | 1 +
libavcodec/mpeg4videodec.c | 1 +
libavcodec/mpeg4videoenc.c | 1 +
libavcodec/mpegaudiodec_fixed.c | 1 +
libavcodec/mpegaudiodec_float.c | 1 +
libavcodec/mpegvideo.c | 1 +
libavcodec/mpegvideo_enc.c | 1 +
libavcodec/mpegvideo_motion.c | 1 +
libavcodec/msmpeg4.c | 1 +
libavcodec/msmpeg4dec.c | 1 +
libavcodec/mvcdec.c | 1 +
libavcodec/options.c | 1 +
libavcodec/options_table.h | 2 +-
libavcodec/opus_pvq.c | 1 +
libavcodec/packet.h | 2 +-
libavcodec/pcm.c | 1 +
libavcodec/pngdec.c | 1 +
libavcodec/pnmdec.c | 1 +
libavcodec/pnmenc.c | 1 +
libavcodec/proresdec2.c | 1 +
libavcodec/pthread_frame.c | 2 +-
libavcodec/qpeldsp.c | 1 +
libavcodec/r210dec.c | 1 +
libavcodec/r210enc.c | 1 +
libavcodec/rv34_parser.c | 1 +
libavcodec/sonic.c | 1 +
libavcodec/sp5xdec.c | 1 +
libavcodec/speedhq.c | 1 +
libavcodec/speedhqenc.c | 1 +
libavcodec/srtdec.c | 1 +
libavcodec/srtenc.c | 1 +
libavcodec/textdec.c | 1 +
libavcodec/tiffenc.c | 1 +
libavcodec/v408dec.c | 1 +
libavcodec/v408enc.c | 1 +
libavcodec/vaapi_encode_h264.c | 1 +
libavcodec/vc1dec.c | 1 +
libavcodec/vc1dsp.c | 1 +
libavcodec/version.h | 28 ++-------------
libavcodec/version_major.h | 55 ++++++++++++++++++++++++++++++
libavcodec/videotoolbox.c | 1 +
libavcodec/vorbis_parser.c | 1 +
libavcodec/vp3.c | 1 +
libavcodec/vp56dsp.c | 1 +
libavcodec/vp8.c | 1 +
libavcodec/vp8dsp.c | 1 +
libavcodec/vp9.c | 1 +
libavcodec/wmadec.c | 1 +
libavcodec/wmaenc.c | 1 +
libavcodec/x86/blockdsp_init.c | 1 -
libavcodec/x86/flacdsp_init.c | 1 +
libavcodec/x86/hpeldsp_init.c | 1 +
libavcodec/xvmc.h | 1 -
libavdevice/Makefile | 1 +
libavdevice/android_camera.c | 2 --
libavdevice/avdevice.c | 1 +
libavdevice/avdevice.h | 2 +-
libavdevice/pulse_audio_dec.c | 1 +
libavdevice/pulse_audio_enc.c | 1 +
libavdevice/version.h | 10 ++----
libavdevice/version_major.h | 37 ++++++++++++++++++++
libavfilter/Makefile | 1 +
libavfilter/aeval.c | 1 +
libavfilter/af_afade.c | 1 +
libavfilter/af_agate.c | 1 +
libavfilter/af_biquads.c | 1 +
libavfilter/af_sidechaincompress.c | 1 +
libavfilter/avf_showspectrum.c | 1 +
libavfilter/avf_showwaves.c | 1 +
libavfilter/avfilter.c | 1 +
libavfilter/avfilter.h | 2 +-
libavfilter/f_bench.c | 1 +
libavfilter/f_cue.c | 1 +
libavfilter/f_drawgraph.c | 1 +
libavfilter/f_graphmonitor.c | 1 +
libavfilter/f_interleave.c | 1 +
libavfilter/f_latency.c | 1 +
libavfilter/f_loop.c | 1 +
libavfilter/f_metadata.c | 1 +
libavfilter/f_perms.c | 1 +
libavfilter/f_realtime.c | 1 +
libavfilter/f_reverse.c | 1 +
libavfilter/f_segment.c | 1 +
libavfilter/f_select.c | 1 +
libavfilter/f_sendcmd.c | 1 +
libavfilter/f_sidedata.c | 1 +
libavfilter/internal.h | 1 -
libavfilter/lut3d.h | 1 +
libavfilter/setpts.c | 1 +
libavfilter/settb.c | 1 +
libavfilter/src_movie.c | 1 +
libavfilter/trim.c | 1 +
libavfilter/version.h | 13 ++-----
libavfilter/version_major.h | 42 +++++++++++++++++++++++
libavfilter/vf_aspect.c | 1 +
libavfilter/vf_blend.c | 1 +
libavfilter/vf_colorconstancy.c | 1 +
libavfilter/vf_colorkey.c | 1 +
libavfilter/vf_convolution.c | 1 +
libavfilter/vf_convolve.c | 1 +
libavfilter/vf_drawbox.c | 1 +
libavfilter/vf_extractplanes.c | 1 +
libavfilter/vf_format.c | 1 +
libavfilter/vf_histogram.c | 1 +
libavfilter/vf_identity.c | 1 +
libavfilter/vf_lut.c | 1 +
libavfilter/vf_lut2.c | 1 +
libavfilter/vf_mix.c | 1 +
libavfilter/vf_neighbor.c | 1 +
libavfilter/vf_premultiply.c | 1 +
libavfilter/vf_setparams.c | 1 +
libavfilter/vf_stack.c | 1 +
libavfilter/vf_xmedian.c | 1 +
libavfilter/vsrc_testsrc.c | 1 +
libavformat/Makefile | 1 +
libavformat/ac3dec.c | 1 +
libavformat/alp.c | 1 +
libavformat/amr.c | 1 +
libavformat/apm.c | 1 +
libavformat/aptxdec.c | 1 +
libavformat/argo_asf.c | 1 +
libavformat/argo_cvg.c | 1 +
libavformat/asfenc.c | 1 +
libavformat/au.c | 1 +
libavformat/av1dec.c | 1 +
libavformat/avformat.h | 2 +-
libavformat/avidec.c | 1 +
libavformat/avienc.c | 1 +
libavformat/avio.h | 2 +-
libavformat/bintext.c | 1 +
libavformat/bit.c | 1 +
libavformat/codec2.c | 1 +
libavformat/concat.c | 1 +
libavformat/dashenc.c | 1 +
libavformat/demux.c | 1 +
libavformat/file.c | 1 +
libavformat/flacenc.c | 1 +
libavformat/format.c | 1 +
libavformat/framehash.c | 1 +
libavformat/g726.c | 1 +
libavformat/gopher.c | 1 +
libavformat/hashenc.c | 1 +
libavformat/hls.c | 1 +
libavformat/hlsenc.c | 1 +
libavformat/http.c | 1 +
libavformat/ilbc.c | 1 +
libavformat/img2dec.c | 1 +
libavformat/img2enc.c | 1 +
libavformat/kvag.c | 1 +
libavformat/matroskadec.c | 1 +
libavformat/matroskaenc.c | 2 ++
libavformat/mj2kdec.c | 1 +
libavformat/mlpdec.c | 1 +
libavformat/mmf.c | 2 ++
libavformat/mov.c | 1 +
libavformat/movenc.c | 3 ++
libavformat/mpeg.c | 1 +
libavformat/mpegenc.c | 1 +
libavformat/mpegts.c | 1 +
libavformat/mux.c | 1 +
libavformat/mxfenc.c | 1 +
libavformat/network.c | 1 +
libavformat/nutenc.c | 1 +
libavformat/oggenc.c | 2 ++
libavformat/pcmdec.c | 1 +
libavformat/pcmenc.c | 1 +
libavformat/rawdec.c | 1 +
libavformat/rawenc.c | 1 +
libavformat/rawvideodec.c | 1 +
libavformat/riff.c | 1 +
libavformat/rtmpproto.c | 2 ++
libavformat/rtsp.c | 2 ++
libavformat/rtspdec.c | 2 ++
libavformat/sdp.c | 1 +
libavformat/segment.c | 1 +
libavformat/swfenc.c | 1 +
libavformat/url.h | 1 -
libavformat/utils.c | 1 +
libavformat/version.h | 23 ++-----------
libavformat/version_major.h | 52 ++++++++++++++++++++++++++++
libavformat/voc.c | 1 +
libavformat/wavdec.c | 1 +
libavformat/wavenc.c | 1 +
libavutil/common.h | 1 -
libavutil/internal.h | 1 -
libpostproc/Makefile | 1 +
libpostproc/postprocess.c | 1 +
libpostproc/postprocess.h | 2 +-
libpostproc/version.h | 3 +-
libpostproc/version_major.h | 31 +++++++++++++++++
libswresample/Makefile | 1 +
libswresample/swresample.c | 1 +
libswresample/swresample.h | 2 +-
libswresample/version.h | 3 +-
libswresample/version_major.h | 31 +++++++++++++++++
libswscale/Makefile | 1 +
libswscale/swscale.h | 2 +-
libswscale/swscale_internal.h | 1 -
libswscale/utils.c | 1 +
libswscale/version.h | 9 ++---
libswscale/version_major.h | 35 +++++++++++++++++++
273 files changed, 586 insertions(+), 100 deletions(-)
create mode 100644 libavcodec/version_major.h
create mode 100644 libavdevice/version_major.h
create mode 100644 libavfilter/version_major.h
create mode 100644 libavformat/version_major.h
create mode 100644 libpostproc/version_major.h
create mode 100644 libswresample/version_major.h
create mode 100644 libswscale/version_major.h
--
2.32.0 (Apple Git-132)
More information about the ffmpeg-devel
mailing list