[MPlayer-dev-eng] recursive make considered harmful (was: Re: The new Makefiles)

[Michael Niedermayer]

On Tue, Jun 03, 2008 at 02:31:07AM +0200, Diego Biurrun wrote:
> On Tue, Jun 03, 2008 at 12:17:05AM +0300, Ivan Kalvachev wrote:
> > On 6/2/08, Diego Biurrun <diego at biurrun.de> wrote:
> > > On Mon, Jun 02, 2008 at 07:07:22PM +0200, Nico Sabbi wrote:
> > >> Il Monday 02 June 2008 17:18:49 Diego Biurrun ha scritto:
> > >> > I did read your patch when you sent it to me.  However, I do not
> > >> > remember every single detail and it does not apply anymore.  By "I'll
> > >> > look at your patch" I just meant to say that I will investigate how it
> > >> > can be used to fix our linking problems, not that I wanted to look at
> > >> > it for the first time.
> > >> 
> > >> sorry, but I really don't understand the reason for a single and totally
> > >> flat
> > >> Makefile: IMO a Makefile per directory is a kind of modularization (that
> > >> is generally considered "A good thing" ) , clean for its own nature.
> > >> Can you explain what we gain now? or in what respect a single Makefile
> > >> is cleaner?
> > >
> > > The single Makefile is shorter than the single-directory Makefiles
> > > combined, even with common infrastructure factorized into mpcommon.mak.
> > > Our Makefile is just above 1000 lines.  Given that it consists largely
> > > of long lists, it is neither complicated nor particularly long.  In such
> > > a case I prefer having all the information in one central place.
> > >
> > > At work I once had the misfortune of having to debug a recursive make
> > > system where everything was split into tiny snippets.  I continuously
> > > lost track of which part got included from where and after staring at it
> > > for a few hours, I rewrote it non-recursively in a fraction of that
> > > time.  The end result was a very simple, single Makefile with about 30%
> > > of the previous total line count.  It was also faster and correct.
> > >
> > > So unless you go over a certain threshold I generally prefer 60 lines
> > > over 6 x 10 lines.
> > >
> > > But this is not the main reason.  The problem is that recursive Make is
> > > incorrect.  What the Make program does is build a directed acyclic graph
> > > of all dependencies/targets and then perform all the necessary steps the
> > > target you request requires, but no more than those necessary steps.
> > >
> > > Recursive uses of make cut this graph into pieces.  Unfortunately this
> > > process is not lossless because inter-directory dependencies are left
> > > out.  This results in Make systems that continuously recompile things
> > > unnecessarily, fail to recompile necessary parts and generally have
> > > horrible performance.
> > >
> > > For a more in-depth explanation, read Peter Miller's short but
> > > informative paper "Recursive Make Considered Harmful":
> > >
> > > http://miller.emu.id.au/pmiller/books/rmch/
> > >
> > > I hope this clears it up.  If you have more questions, ask, but it's
> > > really been explained many times in many places, the best I know being
> > > the above-mentioned paper.
> > 
> > Diego had found some article about good habits and implements it
> > without fully understanding it. He was inspired by the FFmpeg change
> > in same direction.
> 
> I can claim with confidence to understand that paper (I read it twice)
> and I have read the GNU Make manual cover to cover.
> 
> I understand the MPlayer build system, of which I built the Makefile
> single-handedly.  I also understand the FFmpeg build system and wrote
> large parts of it.  I also reworked some Make build system structures at
> work, making them non-recursive in the process.
> 
> I have been planning to make MPlayer's (and FFmpeg's) build system
> non-recursive for years and started thinking and working on it long
> before Mans sat down to implement it for FFmpeg.

I just wanted to say that i belive that having the whole dependancy graph
available to make is the better thing than the splitted up recursive calling.
Though IMHO libmp* should be buildable seperately. It was IIRC the way it
was intended. Not that i need either a non recursive make nor seperately
build libmp*.

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Complexity theory is the science of finding the exact solution to an
approximation. Benchmarking OTOH is finding an approximation of the exact
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/attachments/20080605/59388119/attachment.pgp>