hey, i've been trying to clean up old/broken/etc. cruft in mplayer code an i realized the hated build system where everything gets rebuilt unless you touch .developer is still in place. as much as coreteam liked forcing users to perform such favors on them, i think it's time we remove the need for users to 'touch .developer' in order to have sane build dependencies. does anyone reject to: - removing support for .developer - fixing dependencies so that version.h is handled correctly. - moving the old .libdeps hack i added out of etc and making it part of the default makefile for automatic subdir dependency handling. - removing the dependency on configure that forces rebuild whenever configure has changed (it's wrong anyway since configure needs to be rerun first). ?? rich
On Friday 31 March 2006 03:05, Rich Felker wrote:
- fixing dependencies so that version.h is handled correctly.
PLEASE, for the love of god, fix the version.h insanity!
- removing the dependency on configure that forces rebuild whenever configure has changed (it's wrong anyway since configure needs to be rerun first).
As long as the warning is kept?
On Fri, Mar 31, 2006 at 03:14:55AM +0300, Jan Knutar wrote:
On Friday 31 March 2006 03:05, Rich Felker wrote:
- fixing dependencies so that version.h is handled correctly.
PLEASE, for the love of god, fix the version.h insanity!
:))
- removing the dependency on configure that forces rebuild whenever configure has changed (it's wrong anyway since configure needs to be rerun first).
As long as the warning is kept?
Yes, the warning would be kept just like .developer mode now. The only difference is that it would not force a (nonsense) recompile without reconfigure, which only serves to _hide_ the fact that reconfigure was needed and skipped. Rich
On Thu, Mar 30, 2006 at 07:05:40PM -0500, Rich Felker wrote:
i've been trying to clean up old/broken/etc. cruft in mplayer code an
I have made a list of candidates for speedy deletion ages ago, shall I dig it out while we're at it? :)
i realized the hated build system where everything gets rebuilt unless you touch .developer is still in place. as much as coreteam liked forcing users to perform such favors on them, i think it's time we remove the need for users to 'touch .developer' in order to have sane build dependencies.
lol
does anyone reject to:
- removing support for .developer - fixing dependencies so that version.h is handled correctly. - moving the old .libdeps hack i added out of etc and making it part of the default makefile for automatic subdir dependency handling. - removing the dependency on configure that forces rebuild whenever configure has changed (it's wrong anyway since configure needs to be rerun first).
I have had excising all of these hacks from the build system on the ToDo for ages. I'll be most happy if you beat me to it. Does anybody understand the sideeffects anyway? While we're at it, why is .libdeps needed at all? Diego
On Fri, Mar 31, 2006 at 09:30:09AM +0200, Diego Biurrun wrote:
does anyone reject to:
- removing support for .developer - fixing dependencies so that version.h is handled correctly. - moving the old .libdeps hack i added out of etc and making it part of the default makefile for automatic subdir dependency handling. - removing the dependency on configure that forces rebuild whenever configure has changed (it's wrong anyway since configure needs to be rerun first).
I have had excising all of these hacks from the build system on the ToDo for ages. I'll be most happy if you beat me to it. Does anybody understand the sideeffects anyway?
While we're at it, why is .libdeps needed at all?
It's just a set of rules to make the main makefile recurse into subdirectories if any files in the subdirs have changed. Unconditionally recursing into them is also possible but would make performance extremely bad in the common case where nothing has changed. Of course I'd be happiest if we could eliminate recursive make altogether... http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html Rich
2006/3/31, Rich Felker <dalias@aerifal.cx>:
On Fri, Mar 31, 2006 at 09:30:09AM +0200, Diego Biurrun wrote:
does anyone reject to:
- removing support for .developer - fixing dependencies so that version.h is handled correctly. - moving the old .libdeps hack i added out of etc and making it part of the default makefile for automatic subdir dependency handling. - removing the dependency on configure that forces rebuild whenever configure has changed (it's wrong anyway since configure needs to be rerun first).
I have had excising all of these hacks from the build system on the ToDo for ages. I'll be most happy if you beat me to it. Does anybody understand the sideeffects anyway?
While we're at it, why is .libdeps needed at all?
It's just a set of rules to make the main makefile recurse into subdirectories if any files in the subdirs have changed. Unconditionally recursing into them is also possible but would make performance extremely bad in the common case where nothing has changed.
Of course I'd be happiest if we could eliminate recursive make altogether... http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html
I don't object the changes, but I would make an request. I would like a mode that doesn't build dependences. I usually compile from copied clean source and building dependeceys is taking same time as the compiling itself (thanks to ccache). This mode is also know in autoconf land as "speed-up one time build" Anyway, fix the dependeces first :)
On Fri, Mar 31, 2006 at 12:09:22PM +0300, Ivan Kalvachev wrote:
I don't object the changes, but I would make an request.
I would like a mode that doesn't build dependences. I usually compile from copied clean source and building dependeceys is taking same time as the compiling itself (thanks to ccache). This mode is also know in autoconf land as "speed-up one time build"
I don't understand this paragraph. Diego
On Fri, Mar 31, 2006 at 03:16:26AM -0500, Rich Felker wrote:
On Fri, Mar 31, 2006 at 09:30:09AM +0200, Diego Biurrun wrote:
While we're at it, why is .libdeps needed at all?
It's just a set of rules to make the main makefile recurse into subdirectories if any files in the subdirs have changed. Unconditionally recursing into them is also possible but would make performance extremely bad in the common case where nothing has changed.
I say go for unconditional recursing. It takes 1s on my machine when nothing has changed, I see no need to improve this "extremely bad performance". Another downside of .libdeps is that it needs to be updated every time things get added, shuffled around etc in the source tree. So far this has always been overlooked, just look at the cvs log of etc/.libdeps. And it's documented nowhere, few devs understand it and/or use it. IMO it is an obscure mechanism with questionable benefit. You'll need some pretty good arguments to sway my opinion.
Of course I'd be happiest if we could eliminate recursive make altogether... http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html
I've read that paper as well, but while the rant is completely right in theory, the practical instructions for implementing non-recursive make are missing from it. Where are the examples of non-recursive make usage? The whole world seems to happily use it recursively... Diego
On Sat, Apr 01, 2006 at 09:55:24AM +0200, Diego Biurrun wrote:
On Fri, Mar 31, 2006 at 03:16:26AM -0500, Rich Felker wrote:
On Fri, Mar 31, 2006 at 09:30:09AM +0200, Diego Biurrun wrote:
While we're at it, why is .libdeps needed at all?
It's just a set of rules to make the main makefile recurse into subdirectories if any files in the subdirs have changed. Unconditionally recursing into them is also possible but would make performance extremely bad in the common case where nothing has changed.
I say go for unconditional recursing. It takes 1s on my machine when nothing has changed, I see no need to improve this "extremely bad performance".
I can guarantee it's a LOT worse on my system. Probably 10s or more. This is unacceptable.
Another downside of .libdeps is that it needs to be updated every time things get added, shuffled around etc in the source tree. So far this has always been overlooked, just look at the cvs log of etc/.libdeps. And it's documented nowhere, few devs understand it and/or use it. IMO it is an obscure mechanism with questionable benefit. You'll need some pretty good arguments to sway my opinion.
My proposal now is eliminating all makesfiles except in the top-level dir and eliminating recursive make. This will solve all the dependency problems with no performance penalty.
Of course I'd be happiest if we could eliminate recursive make altogether... http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html
I've read that paper as well, but while the rant is completely right in theory, the practical instructions for implementing non-recursive make are missing from it. Where are the examples of non-recursive make usage? The whole world seems to happily use it recursively...
The whole world does a lot of stupid things. :))))) Look at my libc for a good example -- a single 69-line makefile drives the whole build with multiple levels of directories. Naturally a bit more is needed to manage all the different imported libs and stuff that MPlayer uses, but I don't think it will be particularly bad. Let's see if I can do it... Until then, if no one objects my first step will be to fix things so that the current system works like having .developer and .libdeps. This will make things better in some cases, no worse in others, and performance will be no worse. Rich
On Sat, Apr 01, 2006 at 03:51:01AM -0500, Rich Felker wrote:
On Sat, Apr 01, 2006 at 09:55:24AM +0200, Diego Biurrun wrote:
On Fri, Mar 31, 2006 at 03:16:26AM -0500, Rich Felker wrote:
On Fri, Mar 31, 2006 at 09:30:09AM +0200, Diego Biurrun wrote:
While we're at it, why is .libdeps needed at all?
It's just a set of rules to make the main makefile recurse into subdirectories if any files in the subdirs have changed. Unconditionally recursing into them is also possible but would make performance extremely bad in the common case where nothing has changed.
I say go for unconditional recursing. It takes 1s on my machine when nothing has changed, I see no need to improve this "extremely bad performance".
I can guarantee it's a LOT worse on my system. Probably 10s or more. This is unacceptable.
Huh? I have the same hardware as you in my PC, I've never had this problem. Provide some steps to reproduce :)
Another downside of .libdeps is that it needs to be updated every time things get added, shuffled around etc in the source tree. So far this has always been overlooked, just look at the cvs log of etc/.libdeps. And it's documented nowhere, few devs understand it and/or use it. IMO it is an obscure mechanism with questionable benefit. You'll need some pretty good arguments to sway my opinion.
My proposal now is eliminating all makesfiles except in the top-level dir and eliminating recursive make. This will solve all the dependency problems with no performance penalty.
Sounds good, let's see it.
Of course I'd be happiest if we could eliminate recursive make altogether... http://www.pcug.org.au/~millerp/rmch/recu-make-cons-harm.html
I've read that paper as well, but while the rant is completely right in theory, the practical instructions for implementing non-recursive make are missing from it. Where are the examples of non-recursive make usage? The whole world seems to happily use it recursively...
The whole world does a lot of stupid things. :)))))
Sure, but when the world just ignores the textbook correct solution it may also be that the textbook correct solution isn't all that hot.
Look at my libc for a good example -- a single 69-line makefile drives the whole build with multiple levels of directories. Naturally a bit more is needed to manage all the different imported libs and stuff that MPlayer uses, but I don't think it will be particularly bad. Let's see if I can do it...
As I said, I'd love to see it work.
Until then, if no one objects my first step will be to fix things so that the current system works like having .developer and .libdeps. This will make things better in some cases, no worse in others, and performance will be no worse.
Let's see your patches/commits. Diego
On Sun, Apr 02, 2006 at 12:12:12AM +0200, Diego Biurrun wrote:
I can guarantee it's a LOT worse on my system. Probably 10s or more. This is unacceptable.
Huh? I have the same hardware as you in my PC, I've never had this problem. Provide some steps to reproduce :)
Buy hdd that is not 7200 rpm udma100 or 166 or whatever the fastest nonsense is now.
I've read that paper as well, but while the rant is completely right in theory, the practical instructions for implementing non-recursive make are missing from it. Where are the examples of non-recursive make usage? The whole world seems to happily use it recursively...
The whole world does a lot of stupid things. :)))))
Sure, but when the world just ignores the textbook correct solution it may also be that the textbook correct solution isn't all that hot.
There's absolutely nothing 'not all that hot' about the correct use of makefiles except that people are too stupid to write them correctly. Heading this fud over and over makes me sick, and hearing it from you is even more disappointing. Rich
On Sat, Apr 01, 2006 at 08:29:03PM -0500, Rich Felker wrote:
On Sun, Apr 02, 2006 at 12:12:12AM +0200, Diego Biurrun wrote:
I can guarantee it's a LOT worse on my system. Probably 10s or more. This is unacceptable.
Huh? I have the same hardware as you in my PC, I've never had this problem. Provide some steps to reproduce :)
Buy hdd that is not 7200 rpm udma100 or 166 or whatever the fastest nonsense is now.
There are 3 SCSI 4GB HDDs in that machine, you can guess their age, although admittedly my dev tree is on a RAID0 partition over all three disks...
I've read that paper as well, but while the rant is completely right in theory, the practical instructions for implementing non-recursive make are missing from it. Where are the examples of non-recursive make usage? The whole world seems to happily use it recursively...
The whole world does a lot of stupid things. :)))))
Sure, but when the world just ignores the textbook correct solution it may also be that the textbook correct solution isn't all that hot.
There's absolutely nothing 'not all that hot' about the correct use of makefiles except that people are too stupid to write them correctly.
Either that or the cost benefit ratio is not appealing enough to make people overcome their inertia. I have yet to see a real world example for recursive make failures. Look at the FFmpeg build system. Modulo a bit of cruft that still needs to be moved into configure I'm rather happy how it's turning out. It's simple, it's understandable, it's (still) recursive, it works(TM). Of course, if you have further suggestions for improvement, I'm all ears.
Heading this fud over and over makes me sick, and hearing it from you is even more disappointing.
Oh, come on, spare us the drama ;) Diego
On Sun, Apr 02, 2006 at 10:23:00AM +0200, Diego Biurrun wrote:
The whole world does a lot of stupid things. :)))))
Sure, but when the world just ignores the textbook correct solution it may also be that the textbook correct solution isn't all that hot.
There's absolutely nothing 'not all that hot' about the correct use of makefiles except that people are too stupid to write them correctly.
Either that or the cost benefit ratio is not appealing enough to make people overcome their inertia. I have yet to see a real world example for recursive make failures.
Unless I'm misunderstanding your words this is completely false. MPlayer is an abysmal failure at recursive make, and I've seen plenty of packages that do it "correctly" (with unconditional recursion) where it takes over a minute to decide that nothing needs to be done. Mozilla comes to mind as a good example.. :)
Look at the FFmpeg build system. Modulo a bit of cruft that still needs to be moved into configure I'm rather happy how it's turning out. It's simple, it's understandable, it's (still) recursive, it works(TM). Of course, if you have further suggestions for improvement, I'm all ears.
The only 'cost' of 'monolithic' make arises in cases like this where you want the individual parts of a project to be able to build independently of one another, when taken out of the tree. This requires a bit of thinking (and it's the one difficulty I see in making the switch in MPlayer), but I think it can be handled with clever use of includes. I'll see what I can come up with.
Heading this fud over and over makes me sick, and hearing it from you is even more disappointing.
Oh, come on, spare us the drama ;)
It's just that you're usually open to correcting broken things and don't side with the ignorant establishment. Rich
On Sun, Apr 02, 2006 at 04:50:22AM -0400, Rich Felker wrote:
On Sun, Apr 02, 2006 at 10:23:00AM +0200, Diego Biurrun wrote:
The whole world does a lot of stupid things. :)))))
Sure, but when the world just ignores the textbook correct solution it may also be that the textbook correct solution isn't all that hot.
There's absolutely nothing 'not all that hot' about the correct use of makefiles except that people are too stupid to write them correctly.
Either that or the cost benefit ratio is not appealing enough to make people overcome their inertia. I have yet to see a real world example for recursive make failures.
Unless I'm misunderstanding your words this is completely false.
Let me rephrase then: Show me the code. This is a case where it will take more than just words to convince me, that's all.
MPlayer is an abysmal failure at recursive make, and I've seen plenty of packages that do it "correctly" (with unconditional recursion) where it takes over a minute to decide that nothing needs to be done. Mozilla comes to mind as a good example.. :)
I'm not saying the MPlayer build system is good. It's not.
Look at the FFmpeg build system. Modulo a bit of cruft that still needs to be moved into configure I'm rather happy how it's turning out. It's simple, it's understandable, it's (still) recursive, it works(TM). Of course, if you have further suggestions for improvement, I'm all ears.
The only 'cost' of 'monolithic' make arises in cases like this where you want the individual parts of a project to be able to build independently of one another, when taken out of the tree. This requires a bit of thinking (and it's the one difficulty I see in making the switch in MPlayer), but I think it can be handled with clever use of includes. I'll see what I can come up with.
This is precisely the case for FFmpeg. As I said, if you have a clever solution, I'm all ears.
Heading this fud over and over makes me sick, and hearing it from you is even more disappointing.
Oh, come on, spare us the drama ;)
It's just that you're usually open to correcting broken things and don't side with the ignorant establishment.
That's not quite what I meant by "sparing the drama" ;) Diego
On Sat, 1 Apr 2006 03:51:01 -0500 Rich Felker <dalias@aerifal.cx> wrote:
On Sat, Apr 01, 2006 at 09:55:24AM +0200, Diego Biurrun wrote:
I say go for unconditional recursing. It takes 1s on my machine when nothing has changed, I see no need to improve this "extremely bad performance".
I can guarantee it's a LOT worse on my system. Probably 10s or more. This is unacceptable.
That higly depends on the disks you have and the amount of RAM, whether you compiled MPlayer just a few minutes ago or not, etc pp. But i agree, unconditional recursing is bad[tm]
Another downside of .libdeps is that it needs to be updated every time things get added, shuffled around etc in the source tree. So far this has always been overlooked, just look at the cvs log of etc/.libdeps. And it's documented nowhere, few devs understand it and/or use it. IMO it is an obscure mechanism with questionable benefit. You'll need some pretty good arguments to sway my opinion.
My proposal now is eliminating all makesfiles except in the top-level dir and eliminating recursive make. This will solve all the dependency problems with no performance penalty.
I have a counter proposal: How about a top makefile that includes the additional makefiles per directory. This way we have the files that declare the dependency local (which is imho a good thing) but still can build the dependency tree globaly. If you provide a "Makefile" in every subdirectory you can even make a build of a subtree the normal way.
Until then, if no one objects my first step will be to fix things so that the current system works like having .developer and .libdeps. This will make things better in some cases, no worse in others, and performance will be no worse.
Go ahead. Attila Kinali -- 心をこめて聞け心をこめて話せ
On Sun, Apr 02, 2006 at 11:04:08AM +0200, Attila Kinali wrote:
That higly depends on the disks you have and the amount of RAM, whether you compiled MPlayer just a few minutes ago or not, etc pp. But i agree, unconditional recursing is bad[tm]
Yes.
My proposal now is eliminating all makesfiles except in the top-level dir and eliminating recursive make. This will solve all the dependency problems with no performance penalty.
I have a counter proposal: How about a top makefile that includes the additional makefiles per directory.
This is actually what I meant, except that the included files were not going to be complete makefiles, just module/dependency lists.
This way we have the files that declare the dependency local (which is imho a good thing) but still can build the dependency tree globaly. If you provide a "Makefile" in every subdirectory you can even make a build of a subtree the normal way.
I would like it if this could be done, but I'm doubtful. How do you make the included makefiles change into the correct directory for building their targets?
Until then, if no one objects my first step will be to fix things so that the current system works like having .developer and .libdeps. This will make things better in some cases, no worse in others, and performance will be no worse.
Go ahead.
Done. :) Rich
On Sun, 2 Apr 2006 05:24:05 -0400 Rich Felker <dalias@aerifal.cx> wrote:
This way we have the files that declare the dependency local (which is imho a good thing) but still can build the dependency tree globaly. If you provide a "Makefile" in every subdirectory you can even make a build of a subtree the normal way.
I would like it if this could be done, but I'm doubtful. How do you make the included makefiles change into the correct directory for building their targets?
I have an idea. Don't know whether it can work (i'm not that good at writing makefiles). I'll try as soon as i have more than 5min of free time. Attila Kinali -- 心をこめて聞け心をこめて話せ
On Sun, Apr 02, 2006 at 10:11:20PM +0200, Attila Kinali wrote:
On Sun, 2 Apr 2006 05:24:05 -0400 Rich Felker <dalias@aerifal.cx> wrote:
This way we have the files that declare the dependency local (which is imho a good thing) but still can build the dependency tree globaly. If you provide a "Makefile" in every subdirectory you can even make a build of a subtree the normal way.
I would like it if this could be done, but I'm doubtful. How do you make the included makefiles change into the correct directory for building their targets?
I have an idea. Don't know whether it can work (i'm not that good at writing makefiles). I'll try as soon as i have more than 5min of free time.
Excellent! I'm waiting to hear about it. Rich
On Thu, Mar 30, 2006 at 07:05:40PM -0500, Rich Felker wrote:
- removing the dependency on configure that forces rebuild whenever configure has changed (it's wrong anyway since configure needs to be rerun first).
If you do that, please change configure to not overwrite config.h directly, but create it e.g. as config.h.new. When configure finished, compare the old and new config.h and move it over when they differ. This avoids recompilations, even if configure changed but doesn't have any effect on the current platform. Joerg
On Fri, Mar 31, 2006 at 04:23:40PM +0200, joerg@britannica.bec.de wrote:
On Thu, Mar 30, 2006 at 07:05:40PM -0500, Rich Felker wrote:
- removing the dependency on configure that forces rebuild whenever configure has changed (it's wrong anyway since configure needs to be rerun first).
If you do that, please change configure to not overwrite config.h directly, but create it e.g. as config.h.new. When configure finished, compare the old and new config.h and move it over when they differ. This avoids recompilations, even if configure changed but doesn't have any effect on the current platform.
Good idea. Diego
participants (6)
-
Attila Kinali -
Diego Biurrun -
Ivan Kalvachev -
Jan Knutar -
joerg＠britannica.bec.de -
Rich Felker