[Ffmpeg-devel] [RFC] dlopen vs linking for external libraries

[Uoti Urpala]

On Sun, 2007-02-11 at 22:20 +0100, Michael Niedermayer wrote:
> On Sun, Feb 11, 2007 at 10:53:46PM +0200, Uoti Urpala wrote:
> > On Sun, 2007-02-11 at 20:16 +0000, M?ns Rullg?rd wrote:
> > > I think Michael was complaining about the endless lists of
> > > dependencies that are typical for binary packages.  Most distros tend
> > > to enable everything that can possibly be enabled, so you end up with
> > > many gigabytes of disk space taken up by stuff you never ever use.
> > 
> > On this Debian system /usr/lib is less than a gigabyte and the
> > whole /usr is about 2.1 GiB. That includes things like OpenOffice.
> > Ending up with "many gigabytes" of space taken by unnecessary
> > dependencies alone sounds highly exaggerated.
> 
> exaggeration does not change the validity of the point, if your system
> has only a few megabytes of space, wasting a few hundread megabyte is not
> an option

The question for binary distros is not whether there could possibly
exist at least one person for whom a customized package (such as one
using a minimal amount of disk space) would be useful. How large a
portion of their users would use FFmpeg on a system with less than a
dozen megabytes free?

Exaggeration does change the validity: there are fewer people with many
gigabytes to "waste" than people who can afford 100 MiB.

> > If the binary distros tried to make features optional and used dlopen()
> > or created separate packages with enabled/disabled features the
> > increased complexity would make it a net loss (yes they have considered
> > that). 
> 
> our dlopen code doesnt look that complex ...

In addition to the extra complexity in the program code dlopen() makes
the program more brittle and creates other problems for distros. It
makes it impossible to verify correct dependencies from the binary
directly (which is usually done in Debian) and makes it harder for other
packages to depend on the program (if I need functionality X how do I
express that when program Y might or might not support that depending on
what else is installed? Depend on Y+exactly what?).

> > For a program typically used with video files the space savings
> > matter even less than for most others.
> 
> these statements remind me of http://en.wikipedia.org/wiki/Weasel_words
> 
> in the case here, you make 3 claims (dlopen increases complexity so much
> that its generally not a worth it), (its generally useless) and (even more
> useless for video tools)

Just how did you make up those "3 claims" based on what I said?
Especially the 2nd one seems like quite a "creative" interpretation of
my words. The most plausible way to create that I can see is to take
"space savings matter even less", use that to imply "minor space savings
generally matter little" and then somehow (incorrectly) mutate that to
say "dlopen matters little -> dlopen is generally useless".

>  id like to know who said that? you?
> and why should we care what that person said compared to an actual technical
> statement which could be verified ...

What I do claim is:
1. Moving possible package dependencies to dlopen() when they're not
necessary for every user would be a net loss for binary distributions,
with some (rare) exceptions for huge libraries.

I gave some reasons above; if you want more detailed explanations then
you should probably contact someone directly working on the distros.
Note that this cannot be decided by a 100% pure technical statement. If
most users were say embedded developers then disk space might be
important enough to change the answer - that most aren't is not strictly
a technical statement.

2. Space savings for FFmpeg which is typically used with large files
matter less than for programs which are typically used with only small
external data.

I think this should be common sense...

> for example on embeded or old hardware with little available space
> you might not want to have 1mb space wasted for 3 external libs and compiling

There might be such cases. But how common are they? Using a binary
distro and wanting to use FFmpeg when 1 MiB usage is important?

> so we would end up with cross compilation, is cross compilation done by many
> users really less complex then 10 lines of code for dlopen() ?

You're making several implicit assumptions in that question. At least 1)
There are many users wanting to use FFmpeg on such systems. 2) They
would not prefer to compile their own version even if dlopen() was
available. 3) That there would be no binary suitable for them with
static linking or a constant set of dynamic libs. 4) That there really
would be a suitable binary available using dlopen().

>  which needs
> near 0 maintaince time, just look how many changes the existing dlopen code
> needed over time ...

If the dlopen() features do not interfere with other code then I have
nothing against keeping the option to use them in FFmpeg. But that does
not imply that distros should prefer them over dynamic linking.