[FFmpeg-devel] Upgrade Trouble

Michael Niedermayer michaelni
Sun Dec 27 13:57:37 CET 2009


On Sun, Dec 27, 2009 at 10:55:08AM +0100, Reinhard Tartler wrote:
> Michael Niedermayer <michaelni at gmx.at> writes:
> 
> > On Fri, Dec 25, 2009 at 11:28:49PM +0100, Reinhard Tartler wrote:
> >> Michael Niedermayer <michaelni at gmx.at> writes:
[...]
> >
> >> 
> >> But for the sake of this example, let's proceed.
> >> 
> >> > gcc -shared -fPIC -c libB0.c -o libB0.o
> >> 
> >> this makes libB0.c pickup the versioned symbol of libA1.
> >> 
> >> > ld -shared libB0.o -o libB0.so -L. -lA1
> >> >
> >> > ./app
> >> > #libA0
> >> > #B libB0
> >> > #libA0
> >> > #libA0
> >> > #        (libB0 is linked by the linker to the wrong lib here)
> >> 
> >> This mixes a library with both versioned and unversioned symbols. The
> >> "Base" symbol seems to be treated as a corner case by ld, that's why all
> >> applications needs to be rebuilt to pickup the versioned symbols.
> >> 
> >> >
> >> > gcc -shared -fPIC -c libA0.c -o libA0.o
> >> 
> >> again, superfluous
> >> 
> >> > ld -shared --version-script libA0.v libA0.o -o libA0.so
> >> 
> >> now we introduce symbol versioning to the 'old' lib, which removes the
> >> @Base symbol from the application's view. This unsurprisingly results in:
> >> 
> >> > ./app
> >> > #libA0
> >> > #B libB0
> >> > #libA1
> >> > #libA0 
> >> > #        (correct)
> >> >
> >> 
> >> the expected behavior!
> >> 
> >> > gcc app.c -o app -L. -lB0 -lA0
> >> 
> >> now app picks up the versioned symbol from libA0. This should have been
> >> done earlier.
> >> 
> >> > ./app
> >> > #libA0
> >> > #B libB0
> >> > #libA1
> >> > #libA0 
> >> > #        (correct)
> >> 
> >> yupp
> >> 
> >> > gcc -shared -fPIC -c libA0.c -o libA0.o
> >> > ld -shared libA0.o -o libA0.so
> >> 
> >> so now you remove the version tags. This essentially breaks the ABI and
> >> results in:
> >> 
> >> >
> >> > ./app
> >> > #./app: ./libA0.so: no version information available (required by ./app)
> >> > #Inconsistency detected by ld.so: do-lookup.h: 115: check_match: Assertion `version->filename == ((void *)0) || ! _dl_name_match_p (version->filename, map)' failed!
> >> 
> >> The error message above correctly indicates the problem. The assertion
> >> failure could be probably seen as a very minor cosmetic issue.
> >
> > uhm you got that seriously backward.
> > just out comment that assert and recompile ld.so
> > (you need 10+ vomit bags and a bunch of "gnu is not posix tools" and patience)
> > when you did this, it still prints the harmless warning but works otherwise
> 
> Well, I can imagine that this works in this case. Still, when we
> consider symbols with and without version tags as totally different
> symbols,

The problem is that the linker does not consider them different
We would be a big step forward if it did because then it wouldnt
bind them wrongly ...


> the error message is correct (except that it does not fail
> gracefully but by an confusing assert()).

unless debuging is disabled and the assert isnt checked or unless you
rename the file, you know that assert checks the filename ...
that is it fails if the filename matches what the application expects
to where the symbol is.


> Of course this view is too
> simplistic; for transition purposes, if an application references an
> unversioned symbol, the runtime linker can satisfy this request with an
> versioned symbol. 

app (refers versioned A0 or unversioned as you prefer)
    libA0 (versioned with ver=A0)
    libB0 (unversioned reference)
        libA1  (versioned with ver=A1)

this links libB0 to libA0
so no it does not work any bit better than the case below. If you try to
justify below by it then this applies here as well. But keep in mind we
talk about trivial bugs in ld.so that can be fixed by a few lines being
changed.


> However the reverse, (the application references an
> versioned symbol but only an unversioned one is available) may or may not
> succeed (as in "unspecified behavior").

> 
> > Sorry for the delay in my reply, i just had to retry a few times to get
> > the gnu crap to compile far enough to have a executeable ld.so to proof to
> > myself that the assert() is pure asshatry and theres no problem with any
> > symbols not being found
> >
> >> 
> >> > gcc -shared -fPIC -c libA0.c -o libA0.o
> >> > ld -shared --version-script libA0.v libA0.o -o libA0.so
> >> 
> >> so symbol versioning in libA0.so is restored, the application should
> >> work again.
> >> 
> >> > gcc -shared -fPIC -c libA1.c -o libA1.o
> >> > ld -shared libA1.o -o libA1.so
> >> 
> >> now the version tags from libA1 is removed, which again results in
> >> unsurprising behavior:
> >> 
> >> >
> >> > ./app
> >> > #./app: ./libA1.so: no version information available (required by ./libB0.so)
> >> > #libA0
> >> > #B libB0
> >> > #Inconsistency detected by ld.so: do-lookup.h: 115: check_match: Assertion `version->filename == ((void *)0) || ! _dl_name_match_p (version->filename, map)' failed!
> >> >
> >> 
> >> the runtime linker now fails to find the versioned symbols for libA1,
> >> against which libB0 is linked.
> >
> > no, its all found, its just that some developers should have had their
> > alzheimer pill dosages raised again before they attemped to add that assert
> >
> >
> >> 
> >> I fail to see any bug in ld here, this example demonstrates that broken
> >> usage of ld can produce some confusing error messages.
> >
> > you know, even if that was the case ... A confusing error
> > message in itself is a bug too.
> >
> >
> >> 
> >> I'd suggest to just use symbol versioning as it is intended:
> >> 
> >>  - first introduce symbol tags
> >>  - then recompile applications
> >>  - do not remove symbol versioning afterwards without at least a SONAME
> >>    bump (preferably never)
> >
> > thats what we do
> 
> in ffmpeg yes, your testcase doesn't follow this procedure closely.

It does.
You in addition add dependencies to prevent the assert failure in ld.so


> 
> in any case, I think I've really done my homework now, checked this
> approach with the debian release team and got the green light to
> proceed. No I've proposed patches in the hope to have them accepted in
> trunk. Once I get confirmation that this approach will get into ffmpeg
> trunk in a forseeable timeframe, I'll start the transition in debian and
> ubuntu.
> 
> 
> >> >> I think that when introducing symbol versioning all applications need
> >> >> to be rebuilt in order to pick up the new versioned symbols and the
> >> >> problem doesn't arise in the first place.
> >> >
> >> > An application that is not rebuild links to the correct symbols
> >> > A not rebuilt lib can link to the wrong symbols, you can rebuild all libs
> >> > But does this ensure that the user doesnt upgrade just a subset?
> >> 
> >> By using versioned dependencies in application packages, it is
> >> impossible to install a rebuilt application without also installing a
> >> rebuilt library. We use the dpkg-shlibdeps mechanism to automate this,
> >> so nothing to worry about that here.
> >
> > Its great that you can workaroud ld.so bugs by adding hundreads of otherwise
> > unneeded dependancies. My oppinion stands though that the linker should be
> > fixed. Part of that would be to comment that idiotic assert out and donate
> > some more alzheimer pills to the guy who added it in the first place
> 
> I think we can agree to disagree: IMO, this is not a bug but undefined
> behavior.  Can we move on or is this disagreement so critical to you that
> you would block introducing symbol versioning in the ffmpeg svn?

i dont mind symbol versioning in ffmpeg.
About the linker, read my blog theres a patch to fix it mostly.
If debian prefers not to fix their linker thats ok but i  maintain my
oppinon that fixing the linker is the proper way to deal with it.
The hundereads of dependencies you will need (and the hundreads you
will need but forget) are really not a reasonable way to enable
versioning, its a hell of work thats completely unneeded.

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The educated differ from the uneducated as much as the living from the
dead. -- Aristotle 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20091227/7d2fd3dc/attachment.pgp>



More information about the ffmpeg-devel mailing list