[FFmpeg-devel] Round 2: Symbol versioning

[Reinhard Tartler]

Michael Niedermayer <michaelni at gmx.at> writes:

> On Tue, Dec 29, 2009 at 10:54:16AM +0100, Reinhard Tartler wrote:
>> Michael Niedermayer <michaelni at gmx.at> writes:
>> 
>> > On Mon, Dec 28, 2009 at 04:07:02PM +0100, Reinhard Tartler wrote:
>> >> Diego Biurrun <diego at biurrun.de> writes:
>> >> 
>> >> > On Sun, Dec 27, 2009 at 10:35:14AM +0100, Reinhard Tartler wrote:
> [...]
>> > Thus if you introduce a libavutil with same soname but enabled versioning
>> > and with the shlibs stuff set correctly by hand
>> > any package rebuild with it could not be installed as long as an older
>> > libavutil of that soname was installed.
>> > An update to one with versioning
>> > would be required if a rebuild application or lib where installed.
>> > This should prevent 1 of 3 bugs in ld.so 
>> 
>> correct
>> 
>> > at the expense of hundreads of otherwise unneeded dependancies.
>> 
>> these dependencies are already implemented archive wide, so the absolute
>> number of (versioned) dependencies does *not* increase.
>
> well yes if you count them that way ...
> but the dependancies become stricter than they otherwise would be

that's correct.

>> > But at this point you have not introduced the new libavutil with new
>> > ABI and new soname.
>> 
>> Indeed.
>> 
>> > Thats something you have apparently postponed for later and i wonder
>> > how you plan to do it without hitting bugs 2 and 3 It sure can be done
>> > with many additional (otherwise unneeded) dependancies
>> 
>> That depends on several factors, mainly on how fast this transition will
>> be implemented and at what stage of the release cycle for the respective
>> distro we are.
>
> Well, if your chain of reasoning is that simply waiting long enough will
> make the conflicts likeliness aproach zero, then i dont agree.

No, that is *not* my chain of reasoning, let me elaborate on this more
further below in this posting.

> I myself still have a few packages installed on some computers that i
> did not upgrade and that dont even exist anymore in current debian.
> xmms is one of them, i didnt "upgrade" to xmms2 and wont in the future
> either.

I don't see any problem with that.

>> I'd really like to get this deployed distro wide first and decide then
>> how to proceed from there afterwards in order to avoid mixing too many
>> (related) issues unnecessarily.
>
> noone is stoping you from introducing symbol versioning with stricter
> than needed dependancies in debian. I think its not the ideal approach
> and even less so when one considers more than a single package.

that's right, but still an important improvement to the current
situation, IMO.

>> > and i claim that its very easy to miss some of these dependancies and
>> > in the end what you create with all these dependancies is not that far
>> > from the rebuild the world you are trying to avoid.
>> 
>> I claim that this is not true, because of the mechanics of the shlibs
>> system and me as package maintainer, but this is something that has to
>> been shown by practice, not theory?
>
> I understand how the shlibs system works around the assertion bug, i dont
> see how it will help against the other bugs.

Bug #2 can only happen if a (packaged) application can have the same
symbol loaded one time with and one time without symbol versioning. With
the *current* upgrade paths this cannot happen, as the shared libraries
with versioned symbols will replace the old ones at the same moment as
they are installed.

The only way this situation could occur is if we would target getting
libavutil50 for the next Debian release (squeeze). I don't do that
currently for several reasons; one is the issue at hand, we can discuss
the other ones in a separate thread.

Even if we would target libavutil for squeeze, there would still be a
standard practice for shared library packaging as I already indicated
before. I wanted to avoid annoying you with these details, but it seems
you are interested in them anyways: rename the source package
libavutil49 to libavutil49v or something and make it conflict against
libavutil49, as described in section 3.3 of [1]. Please note that this
is not an ad-hoc solution, but a procedure that has been used for other
libraries before. Since this is rather disruptive and I don't see the
necessity for this, I'd like to avoid it if I can.

Okay, you can now argue that this again works around "bugs" in ld.
That's right then, but I don't really see a problem with that either;
apt handles that just fine.

For the same reason, bug #3 cannot occur in practice either.

Still, I of course understand that the issues you've identified are real
and should be addressed upstream.

>> >> Besides, the whole stunt is coordinated with the debian release
>> >> team[2], who have supervised many similar transitions before. I'm
>> >> therefore very confident that the current approach (w/o patching
>> >> the linker) is proven (and therefore "good enough") for debian.
>> >
>> > Its also proofen one can swim across the english channel, still
>> > trying to make this an argument that this method of crossing the
>> > channel is "good enough" is probably only successfull as british
>> > humor
>> 
>> I'm not a native speaker, I meant prooven as in "bew?hrt", not
>> "bewiesen" (both german)
>
> So you mean its along the line of
> The number of people stepping on a mine and blowing their legs off is small
> enough so we continue with storing the mines in the shool yard
> ?
> if thats what you call prooven then yes we agree

I think the issue is way less dramatic as you describe here.

>> >> The issues Michael is adressing seem to me rather farfetched and
>> >> very unlikely to occur in the field when deploying them in debian
>> >> in the way I intend to do. For ffmpeg upstream, I can see no
>> >> regressions either.
>> >
>> > The issues arent much more farfetched than the ones you try to fix
>> > with introduction of versioning
>> 
>> Well I can show you dozens of bug reports that shows that people
>> *are* already hit by the problem I intend to fix with symbol
>> versioning. The issues you adress of course also exist, but are way
>> less likely to occur in practice,
>
>> and impossible to hit if you follow the recommended (as in
>> the release notes) upgrade procedures closely.
>
> i belive it if i see what that procedure exactly would require to be done
> and that actually is enough.

see above.

>> > Besides with my patch introducing versioning should become a much
>> > simpler task without all the unexcpected dependancy needs. That
>> > sure would make the release teams work and all involved maintainers
>> > work easier for future such changes ...
>> 
>> Yes, and that's why I think they shouldn't be implemented in debian
>> only, but in binutils/glibc upstream. But then we probably have to deal
>> with Drepper?
>
> I wont deal with drepper :)
> What i will do is fix issues with my patch people notify me off

I'll make a note and will consider forwarding it upstream. Currently I'm
still a bit unsure about the side effects your patch can cause and I
need to experiment a bit more with it before I can proceed with that.

> Its not me who is inconvenienced by this, at least not more than any
> other user. Its the various distros and maintainers who have to deal
> with very hard to debug bugs due to the linker mislinking symbols and
> with all the prooven procedures to introduce versioning and workaround
> these issues.
>
> With my patch you just enable versioning and thats it
>
> Without my patch you need to update the shlibs dependancies, enable
> versioning and closely follow the upgrade procedures (which must be
> quite involved as you just refer to it but dont tell us what exactly
> that would involve) that said that doesnt mean that procedure works,
> until i see it iam still suspecting it will not introduce all
> dependancies one needs to avoid bugs 2 and 3

I hope the above was now clear enough.

BTW, how do RPM based distro packagers deal with this situation? AFAIUI
they avoid partial upgrades as well as possible to prevent these
situations completely, but I'd be glad to be proven wrong here.

[1]
http://www.netfort.gr.jp/~dancer/column/libpkg-guide/libpkg-guide.html#id293373

-- 
Gruesse/greetings,
Reinhard Tartler, KeyID 945348A4