[FFmpeg-devel] [RFC] Runtime-inited versus Hardcoded tables.

[Zdenek Kabelac]

2008/1/31, Michael Niedermayer <michaelni at gmx.at>:
> On Thu, Jan 31, 2008 at 10:01:28AM +0100, Zdenek Kabelac wrote:
> > 2008/1/31, Michael Niedermayer <michaelni at gmx.at>:
> > > On Thu, Jan 31, 2008 at 12:20:34AM +0100, Zdenek Kabelac wrote:
> > > > 2008/1/30, Michael Niedermayer <michaelni at gmx.at>:
> > > > > On Tue, Jan 29, 2008 at 07:09:39PM +0100, Zdenek Kabelac wrote:
> > > > > > 2008/1/12, Diego 'Flameeyes' Petten? <flameeyes at gmail.com>:
> > > > > > > Roman Shaposhnik <rvs at sun.com> writes:
> > > > > > >
> > > > >
> > > > >
> > > > > > but I think increasing the
> > > > > > library size for a megabyte is IMHO bad idea - at runtime the user
> > > > > > will hardly use few codecs  at same time and the price of binary
> > > > > > distribution of such monster hardly compressible binary tables vie net
> > > > > > will ever hardly justify few ms you spend via runtime initialization.
> > > > > >
> > > > > > All I would like to see is moving static tables to static pointer
> > > > > > which would be initialized at the codec's first initialization and
> > > > > > released when library is being freed. This is actually far more space
> > > > > > saving than your proposal :)
> > > > >
> > > > > This by far is the worst possible way by which one can allocate static
> > > > > storeage.
> > > > >
> > > > > If one uses
> > > > > static table[1234];
> > > > > and initalizes it during init. Then this means small binary, and only
> > > > > wasted memory if the table is initalized. Though it is duplicated for
> > > > > each process which initalized it.
> > > > >
> > > > > If one uses
> > > > > static table[1234]= {1,2,5,6,7,0,0,1,2,
> > > > > Then this means large binary, and some always wasted memory but it is
> > > > > shared between processes.
> > > > >
> > > > > If one uses
> > > > > static *table;
> > > > > and malloc() and initialization during init. Then this is identical to the
> > > > > first except that it causes a memleak on library unload on at least some OSs.
> > > > >
> > > > > What you suggest is the 3rd variant with a workaround to its memleak
> > > > > requireing thread synchronization.
> > > >
> > > > Well yes - except I do not see the problem with the memory releasing -
> > > > I think each platform supports in some way the callback to be called
> > > > by library when it's being deallocated (i.e. similar to atexit()) -
> > > > and you don't need the mutex IMHO - as in the case of library being
> > > > removed its the application responsibility to have all the
> > > > codecs/muxers/... closed at this time.
> > >
> > > Not on dealloc, on ALLOC, the tables are allocated when they are needed
> > > for the first time, this DOES need a mutex the way you did implement it,
> > > that is with a non constant global realloc()ed array of pointers.
> > > The code is just broken! If 2 threads open the same codec at the same
> > > time random things can happen, including arbitrary code execution exploits.
> >
> > At the time of this implementation it was fully confirming the API of
> > that time - it was clearly stated that the application had to ensure
> > thread safety during the codec opening - I'm not sure you are so mad
> > about this???
>
> Iam mad abaout it because it is COMPLETELY unneeded complexity. And it
> was called from many more places than codec init, i dont know if that was
> initially or added later, but it was.
> Also as i said it causes a memleak unless you call that static free function
> before unloading the lib. Which mean even more _UNPORTABLE_ complexity.
>

I was aware only about some mp3 table initilization and they were IMHO
quite clean. And yes many things are complex and unportable  - that's
live :)
There is avcodec_register_all() - so there would be
avcodec_unregister_all() - everything could be done in clean way...


>
> >
> > > >
> > > > IMHO I don't see any hack in this - but if you think it's fine to
> > > > waste couple megs of memory and you expect that every OS is smart
> > > > enough to use the COW mechanism for BSS allocation - then of course
> > > > there is need to do any changes.
> > > >
> > > > Anyway - everyone has 2G of RAM these days so who cares :) I
> > >
> > > $free -m
> > >              total       used       free     shared    buffers     cached
> > > Mem:           181        177          3          0          0         49
> > > -/+ buffers/cache:        127         53
> > > Swap:          980        357        622
> > >
> > > And which OSs dont support COW?
> >
> > Are you sure the OS like Windows95 or Amiga have this advantage to map
> > '/dev/zero' pages for .bss sections ?
>
> I didnt ask about which OSs you are not sure about. My question was if anyone
> knew for sure one which didnt support shared readonly table or COW.


Obviously I'm not a Windows expert and I do not know about anyone who
works with Windows and knows anything more advanced about what is
hidden below the dll-hell ;)

But according to this article :

 http://support.microsoft.com/kb/103858

I'd have guessed that COW is a new WinNT feature so the Windows
kernels which are not NT-based do not have this.

Zdenek