[FFmpeg-devel] [RFC] Runtime-inited versus Hardcoded tables.

[Zdenek Kabelac]

2008/1/31, Michael Niedermayer <michaelni at gmx.at>:
> On Thu, Jan 31, 2008 at 12:20:34AM +0100, Zdenek Kabelac wrote:
> > 2008/1/30, Michael Niedermayer <michaelni at gmx.at>:
> > > On Tue, Jan 29, 2008 at 07:09:39PM +0100, Zdenek Kabelac wrote:
> > > > 2008/1/12, Diego 'Flameeyes' Petten? <flameeyes at gmail.com>:
> > > > > Roman Shaposhnik <rvs at sun.com> writes:
> > > > >
> > >
> > >
> > > > but I think increasing the
> > > > library size for a megabyte is IMHO bad idea - at runtime the user
> > > > will hardly use few codecs  at same time and the price of binary
> > > > distribution of such monster hardly compressible binary tables vie net
> > > > will ever hardly justify few ms you spend via runtime initialization.
> > > >
> > > > All I would like to see is moving static tables to static pointer
> > > > which would be initialized at the codec's first initialization and
> > > > released when library is being freed. This is actually far more space
> > > > saving than your proposal :)
> > >
> > > This by far is the worst possible way by which one can allocate static
> > > storeage.
> > >
> > > If one uses
> > > static table[1234];
> > > and initalizes it during init. Then this means small binary, and only
> > > wasted memory if the table is initalized. Though it is duplicated for
> > > each process which initalized it.
> > >
> > > If one uses
> > > static table[1234]= {1,2,5,6,7,0,0,1,2,
> > > Then this means large binary, and some always wasted memory but it is
> > > shared between processes.
> > >
> > > If one uses
> > > static *table;
> > > and malloc() and initialization during init. Then this is identical to the
> > > first except that it causes a memleak on library unload on at least some OSs.
> > >
> > > What you suggest is the 3rd variant with a workaround to its memleak
> > > requireing thread synchronization.
> >
> > Well yes - except I do not see the problem with the memory releasing -
> > I think each platform supports in some way the callback to be called
> > by library when it's being deallocated (i.e. similar to atexit()) -
> > and you don't need the mutex IMHO - as in the case of library being
> > removed its the application responsibility to have all the
> > codecs/muxers/... closed at this time.
>
> Not on dealloc, on ALLOC, the tables are allocated when they are needed
> for the first time, this DOES need a mutex the way you did implement it,
> that is with a non constant global realloc()ed array of pointers.
> The code is just broken! If 2 threads open the same codec at the same
> time random things can happen, including arbitrary code execution exploits.

At the time of this implementation it was fully confirming the API of
that time - it was clearly stated that the application had to ensure
thread safety during the codec opening - I'm not sure you are so mad
about this???

> >
> > IMHO I don't see any hack in this - but if you think it's fine to
> > waste couple megs of memory and you expect that every OS is smart
> > enough to use the COW mechanism for BSS allocation - then of course
> > there is need to do any changes.
> >
> > Anyway - everyone has 2G of RAM these days so who cares :) I
>
> $free -m
>              total       used       free     shared    buffers     cached
> Mem:           181        177          3          0          0         49
> -/+ buffers/cache:        127         53
> Swap:          980        357        622
>
> And which OSs dont support COW?

Are you sure the OS like Windows95 or Amiga have this advantage to map
'/dev/zero' pages for .bss sections ?

Zdenek