[FFmpeg-devel] [RFC] Runtime-inited versus Hardcoded tables.

[Michael Niedermayer]

On Thu, Jan 31, 2008 at 10:01:28AM +0100, Zdenek Kabelac wrote:
> 2008/1/31, Michael Niedermayer <michaelni at gmx.at>:
> > On Thu, Jan 31, 2008 at 12:20:34AM +0100, Zdenek Kabelac wrote:
> > > 2008/1/30, Michael Niedermayer <michaelni at gmx.at>:
> > > > On Tue, Jan 29, 2008 at 07:09:39PM +0100, Zdenek Kabelac wrote:
> > > > > 2008/1/12, Diego 'Flameeyes' Petten? <flameeyes at gmail.com>:
> > > > > > Roman Shaposhnik <rvs at sun.com> writes:
> > > > > >
> > > >
> > > >
> > > > > but I think increasing the
> > > > > library size for a megabyte is IMHO bad idea - at runtime the user
> > > > > will hardly use few codecs  at same time and the price of binary
> > > > > distribution of such monster hardly compressible binary tables vie net
> > > > > will ever hardly justify few ms you spend via runtime initialization.
> > > > >
> > > > > All I would like to see is moving static tables to static pointer
> > > > > which would be initialized at the codec's first initialization and
> > > > > released when library is being freed. This is actually far more space
> > > > > saving than your proposal :)
> > > >
> > > > This by far is the worst possible way by which one can allocate static
> > > > storeage.
> > > >
> > > > If one uses
> > > > static table[1234];
> > > > and initalizes it during init. Then this means small binary, and only
> > > > wasted memory if the table is initalized. Though it is duplicated for
> > > > each process which initalized it.
> > > >
> > > > If one uses
> > > > static table[1234]= {1,2,5,6,7,0,0,1,2,
> > > > Then this means large binary, and some always wasted memory but it is
> > > > shared between processes.
> > > >
> > > > If one uses
> > > > static *table;
> > > > and malloc() and initialization during init. Then this is identical to the
> > > > first except that it causes a memleak on library unload on at least some OSs.
> > > >
> > > > What you suggest is the 3rd variant with a workaround to its memleak
> > > > requireing thread synchronization.
> > >
> > > Well yes - except I do not see the problem with the memory releasing -
> > > I think each platform supports in some way the callback to be called
> > > by library when it's being deallocated (i.e. similar to atexit()) -
> > > and you don't need the mutex IMHO - as in the case of library being
> > > removed its the application responsibility to have all the
> > > codecs/muxers/... closed at this time.
> >
> > Not on dealloc, on ALLOC, the tables are allocated when they are needed
> > for the first time, this DOES need a mutex the way you did implement it,
> > that is with a non constant global realloc()ed array of pointers.
> > The code is just broken! If 2 threads open the same codec at the same
> > time random things can happen, including arbitrary code execution exploits.
> 
> At the time of this implementation it was fully confirming the API of
> that time - it was clearly stated that the application had to ensure
> thread safety during the codec opening - I'm not sure you are so mad
> about this???

Iam mad abaout it because it is COMPLETELY unneeded complexity. And it
was called from many more places than codec init, i dont know if that was
initially or added later, but it was.
Also as i said it causes a memleak unless you call that static free function
before unloading the lib. Which mean even more _UNPORTABLE_ complexity.


> 
> > >
> > > IMHO I don't see any hack in this - but if you think it's fine to
> > > waste couple megs of memory and you expect that every OS is smart
> > > enough to use the COW mechanism for BSS allocation - then of course
> > > there is need to do any changes.
> > >
> > > Anyway - everyone has 2G of RAM these days so who cares :) I
> >
> > $free -m
> >              total       used       free     shared    buffers     cached
> > Mem:           181        177          3          0          0         49
> > -/+ buffers/cache:        127         53
> > Swap:          980        357        622
> >
> > And which OSs dont support COW?
> 
> Are you sure the OS like Windows95 or Amiga have this advantage to map
> '/dev/zero' pages for .bss sections ?

I didnt ask about which OSs you are not sure about. My question was if anyone
knew for sure one which didnt support shared readonly table or COW.

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Breaking DRM is a little like attempting to break through a door even
though the window is wide open and the only thing in the house is a bunch
of things you dont want and which you would get tomorrow for free anyway
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080131/bef09e32/attachment.pgp>