[MPlayer-dev-eng] Patches for NUT

[Oded Shimon]

On Thu, Feb 09, 2006 at 06:43:30PM +0100, Michael Niedermayer wrote:
> On Thu, Feb 09, 2006 at 03:04:18PM +0200, Oded Shimon wrote:
> [...]
> > > and iam not happy about the 24bit hack either, either store adler32 or crc32
> > > everywhere IMHO
> > 
> > I suppose crc32 then, it's more common...
> 
> ok

Can I use the lutless crc in libnut or is it unusably slow? The biggest 
single chunk passed to it is 80kb index, and a total of 200-300kb of 
syncpoint data, with 15 byte chunks 30,000 times...

> > Makes it kind of hard to measure though, I suppose real time would be the 
> > right way... Or basically just specify a timebase and the max pts 
> > difference would be a single unit of that timebase (so, 1/2 for making max 
> > pts diff without a syncpoint never more than half a second.)
> > 
> > I like this solution the most.. In implementation:
> > 
> > tmp = convert_ts(max_last_pts, max_last_pts_timebase, this_timebase);
> > if (compare_ts(ABS(this_pts - tmp), this_timebase, 1, max_pts_timebase) >= 0) put_syncpoint();
> > 
> > Maybe without the ABS. Whatever.
> 
> hmm somehow i dont like this, convert_ts & compare_ts are slow, we
> should avoid executing them too often, this would need 3xconvert_ts() per
> frame on the _demuxer_ side just to check if the pts are ok
> a common timebase would only need 1 convert_ts()

Rich suggested an implementation for compare_ts that does not require 
convert_ts or division (long 1st grade multiplication). As for convert_ts, 
it's use could be eliminated by preparing threshold in every timebase at 
init, and:
(t2 - t1 > thres)

Is identical to:
(t2 - thres > t1)

So you could do this whole thing using a single call to compare_ts alone. 
(There is already an additional call for compare_ts for last_dts checking 
- actually, currently, there is one compare_ts PER stream for checking 
against all last_dts. This can be changed into 2 compare_ts by storing 
max_last_dts - one for checking, and one for checking if you are higher 
than max)

BTW, for simplicity, I think the pts to be used for checking threshold 
should be simply the pts of the last frame written to file. To use max_ 
you'll need to add a compare_ts .

Also, a muxer should be careful to not make the threshold bigger than any 
timebase, this would cause a syncpoint to be written before every frame...

> > Obviously this idea has many flaws...
> > 
> > > C: give syncpoints and frame headers their own checksums, add a checksum
> > >    flag to the frame header table
> > >    this avoids the 1 checksum covers syncpoint and following header 
> > >    complexity and the overhead of case A for large pts frames is 
> > >    significantly reduced, there is also no issue with storing the max_pts
> > >    after muxing (seeking back or truncation issue)
> > > 
> > > personally iam in favor of C
> > 
> > The complexity is litte, syncpoint are always accompanied by their 
> > following frame, you can't have a syncpoint without a frame. But I am 
> > a bit confused, you want the ability to store a 4 byte checksum for a 3 
> > byte frame header? And now both syncpoint and the following frame header 
> > needs a total of 8 bytes of checksum? This sounds like horrible overhead. 
> 
> 8 bytes per 16384 bytes isnt that much, iam just fearing few demuxers will
> check the checksum during binary search at all if they need to decode the
> following frame header, does/will libnut?

I plan on adding a get_frame_header() function which will ease that for me, 
basically find_syncpoint() will just ignore the possible syncpoint it found 
if it did not match checksum. And I think an additional 8 bytes per 
syncpoint is a lot of overhead... (200kb for 700mb file, we have now a 
total 900kb of overhead, maybe less.)

I'll need the equivalent put_frame_header() for muxer put_syncpoint() 
too...


I'm not sure how I feel about 'checksum flag' regardless. It might be nice, 
but on other hand it's more spec/muxer/demuxer bloat...

> > > > 2. index_ptr comes out of nowhere when reading the file linearly
> > > > 3. split up index?
> > > 
> > > yes that would be nice
> > 
> > Well, best I got, just split em up at some syncpoint position, it could 
> > work just fine, but I don't really see the point. Damage usually comes in 
> > blocks, and yes the index is big, but now you can repeat it. (which is also 
> > highly non trivial if you repeat anywhere except EOF and begginning of 
> > file, the vlc's will keep changing lengths and hence changing all other 
> > vlc's...)
> > What is the rule for splitting? Every 5,000 syncpoint? When the index 
> > reaches 4kb? (not a good idea, near impossible to measure, because there 
> > are 2 parts.)
> 
> i think we need to better understand the distribuion of errors on common
> media, it would be silly to complicate the index unneccesariely but it
> would also be silly if a 1k lost block in a 100k index makes a files
> sloooow-seekable

If you are talking about index, then you are not talking about 
streaming, then the only damage left is either p2p and local storage. 
AFAIK, those 2 only ever give damage in chunks, usually big chunks... BTW, 
if you truncate a NUT file by a single byte (or, even append a single byte), 
libnut will not be able to read the index, because there is no index_ptr...

- ods15