[MPlayer-dev-eng] More on timestamps in NUT

[Michael Niedermayer]

Hi

On Monday 03 May 2004 07:01, D Richard Felker III wrote:
[...]
> First, an explanation of the problem:
>
> Michael noted on the cvslog list (in a thread originating from some
> changes he made to mpcf.txt, the in-progress NUT spec) some
> unfortunate consquences of the current timestamp coding system in NUT.
> In the working spec, the muxer and demuxer keep track of the three
> most recent timestamp deltas for each stream, and then subsequent
> frames can be coded to reuse these deltas, rather than storing the lsb
> timestamp or full timestamp explicitly. This is particularly useful
> for video streams with B frames (which will have a timestamp pattern
> of +N+1, -N, +1 (N times), where N is the number of B frames, and for
> vorbis audio where there are 3 size frames (128, 576, and 1024), and
> it allows most packet headers to be encoded in a single byte for
> very-low-bitrate streams (where overhead matters a lot).
>
> The problem arises with error resilience. I had been supposing that
> after an error, we could resync to the next valid packet using some
> nice tricks I worked out (which are explained in another thread). But
> Michael pointed out that such damage can mess up the timestamp delta
> prediction entirely, leading to completely bogus timestamps. Thus, it
> seems that in the presence of timestamp delta predictors, error
> resilience can only recover at the next lsb-coded or fully-coded
> timestamp.
[...]
> THEREFORE, I propose that we remove timestamp delta prediction from
> NUT, and put in its place fixed timestamp deltas in the framecode
> table.
agree

> (As a less radical proposal, we could choose to support both 
> and strongly recommend that delta prediction NOT be used.)
>
> On to the next topic...
>
> The current draft of the spec calls for each stream to fully code the
> timestamp in its next frame after a type-2 startcode. Depending on the
> time base units in use, this results in an overhead of at least 8+3*N
> bytes per startcode, where N is the number of streams (for a 1-2 hour
> movie, you need at least 3 bytes to store a timestamp, or worse if you
> choose/need a bad time base). For error resilience purposes, it may be
> desirable to put startcodes fairly frequently, and at low bitrates
> this could result in considerable overhead.
>
> If we have lots of streams, it seems redundant to code the full
> timestamps for ALL of them. In fact, all the timestamps should be
> approximately the same, due to proper interleaving. Let's take
> advantage of that redundancy by storing a single timestamp with the
> startcode, and calling the startcode+timestamp unit a "sync point".
> This timestamp can be in a global time base, specified in the global
> header.
>
> Having a global time base (that may, but doesn't necessarily,
> correspond to one or more of the stream time bases) has an additional
> advantage, in that we can use it for indexing. Our intent has been
> that the index would point to startcodes anyway, rather than packets
> of a particular stream, so it makes sense for the startcodes to have
> their own timestamps. When a demuxer encounters a sync point, it would
> convert the associated timestamp into the separate time bases of each
> stream, and consider future delta/lsb timestamps to be relative to
> that time.
>
> Note that under this system, the overhead from startcodes is reduced
> from 8+3*N (or more) to something like 11+N. Or, in the case of a
> naive muxer that is always coding the lsb timestamps on each frame,
> the additional overhead for a sync point is only a constant 11 bytes
> (or 10 or 12, depending on the time base)!
>
> THEREFORE, I propose that we replace the type-2 startcodes with a sync
> point packet containing its own timestamp, specify that subsequent
> relative/lsb timestamps are based on the sync point timestamp, and
> remove the requirement that frames following a type-2 startcode fully
> code their timestamps.
agree

>
> Finally, I want to get "type-1" start codes into the spec. We've been
> talking about them long enough, but they're still not there.
> Basically, "type-1" startcodes, which I'll call recovery points, are
> short 3-byte startcodes used to aid in error recovery. The reason they
> can be short is that they do not need to effectively avoid collisions,
> because unlike sync points they will not be used for seeking. Instead
> they are used after an invalid packet is decoded in order to find the
> next valid packet. Demuxers that don't want perfect error resilience
> can just resume at the next recovery point or sync point, while
> "hardcore" demuxers can use a brute force approach of testing each
> byte to see if it is the start of a chain leading up to the next
> recovery or sync point.
>
> THEREFORE, I propose that we add optional 3-byte recovery points to
> the nut spec, which muxers can use at their discression to improve the
> error resilience of the file. (Discussion point: perhaps 2 bytes is
> enough?)
agree
btw, i did some tests a few days ago, and interrstingly there are several 
3byte sequences starting with 'N' which never occred in ~500mb of test 
videos, for 2 bytes the best (0x4EFE) was IIRC occuring approximately once 
every 100k but thats just IIRC, as i cant find my notes about it ATM

[...]

-- 
Michael
level[i]= get_vlc(); i+=get_vlc();		(violates patent EP0266049)
median(mv[y-1][x], mv[y][x-1], mv[y+1][x+1]);	(violates patent #5,905,535)
buf[i]= qp - buf[i-1];				(violates patent #?)
for more examples, see http://mplayerhq.hu/~michael/patent.html
stop it, see http://petition.eurolinux.org & http://petition.ffii.org/eubsa/en