[MPlayer-dev-eng] NUT dts proposal

[Rich Felker]

After more discussion on irc, I'm convinced that the current "dts"
(decode timestamp) spec in NUT has both some pros and cons..

Pros:
- dts is very easy to compute.
- no future context required.
- for streams with a steady flow of frames, accurately represents the
optimal time to do the actual decoding for smooth cpu utilization.

Cons:
- in the presence of long gaps with no frames, dts requires the first
decode_delay frames after the gap to be stored before the gap, making
them impossible to find efficiently.
- generally makes it difficult to search out a frame by pts, since the
storage location of a frame might vary considerably from the location
of frames in other streams with similar pts (mainly if framerates are
not comparable).

Recently uau proposed on IRC, and I independently came up with
something similar as well, for an alternate definition of dts:

frame[n].dts = min(frame[n].pts, frame[n+1].pts, ..., frame[n+d].pts)
(where d=decode_delay)

Consequences:

Consider the typical IPB stream:

Muxed frame order:  I0 P3 B1 B2 P6 B4 B5 ...
Current dts system: -1  0  1  2  3  4  5
Possible new dts:    0  1  1  2  4  4  5

And the pathological IPB stream with huge gaps:

Muxed frame oder:   I0 P3 B1 B2 [gap] I100 P103 B101 B102
Current dts system: -1  0  1  2          3  100  101  102
Possible new dts:    0  1  1  2        100  101  101  102

The biggest apparent problem with the possible new system is that you
(in principle) have to look at future context to know dts. This would
appear to introduce latency in muxing and whatnot. In actuality, you
may or may not know additional information that makes it possible to
know dts in advance. The best part is, to mux you don't have to
actually compute the dts as long as you can ensure that packets are
written in the same order they would be if you had computed dts for
each. :)

The biggest advantage of this proposal is that all frames are stored
as close as possible to the position determined by their pts, never
later than their pts, and only at a position earlier than their pts if
they serve as a reference frame for earlier frames. Presumably this
makes seeking work "nicely".

Personally, I don't really like either method of assigning dts.
(Actually I despise out of order decoding... >_<) If someone could
propose something better, such as a reasonable way around the problems
with the existing system, I'd be very happy to hear it!

Rich