[MPlayer-dev-eng] Lots of stuff for NUT

[Oded Shimon]

On Sat, Dec 31, 2005 at 04:54:25PM +0100, Michael Niedermayer wrote:
> Hi
> 
> On Sat, Dec 31, 2005 at 04:27:59PM +0200, Oded Shimon wrote:
> [...]
> > @@ -221,12 +229,37 @@
> >  index:
> >      index_startcode                     f(64)
> >      packet header
> > -    stream_id                           v
> >      max_pts                             v
> > -    index_length                        v
> > -    for(i=0; i<index_length; i++){
> > -        index_pts                       v
> > -        index_position                  v
> > +    syncpoints                          v
> > +    for(i=0; i<syncpoints; i++){
> > +        syncpoint_pts                   v
> > +        syncpoint_pos_div8              v
> > +    }
> > +    for(i=0; i<stream_count; i++){
> > +        type                            v
> > +        j=0
> > +        if (type) while (j<syncpoints){
> > +            repeat                      v
> > +            b=repeat&1
> > +            repeat=(repeat>>1)+1
> > +            for(k=0; k<repeat; k++){
> > +                syncpoint[j+k].stream[i].back_ptr= syncpoint[j-b].pos_div8
> > +            }
> > +            j+= repeat
> > +        } else while (j<syncpoints) {
> > +            repeat                      v
> > +            repeat++
> > +            b=repeat&1
> > +            repeat=repeat>>1
> > +            for(k=0; k<repeat; k++){
> > +                syncpoint[j+k].stream[i].back_ptr= syncpoint[j+k-b].pos_div8
> > +            }
> > +            if (!repeat) {
> > +                syncpoint[j].stream[i].back_ptr= syncpoint[j-1].stream[i].back_ptr
> > +                j++
> > +            }
> > +            j+= repeat
> > +        }
> 
> do we really need to allow a backptr to point to the current syncpoint? what
> do we loose if we would point to the previous?

Well, an unnecessary seek. I think it also causes breakage with my keyframe 
rule, as it's based on back_ptr being able to be zero.
Yes, it breaks the definition of back_ptr - back_ptr is the closest 
syncpoint so that there's a keyframe between that syncpoint and this 
syncpoint. If you point one syncpoint back, the linear search would be 
looking for a keyframe, end up in another syncpoint, and not find any 
keyframe!

However I must admit that "ignoring" this zero back_ptr extremely reduces 
index size, atleast the repeat part. goes from 4-10kb to 4-10 BYTES.

Let me see if it really is meaningful...

.. S2 K..... S1 K .... P


I think you can just extend the rules I gave for seeking by adding:

(to recall:
1. you pass P
2. you pass max_distance from last keyframe in this stream (X)
3. you see S3, AND "S1 - (last keyframe)" is bigger than max_distance.
)

If you're using index, and back_ptr is pointing exactly 1 syncpoint 
backwards, read atleast until you reach S1, read the back_ptr's there, and 
"restart" accordingly.


Do you think we should not make a difference between them in the index 
then?

BTW, I'm not sure if it's worth it, but we might want to make 'type' 
setable by repeat's lsb. In the case of subtitles, you'll have these back 
ptr's...

1 2 3 4 0 0 0 1 2 3 4 5 0 0 0 0 0 ...

Which is basically bringing out the worst in both methods. I'll investigate 
this later...

> > +    syncpoints MUST be placed immediately before a keyframe if more
> > +    than max_distance passed since previous keyframe of the same stream,
> > +    unless the timestamp to be set for this syncpoint is identical to the
> > +    last syncpoints's timestamp. In which case, the syncpoint is to be put
> > +    immediately BEFORE the timestamp will change. (So it will still have
> > +    the same timestamp)
> 
> i dont like having multiple syncpoints with the same timestamp, this is going
> to cause problems iam pretty sure

Trying to think how this works with binary search...
The binary search will always pick the "biggest" syncpoint that's still 
smaller than P. It'll work fine.

..S1..S1..S1..P..S2..S2..S3..

If I end up in any of the first S1's, the next binary search is bound to 
find the other S1's, and they will replace the other ones, as the rule of 
the binary search goes. (if (result < P) lo = guess; else hi = guess; )

Besides, syncpoints with same pts will ALWAYS happen, due to max_distance 
...

> [...]
> >  flags[frame_code]
> > -    first of the flags from MSB to LSB are called KD
> > -    if D is 1 then data_size_msb is coded, otherwise data_size_msb is 0
> > -    K is the keyframe_type
> > -        0 -> no keyframe,
> > -        1 -> keyframe,
> > -    flags=4 can be used to mark illegal frame_code bytes
> > -    frame_code=78 must have flags=4
> > -    Note: frames MUST NOT depend(1) upon frames prior to the last
> > -          frame_startcode
> > -    Important: depend(1) means dependency on the container level (NUT) not
> > -    dependency on the codec level
> > +    Bit  Name             Description
> > +      1  data_size_msb    if set, data_size_msb is at frame header,
> > +                          otherwise data_size_msb is 0
> > +      2  more_flags       if set, stream control flags are at frame header.
> > +      4  invalid          if set, frame_code is invalid.
> > +
> > +    frame_code=78 ('N') MUST have flags=64
> > +
> > +stream_flags
> > +    stream_flags is "stream_flags[frame_code] | coded_stream_flags"
> > +
> > +    Bit  Name             Description
> > +      1  is_key           if set, frame is keyframe
> > +      2  end_of_relavence if set, stream has no relavence on
> > +                          presentation until next frame. (EOR)
> > +
> > +    EOR frames MUST be zero-length and must be set keyframe. EOR is unset
> 
> hmm, this is somewhat redundant, zero byte keyframes cant occur, so 
> zero byte keyframes could mean EOR

Hmm, well, not necessarily, you never know with weird codecs...

> > +    by the first frame following the EOR of the same stream.
> 
> and what is this good for? why should we allow anything to follow the
> end of a stream?
> 
> > +    When EOR is set, all back_ptr's for this stream are set to zero.
> > +    All streams SHOULD end with EOR.
> 
> how exactly do you seek if the back_ptrs dont point to the previous keyframe?

Seems you misunderstand EOR - It's not "end of stream" (old name was bad), 
it's "end of relavence".
for ex., a subtitle stream and there's currently no subtitle showing, 
that's EOR. The stream is completely irrelavent for presenting this part of 
the video. Basically it's any "gap" in a stream.

back_ptr's are zero because they are just not interesting, the stream is 
completely "irrelavent".

> [...]
> 
> iam not sure if i like the mts stuff ...

It has one big flaw IMO, it makes the muxer knowing the future manditory. 
Or at the very least a smart muxer-caller.

I decided with Rich that the API for the muxer will be that you MUST pass 
it the mts of a frame whenever you pass it a frame. Meaning either the 
caller has to be smart, or it has to use the high level reorderer, which 
always uses buffering.
MTS however is still the most correct way to store the data IMO... old dts 
method is wrong (depends on decode_delay), and MN rule is insufficient.

We are also discussing adding another stream control flag: "no frames past 
this one with a lower pts", so a player knows when to stop reading ahead 
for playing. Another way to define this: "mts == pts" for this frame.
I want to implement this first and see how much overhead it takes before we 
decide to add it or not. If neglagent, IMO it should be added.

- ods15