[NUT-devel] Broadcasting nuts [PATCH]

[Måns Rullgård]

Michael Niedermayer <michaelni at gmx.at> writes:

> On Tue, Feb 05, 2008 at 09:41:49PM +0000, Måns Rullgård wrote:
>> Michael Niedermayer <michaelni at gmx.at> writes:
>> 
>> > Hi
>> >
>> > Attached patch should make broadcast of single program nuts possible.
>> > Its the minimal/least restrictive solution i found.
>> >
>> > Comments welcome, especially from mans/nico, others who know mpeg-ps/ts.
>> >
>> > Index: nut.txt
>> > ===================================================================
>> > --- nut.txt	(revision 588)
>> > +++ nut.txt	(working copy)
>> > @@ -412,6 +412,7 @@
>> >  syncpoint:
>> >      global_key_pts                      t
>> >      back_ptr_div16                      v
>> > +    transmit_ts                         t
>> >      reserved_bytes
>> 
>> You could make this field optional.
>
> ok
>
>> 
>> >              Complete definition:
>> > @@ -775,6 +776,11 @@
>> >      global_key_pts MUST be bigger or equal to dts of all past frames across
>> >      all streams, and smaller or equal to pts of all future frames.
>> >  
>> > +transmit_ts (t)
>> > +    The timestamp at which the first bit of the syncpoint is transmitted.
>> > +    MUST be less than or equal to all following dts.
>> > +    See broadcast buffering model.
>> 
>> This is not strict enough.  The transmit_ts must be less than the DTS
>> of any not completely received frame.
>
> Frames cannot be split in nut, thus i think this is sufficient. Also there is
> no problem in principle by instantaneosly inserting and removing a frame at
> the same time. That is transmit_ts == dts.

I still think there's a problem.  Consider this sequence:

  syncpoint, transmit_ts=T
  frame header, dts=T
  frame data

With your rule, this would be valid, even though the frame data would
not yet be received at the DTS of the frame.  Frames may be decoded
instantaneously, but they are not received instantaneously.

>> > +As data is received (between the 2 transmit_ts of the 2 enclosing syncpoints)
>> > +it is inserted into a FIFO buffer. The spec does not mandate a single or
>> > +multiple FIFOs,. Nor that demuxing is done before or after the FIFO. Just
>> > +that a valid nut broadcast never causes an overflow nor underflow in the
>> > +target receiver/demuxer/decoder.
>> > +
>> > +Frames are instantaneosly removed from the fifo at their dts and inserted
>> > +into the decoder.
>> 
>> These paragraphs are not really necessary.  If you want to provide a
>> timing/buffering model akin to that in the MPEG spec, you have to be
>> much more precise.
>
> I would like to provide what is needed to make nut useable for broadcast.
> If that ends similar to mpeg or not is not of importance to me.
> The question here is, if it makes sense to specify precissely how buffering
> works. After all even if it is specified here it depends on the
> demuxer/decoder capabilities. And a demuxer/decoder which places the buffers
> differently from how the idealized buffer model does, will likely need bigger
> buffers. Thus it might be advantaneous to treat the FIFO placement like the
> FIFO size. And rather specify them together for a specific profile or other
> seperate spec.

The MPEG spec defines a hypothetical "system target decoder" in great
detail, providing a reference for the buffer sizes and bitrates coded
in various fields.  A conforming implementation may be constructed in
any way whatsoever, provided it can correctly decode any stream the
system target decoder would.  Conversely, an MPEG stream is not valid
if it cannot be processed by the system target decoder.

Since the Nut spec makes no mention of buffer sizes or bitrates
elsewhere, it would make sense to leave all such concerns to be
settled outside the scope of Nut.

> Index: nut.txt
> ===================================================================
> --- nut.txt	(revision 588)
> +++ nut.txt	(working copy)
> @@ -412,6 +412,8 @@
>  syncpoint:
>      global_key_pts                      t
>      back_ptr_div16                      v
> +    if(bytes remaining in the syncpoint)
> +        transmit_ts                     t
>      reserved_bytes
>  
>              Complete definition:
> @@ -775,6 +777,12 @@
>      global_key_pts MUST be bigger or equal to dts of all past frames across
>      all streams, and smaller or equal to pts of all future frames.
>  
> +transmit_ts (t)
> +    The timestamp at which the first bit of the syncpoint is transmitted.
> +    MUST be less than or equal to all following dts.
> +    This field may only be ommited if the file is not intended for broadcast.
> +    See broadcast buffering model.

See above.

>  Index tags:
>  -----------
>  
> @@ -978,6 +986,39 @@
>  
>  Demuxers SHOULD NOT search the whole file for info packets.
>  
> +
> +Broadcast buffering model:
> +--------------------------
> +Nut files can be broadcast. Such specific nut files must be encoded and
> +muxed so as to not exceed the decoder side buffering or available bandwidth.
> +This spec does not provide any restrictions on the decoder buffers or
> +bandwidth. Nor does it mandate that the channel be transmitting at a constant
> +rate.
> +For sake of simplicity a zero latency channel is assumed. Note, due to the
> +lack of a back channel any constant latency channel is indistingiushable from
> +a zero latency channel.
> +The time at which a syncpoint is transmitted as well as received is stored in
> +the transmit_ts of it. The receiver/demuxer/decoder can use these timestamps
> +to synchronize its clock. The (possibly variable) rate at which the following
> +bits/bytes are transmitted depends on the hardware and is outside this spec.
> +Byte n of the file must be transmitted before or at the same time as byte n+1.

Is this just another way of saying that the stream must be transmitted
sequentially?  I try to avoid the word "file" when discussing
streaming, since it is entirely plausible (I'd even say likely) that
the data never exists as a file in the usual sense of the word,
i.e. it is transmitted directly from an encoder source without
intermediate storage (think of live broadcasts).

-- 
Måns Rullgård
mans at mansr.com