Broadcast sufficiency - my assumptions/reasoning/design for it
OK since there seems to be a disagreement (to put it lightly :) as to what the current NUT framework can do for synchronizing timing between a transmitter and receiver, I'd like to lay down the hypotheses and reasoning I'm working from. Maybe others disagree on the basic assumptions or haven't though of them. So, here it goes: Transmitter side: Key assumption is that frames are actually transmitted according to dts. In a VBR transmission with immensely excessive bandwidth, or a pure CBR (e.g. uncompressed, single-stream) transmission, this can be done almost exactly. In CBR with buffer restraints, obviously it will not be exact. It is sufficient that the mean difference between dts and actual transmission time be constant over intervals of time whose length is on the order of the maximal length of time in which transmitter and receiver clocks can be expected to remain within an acceptable error relative to one another. Ensuring that this mean difference requirement can be met is a matter of bitrate constraints on the codecs in use. Unless my casual estimates are horribly mistaken, it is much easier to meet these requirements than to meet most buffering requirements, i.e. any broadcast setup already meeting moderate to strict buffer size constraints will automatically meets the requirements for the transmitter to be able to ensure the mean difference requirement. Note that if content is created live and transmitted with minimal delay, the constraint will automatically be met. Only non-live transmissions need to make particular efforts to synchronize transmit times with dts, and even then if bitrate/buffer constraints are strict it may be trivial. Many years ago I wrote a non-live streaming server based on ogg container (eew!) which synchronized transmission of ogg pages to the system clock and it involved very little effort. Receiver side: Upon receiving each frame, the receiver stamps it with the actual time the frame was received, according to the local clock. Drift in the mean difference between receive-timestamps and dts is tracked and used to adjust the local clock as it diverges from the transmitter's clock. Note that there's nothing NUT-specific about this design. It will work with any existing container, and it can be used in place of the redundant timestamps in MPEG if one desires. In the interests of improving the media software/hardware landscape in general, it's much more worthwhile to have designs that work with any container, and to push developers to move towards supporting those rather than highly container-specific or application-specific ("application" in the sense of "broadcast tv" or "video on demand", not "firefox" or "ms word") implementations of synchronization. I'd be very happy to see (maybe even help write) an RFC on synchronization of unidirectional broadcast streams for arbitrary streamable containers. Is anyone willing to consider that as a constructive alternative to arguing about putting MPEG-type stuff in NUT? Rich
On Wed, Feb 06, 2008 at 12:08:37PM -0500, Rich Felker wrote:
OK since there seems to be a disagreement (to put it lightly :) as to what the current NUT framework can do for synchronizing timing between a transmitter and receiver, I'd like to lay down the hypotheses and reasoning I'm working from. Maybe others disagree on the basic assumptions or haven't though of them. So, here it goes:
Transmitter side:
Key assumption is that frames are actually transmitted according to dts. In a VBR transmission with immensely excessive bandwidth, or a pure CBR (e.g. uncompressed, single-stream) transmission, this can be done almost exactly. In CBR with buffer restraints, obviously it will not be exact.
It is sufficient that the mean difference between dts and actual transmission time be constant over intervals of time whose length is on the order of the maximal length of time in which transmitter and receiver clocks can be expected to remain within an acceptable error relative to one another.
This is too vague to argue about it. Also the constraint proposed by you will multiple the buffer and latency relative to mpeg by huge integers. That is it wont be a viable alternative unless pushed by lies claiming it wouldn have such disadvantages. Also this would only solve ONE single problem brought up. Your design still leaves the preload time up to pure luck. [...]
I'd be very happy to see (maybe even help write) an RFC on synchronization of unidirectional broadcast streams for arbitrary streamable containers. Is anyone willing to consider that as a constructive alternative to arguing about putting MPEG-type stuff in NUT?
You can write whatever you like. This has no effect on me wanting nut to be useable for broadcast. If you cannot offer a solution now then iam not interrested in it. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB If a bugfix only changes things apparently unrelated to the bug with no further explanation, that is a good sign that the bugfix is wrong.
On Wed, Feb 06, 2008 at 06:27:47PM +0100, Michael Niedermayer wrote:
On Wed, Feb 06, 2008 at 12:08:37PM -0500, Rich Felker wrote:
OK since there seems to be a disagreement (to put it lightly :) as to what the current NUT framework can do for synchronizing timing between a transmitter and receiver, I'd like to lay down the hypotheses and reasoning I'm working from. Maybe others disagree on the basic assumptions or haven't though of them. So, here it goes:
Transmitter side:
Key assumption is that frames are actually transmitted according to dts. In a VBR transmission with immensely excessive bandwidth, or a pure CBR (e.g. uncompressed, single-stream) transmission, this can be done almost exactly. In CBR with buffer restraints, obviously it will not be exact.
It is sufficient that the mean difference between dts and actual transmission time be constant over intervals of time whose length is on the order of the maximal length of time in which transmitter and receiver clocks can be expected to remain within an acceptable error relative to one another.
This is too vague to argue about it.
It's trivially true for any live broadcast or broadcast with a fixed channel bitrate.
Also the constraint proposed by you will multiple the buffer and latency relative to mpeg by huge integers. That is it wont be a
If you claim added latency you'll have to explain how. Since the constraints are ALREADY met in all real-world applications, claiming that they would add anything is nonsense IMNSHO.
Also this would only solve ONE single problem brought up. Your design still leaves the preload time up to pure luck.
If you really need to know a minimum preload time, put it in a protocol header or in the spec for the broadcast constraints.
I'd be very happy to see (maybe even help write) an RFC on synchronization of unidirectional broadcast streams for arbitrary streamable containers. Is anyone willing to consider that as a constructive alternative to arguing about putting MPEG-type stuff in NUT?
You can write whatever you like. This has no effect on me wanting nut to be useable for broadcast. If you cannot offer a solution now then iam not interrested in it.
It's already usable, as is any other existing streamable container. If you continue to point out the ways in which you do not believe they are usable, I will happily explain. Rich
On Wed, Feb 06, 2008 at 12:40:53PM -0500, Rich Felker wrote:
Also this would only solve ONE single problem brought up. Your design still leaves the preload time up to pure luck.
If you really need to know a minimum preload time, put it in a protocol header or in the spec for the broadcast constraints.
An info packet field for bitrate information, including minimum preload, would be totally acceptable too if you like. Rich
On Wed, Feb 06, 2008 at 12:43:21PM -0500, Rich Felker wrote:
On Wed, Feb 06, 2008 at 12:40:53PM -0500, Rich Felker wrote:
Also this would only solve ONE single problem brought up. Your design still leaves the preload time up to pure luck.
If you really need to know a minimum preload time, put it in a protocol header or in the spec for the broadcast constraints.
An info packet field for bitrate information, including minimum preload, would be totally acceptable too if you like.
The preload changes depending on buffer fullness thus is not constant over a stream. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Dictatorship naturally arises out of democracy, and the most aggravated form of tyranny and slavery out of the most extreme liberty. -- Plato
On Wed, Feb 06, 2008 at 07:22:14PM +0100, Michael Niedermayer wrote:
On Wed, Feb 06, 2008 at 12:43:21PM -0500, Rich Felker wrote:
On Wed, Feb 06, 2008 at 12:40:53PM -0500, Rich Felker wrote:
Also this would only solve ONE single problem brought up. Your design still leaves the preload time up to pure luck.
If you really need to know a minimum preload time, put it in a protocol header or in the spec for the broadcast constraints.
An info packet field for bitrate information, including minimum preload, would be totally acceptable too if you like.
The preload changes depending on buffer fullness thus is not constant over a stream.
Is this your whole point? You could have said that a long time ago and saved us a lot of trouble. I think it's entirely unrealistic that someone concerned with low-latency broadcast would have non-constant preload requirements. Or, said differently, if the sender's requirement is that the preload be less than 0.1 second, it should not matter to them whether the receiver is able to get by with only 0.03 seconds of preload when seeking to certain points in the stream. Constraints like these are about worst-case. Years and years ago, when talking about AVI and its audio preload feature, I thought we agreed to ignore highly device-dependent buffering details in the interest of making a container that is clean and simple. If there's a need to know the amount of preload required for streaming, a single global value in an info packet should be sufficient. IIRC info is still allowed to change mid-stream too (or was that removed?) -- if so one could hypothetically even intersperse variable preload recommendations but I really doubt the usefulness of doing so. Rich
On Wed, Feb 06, 2008 at 01:36:56PM -0500, Rich Felker wrote:
On Wed, Feb 06, 2008 at 07:22:14PM +0100, Michael Niedermayer wrote:
On Wed, Feb 06, 2008 at 12:43:21PM -0500, Rich Felker wrote:
On Wed, Feb 06, 2008 at 12:40:53PM -0500, Rich Felker wrote:
Also this would only solve ONE single problem brought up. Your design still leaves the preload time up to pure luck.
If you really need to know a minimum preload time, put it in a protocol header or in the spec for the broadcast constraints.
An info packet field for bitrate information, including minimum preload, would be totally acceptable too if you like.
The preload changes depending on buffer fullness thus is not constant over a stream.
Is this your whole point? You could have said that a long time ago and saved us a lot of trouble. I think it's entirely unrealistic that someone concerned with low-latency broadcast would have non-constant preload requirements.
Constant preload is pretty much the same as saying constant sized frames. That is much stricter than CBR. You want to limit nut to that? For this i guess the current design is sufficient.
Or, said differently, if the sender's requirement is that the preload be less than 0.1 second, it should not matter to them whether the receiver is able to get by with only 0.03 seconds of preload when seeking to certain points in the stream. Constraints like these are about worst-case.
well, if the decoder preloads X seconds while needing only 0 seconds then it will need X "seconds" more buffer space. -> this effectively doubles the needed buffer space.
Years and years ago, when talking about AVI and its audio preload feature,
avi audio preload is stupid.
I thought we agreed to ignore highly device-dependent buffering details in the interest of making a container that is clean and simple. If there's a need to know the amount of preload required for streaming, a single global value in an info packet should be sufficient.
the transmit_ts are a better choice, also if you seriously want device indpendance. It might be possible to specify the minimum needed preload vs bandwidth curve somehow on each syncpoint. This would not add any dependance on the actually used bitrate or buffers. But allow a demuxer (which approximately knows the bandwidth) to choose the ideal preload, or even download the whole file if the bandwidth is too low. It also might be possible to synchronize the clocks based on such preload/bandwidth data.
IIRC info is still allowed to change mid-stream too (or was that removed?)
Removed after your flames IIRC. Its also a point which will cause problems as soon as anyone wants to use info for anything realtime.
-- if so one could hypothetically even intersperse variable preload recommendations but I really doubt the usefulness of doing so.
I guess the spec contains a few rules by now forbidding putting info anywhere except after the stream headers (which at least contain the huge vorbis headers). I stoped counting, the how manyth case is this where your strict info design causes problems? [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Its not that you shouldnt use gotos but rather that you should write readable code and code with gotos often but not always is less readable
[I wrote this earlier but had to leave for work so I didn't finish sending. Hope there's nothing newer that renders parts of the message redundant already.] On Wed, Feb 06, 2008 at 08:23:08PM +0100, Michael Niedermayer wrote:
The preload changes depending on buffer fullness thus is not constant over a stream.
Is this your whole point? You could have said that a long time ago and saved us a lot of trouble. I think it's entirely unrealistic that someone concerned with low-latency broadcast would have non-constant preload requirements.
Constant preload is pretty much the same as saying constant sized frames. That is much stricter than CBR. You want to limit nut to that? For this i guess the current design is sufficient.
There are two perspectives on preload. One is asking the question: "For this data, what is the minimum amount I can get away with preloading?" This is a very concrete property of the data to be broadcast, and like you say, it varies. The other perspective is a requirements perspective, e.g. "I want receivers to be able to tune in to my stream and watch it with a maximum preload delay of 0.2 sec." In this case it does not matter whether the minimum preload required (in the former sense) fluctuates between 0.001 sec and 0.2 sec. All that matters is that the maximum minimum preload required is bounded by 0.2.
Or, said differently, if the sender's requirement is that the preload be less than 0.1 second, it should not matter to them whether the receiver is able to get by with only 0.03 seconds of preload when seeking to certain points in the stream. Constraints like these are about worst-case.
well, if the decoder preloads X seconds while needing only 0 seconds then it will need X "seconds" more buffer space. -> this effectively doubles the needed buffer space.
If instead of "how many seconds to preload" you knew some different buffer information, I think the problem would go away. I need to think about it more though and I don't want to give the impression of giving hasty answers since I've been accused of that already. :(
Years and years ago, when talking about AVI and its audio preload feature,
avi audio preload is stupid.
Something we still agree on at least..
-- if so one could hypothetically even intersperse variable preload recommendations but I really doubt the usefulness of doing so.
I guess the spec contains a few rules by now forbidding putting info anywhere except after the stream headers (which at least contain the huge vorbis headers).
Like I just said, repeating the headers frequently is already required if you're doing broadcast... Maybe this makes NUT sufficiently unappealing for broadcast that the whole discussion can be dropped, but for better or worse, I doubt that will happen. Rich
On Wed, Feb 06, 2008 at 08:50:41PM -0500, Rich Felker wrote:
[I wrote this earlier but had to leave for work so I didn't finish sending. Hope there's nothing newer that renders parts of the message redundant already.]
On Wed, Feb 06, 2008 at 08:23:08PM +0100, Michael Niedermayer wrote:
The preload changes depending on buffer fullness thus is not constant over a stream.
Is this your whole point? You could have said that a long time ago and saved us a lot of trouble. I think it's entirely unrealistic that someone concerned with low-latency broadcast would have non-constant preload requirements.
Constant preload is pretty much the same as saying constant sized frames. That is much stricter than CBR. You want to limit nut to that? For this i guess the current design is sufficient.
There are two perspectives on preload. One is asking the question: "For this data, what is the minimum amount I can get away with preloading?" This is a very concrete property of the data to be broadcast, and like you say, it varies.
The other perspective is a requirements perspective, e.g. "I want receivers to be able to tune in to my stream and watch it with a maximum preload delay of 0.2 sec." In this case it does not matter whether the minimum preload required (in the former sense) fluctuates between 0.001 sec and 0.2 sec. All that matters is that the maximum minimum preload required is bounded by 0.2.
This preload limit is also the limit on the buffer. Because at no time can the buffer be required to contain more,. Seeking to that point after all would otherwise require that larger amount to be preloaded. There goes your "end of the tiny mpeg buffers" as in 0.2 seconds you cant fill a buffer bigger than the mpeg buffers.
Or, said differently, if the sender's requirement is that the preload be less than 0.1 second, it should not matter to them whether the receiver is able to get by with only 0.03 seconds of preload when seeking to certain points in the stream. Constraints like these are about worst-case.
well, if the decoder preloads X seconds while needing only 0 seconds then it will need X "seconds" more buffer space. -> this effectively doubles the needed buffer space.
If instead of "how many seconds to preload" you knew some different buffer information, I think the problem would go away. I need to think
Yes, preload and transmit_time are interchangeable i think. You can find one from the other ... [...]
-- if so one could hypothetically even intersperse variable preload recommendations but I really doubt the usefulness of doing so.
I guess the spec contains a few rules by now forbidding putting info anywhere except after the stream headers (which at least contain the huge vorbis headers).
Like I just said, repeating the headers frequently is already required if you're doing broadcast... Maybe this makes NUT sufficiently unappealing for broadcast that the whole discussion can be dropped, but for better or worse, I doubt that will happen.
No no, mpeg absolutely requires all the headers to be repeated all the time. And mpeg has much more bloated headers. Low overhead is not a strength of mpeg ps/ts. Nut does have a huge advantage here, but of course only if it otherwise can handle broadcast. [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB I wish the Xiph folks would stop pretending they've got something they do not. Somehow I fear this will remain a wish. -- Måns Rullgård
Rich Felker <dalias@aerifal.cx> writes:
On Wed, Feb 06, 2008 at 06:27:47PM +0100, Michael Niedermayer wrote:
On Wed, Feb 06, 2008 at 12:08:37PM -0500, Rich Felker wrote:
OK since there seems to be a disagreement (to put it lightly :) as to what the current NUT framework can do for synchronizing timing between a transmitter and receiver, I'd like to lay down the hypotheses and reasoning I'm working from. Maybe others disagree on the basic assumptions or haven't though of them. So, here it goes:
Transmitter side:
Key assumption is that frames are actually transmitted according to dts. In a VBR transmission with immensely excessive bandwidth, or a pure CBR (e.g. uncompressed, single-stream) transmission, this can be done almost exactly. In CBR with buffer restraints, obviously it will not be exact.
It is sufficient that the mean difference between dts and actual transmission time be constant over intervals of time whose length is on the order of the maximal length of time in which transmitter and receiver clocks can be expected to remain within an acceptable error relative to one another.
This is too vague to argue about it.
It's trivially true for any live broadcast or broadcast with a fixed channel bitrate.
Nobody uses fixed channel bitrate. A multiplex has a fixed bitrate dictated by the modulation. This fixed bitrate is then shared by 10-15 independent channels. It is even common to over-allocate the bitrate somewhat, since it is highly unlikely that all channels will need peak rates at the same time. -- Måns Rullgård mans@mansr.com
On Wed, Feb 06, 2008 at 06:57:51PM +0000, Måns Rullgård wrote:
It is sufficient that the mean difference between dts and actual transmission time be constant over intervals of time whose length is on the order of the maximal length of time in which transmitter and receiver clocks can be expected to remain within an acceptable error relative to one another.
This is too vague to argue about it.
It's trivially true for any live broadcast or broadcast with a fixed channel bitrate.
Nobody uses fixed channel bitrate.
I'm looking at applications broader than just TV. Also note that I did not say the constraint is not met for programs without fixed channel bitrate, just that it's _trivially_ met when channel bitrate is fixed, i.e. no argument or analysis is needed to see that it's true. If you want to begin examining more real world scenarios we can do that. I'd be happy if you could analyze my claims for real-world data you work with. :)
A multiplex has a fixed bitrate dictated by the modulation. This fixed bitrate is then shared by 10-15 independent channels. It is even common to over-allocate the bitrate somewhat, since it is highly unlikely that all channels will need peak rates at the same time.
And now the TV stations have quality of service just as low as airlines... So much for all the claims of superior quality from digital TV... Rich
participants (3)
-
Michael Niedermayer -
Måns Rullgård -
Rich Felker