- NUT-devel - lists.mplayerhq.hu

Broadcasting nuts [PATCH]
by Michael Niedermayer 09 Feb '08

09 Feb '08

Hi Attached patch should make broadcast of single program nuts possible. Its the minimal/least restrictive solution i found. Comments welcome, especially from mans/nico, others who know mpeg-ps/ts. -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Complexity theory is the science of finding the exact solution to an approximation. Benchmarking OTOH is finding an approximation of the exact

6 54

[nut]: r594 - docs/nutissues.txt
by michael 09 Feb '08

09 Feb '08

Author: michael Date: Sat Feb 9 18:12:07 2008 New Revision: 594 Log: more solutions Modified: docs/nutissues.txt Modified: docs/nutissues.txt ============================================================================== --- docs/nutissues.txt (original) +++ docs/nutissues.txt Sat Feb 9 18:12:07 2008 @@ -65,6 +65,8 @@ not be stored cleanly in nut. Solutions: A. one backptr per program + any solution for issue info-stream-subsets +B. Design a seperate layer for it +C. Dont support this Issue info-overhead @@ -72,6 +74,12 @@ Issue info-overhead Info can be stored much more compactly by replacing common strings by shorter represenstations. +A. Change syntax so a v value which indexes into a fixed table can select + info names. (breaks compatibility completely) +B. Use/allow 1 char abbreviations (new demuxers could read old files, old + ones could read everything except the abbreviated fields) +C. Leave info as it is. + Issue edit-lists ----------------

1 0

[nut]: r593 - docs/nutissues.txt nutissues.txt
by michael 09 Feb '08

09 Feb '08

Author: michael Date: Sat Feb 9 16:36:05 2008 New Revision: 593 Log: Move to proper directory ... Added: docs/nutissues.txt - copied unchanged from r592, /nutissues.txt Removed: nutissues.txt

1 0

[nut]: r592 - nutissues.txt
by michael 09 Feb '08

09 Feb '08

Author: michael Date: Sat Feb 9 16:34:54 2008 New Revision: 592 Log: List of known issues, so we can keep track of what might need work and what can be done to solve issues. And what the advantages / disadvantages the solutions have. Added: nutissues.txt Added: nutissues.txt ============================================================================== --- (empty file) +++ nutissues.txt Sat Feb 9 16:34:54 2008 @@ -0,0 +1,97 @@ +Issue broadcast-clock-sync +---------------- +Problem: Clock synchronization leading to buffer overflow and underflow +in the decoder. + + +Issue broadcast-preload-len +---------------- +Problem: How long should the receiver preload the nut file before starting +playback and after seeking, so no buffer over/underflows happens. + +Solutions to the above 2 issues are for example storing one of the following +A. preload time +B. preload bytes +C. buffer fullness in seconds +D. buffer fullness in bytes +E. transmit time stamps + +Values in bytes scale nicer with increased bandwidth, that is doubling the +bandwidth from the one assumed by the muxer halfs the needed preload. time +based sync values dont naturally change the needed preload with increased +bandwidth but instead increase the needed buffer size. + +These can be stored in one of the following +a. sync points +b. info packets +c. a new stream type + +Info packets are currently not allowed to change thus they are not suitable +without that being changed. + + +Issue chapter-overlap/gap +------------------------- +If one changes the timebase between chapters then overlaps (forbidden by the +spec) or gaps (problematic with split&merge) can happen. + +Solutions +A dont change the timebase +B store start and end in seperate timebases (breaks compatibility) +C add a field into the info packet overriding the chapter_end with a + more accurate value. +D use the chapter start of the next chapter (breaks compatibility and + error robustness) + + +Issue stream-chapter-overlap +---------------------------- +It is possible especially in the light of the precission of the previous +issue, that chapters might not end exactly at the same points in time in +all streams. There could be a few samples difference. +Do we care about this? + + +Issue info-stream-subsets +------------------------- +Currently if info applies to more than 1 stream but not all, it has to be +duplicated for each stream it applies to. + + +Issue multiple-programs +----------------------- +Multiple programs (that is several audio-video stream pairs for example) can +not be stored cleanly in nut. + +Solutions: +A. one backptr per program + any solution for issue info-stream-subsets + + +Issue info-overhead +------------------- +Info can be stored much more compactly by replacing common strings by +shorter represenstations. + + +Issue edit-lists +---------------- +For editing its very usefull to not rewrite the whole file after every step. + +Solutions. +A. Store such edits in info packets +B. Store such edits in info packets but allow them only in "private" nut + files. That is files distributed must be remuxed. Players must reject + files with edit lists. +C. Dont support this (this would mean no interoperability between video + editing programs using nut as they would use their own formats) + + +Issue alternatives +------------------ +Alternatives like a variant without some scenes or with a happy ending +instead of a tragic one cannot be stored in nut currently. + +Solutions +A. Store such alterative play lists of scenes in info packets somehow +B. Design a seperate layer for it +C. Dont support this

1 0

[nut]: r591 - src/trunk/nututils/nutparse.c
by cladisch 07 Feb '08

07 Feb '08

Author: cladisch Date: Thu Feb 7 09:03:36 2008 New Revision: 591 Log: add FLAG_MATCH_TIME support Modified: src/trunk/nututils/nutparse.c Modified: src/trunk/nututils/nutparse.c ============================================================================== --- src/trunk/nututils/nutparse.c (original) +++ src/trunk/nututils/nutparse.c Thu Feb 7 09:03:36 2008 @@ -27,6 +27,7 @@ #define FLAG_SIZE_MSB 0x0020 #define FLAG_CHECKSUM 0x0040 #define FLAG_RESERVED 0x0080 +#define FLAG_MATCH_TIME 0x0800 #define FLAG_CODED 0x1000 #define FLAG_INVALID 0x2000 @@ -182,6 +183,11 @@ static int64_t read_svar(void) return convert_to_signed(read_var()); } +static int64_t read_svar_restricted(void) +{ + return convert_to_signed(read_var_restricted()); +} + static uint64_t convert_ts(uint64_t t, int tb_from, int tb_to) { uint64_t ln = time_bases[tb_from].num * time_bases[tb_to].denom; @@ -214,6 +220,7 @@ static void parse_main_header(void) int64_t tmp_pts; uint64_t tmp_mul; uint64_t tmp_stream; + int64_t tmp_match; uint64_t tmp_size; uint64_t tmp_res; uint64_t count; @@ -264,6 +271,7 @@ static void parse_main_header(void) tmp_pts = 0; tmp_mul = 1; tmp_stream = 0; + tmp_match = 1 - (INT64_C(1) << 62); for (i = 0; i < 256; ) { tmp_flag = read_var(); printf(" tmp_flag: %"PRIu64"\n", tmp_flag); @@ -313,7 +321,11 @@ static void parse_main_header(void) if (!count) error("count is zero"); } - for (j64 = 6; j64 < tmp_fields; ++j64) + if (tmp_fields > 6) { + tmp_match = read_svar(); + printf(" tmp_match: %"PRId64"\n", tmp_match); + } + for (j64 = 7; j64 < tmp_fields; ++j64) printf(" tmp_reserved[%"PRIu64"]: %"PRIu64"\n", j64, read_var()); for (j = 0; j < count && i < 256; ++j, ++i) { if (i == 'N') { @@ -337,6 +349,8 @@ static void parse_main_header(void) printf(" checksum"); if (tmp_flag & FLAG_RESERVED) printf(" reserved"); + if (tmp_flag & FLAG_MATCH_TIME) + printf(" match_time"); if (tmp_flag & FLAG_CODED) printf(" flag_coded"); if (tmp_flag & FLAG_INVALID) @@ -706,6 +720,7 @@ static void parse_frame(int frame_type) uint64_t coded_pts = 0; int64_t pts_delta; uint64_t data_size_msb; + int64_t match_time_delta; uint64_t reserved_count; uint64_t data_size; uint64_t old_last_pts; @@ -740,6 +755,10 @@ static void parse_frame(int frame_type) data_size_msb = read_var_restricted(); printf(" data_size_msb: %"PRIu64"\n", data_size_msb); } + if (frame_flags & FLAG_MATCH_TIME) { + match_time_delta = read_svar_restricted(); + printf(" match_time_delta: %"PRId64"\n", match_time_delta); + } if (frame_flags & FLAG_RESERVED) { reserved_count = read_var_restricted(); printf(" reserved_count: %"PRIu64"\n", reserved_count);

1 0

revisions for nut-english.txt?
by Rich Felker 07 Feb '08

07 Feb '08

A few things I came across reading it now: Keyframes - update definition from Michael? Representation of Time - "single contiguous interval of time" failes to mention EOR. It's mentioned later though. Should this be improved? Also it might be worth mentioning that multiple 'tracks' could be stored as chapters using info packets to label them, but if so they obviously need to be timed one after another rather than all starting at time=0. Rich

5 24

Broadcast sufficiency - my assumptions/reasoning/design for it
by Rich Felker 07 Feb '08

07 Feb '08

OK since there seems to be a disagreement (to put it lightly :) as to what the current NUT framework can do for synchronizing timing between a transmitter and receiver, I'd like to lay down the hypotheses and reasoning I'm working from. Maybe others disagree on the basic assumptions or haven't though of them. So, here it goes: Transmitter side: Key assumption is that frames are actually transmitted according to dts. In a VBR transmission with immensely excessive bandwidth, or a pure CBR (e.g. uncompressed, single-stream) transmission, this can be done almost exactly. In CBR with buffer restraints, obviously it will not be exact. It is sufficient that the mean difference between dts and actual transmission time be constant over intervals of time whose length is on the order of the maximal length of time in which transmitter and receiver clocks can be expected to remain within an acceptable error relative to one another. Ensuring that this mean difference requirement can be met is a matter of bitrate constraints on the codecs in use. Unless my casual estimates are horribly mistaken, it is much easier to meet these requirements than to meet most buffering requirements, i.e. any broadcast setup already meeting moderate to strict buffer size constraints will automatically meets the requirements for the transmitter to be able to ensure the mean difference requirement. Note that if content is created live and transmitted with minimal delay, the constraint will automatically be met. Only non-live transmissions need to make particular efforts to synchronize transmit times with dts, and even then if bitrate/buffer constraints are strict it may be trivial. Many years ago I wrote a non-live streaming server based on ogg container (eew!) which synchronized transmission of ogg pages to the system clock and it involved very little effort. Receiver side: Upon receiving each frame, the receiver stamps it with the actual time the frame was received, according to the local clock. Drift in the mean difference between receive-timestamps and dts is tracked and used to adjust the local clock as it diverges from the transmitter's clock. Note that there's nothing NUT-specific about this design. It will work with any existing container, and it can be used in place of the redundant timestamps in MPEG if one desires. In the interests of improving the media software/hardware landscape in general, it's much more worthwhile to have designs that work with any container, and to push developers to move towards supporting those rather than highly container-specific or application-specific ("application" in the sense of "broadcast tv" or "video on demand", not "firefox" or "ms word") implementations of synchronization. I'd be very happy to see (maybe even help write) an RFC on synchronization of unidirectional broadcast streams for arbitrary streamable containers. Is anyone willing to consider that as a constructive alternative to arguing about putting MPEG-type stuff in NUT? Rich

3 10

Some sanity...
by Rich Felker 06 Feb '08

06 Feb '08

I think we need a reality check here, as NUT is a FROZEN spec. I'm not sure if the term frozen was ever clearly defined, but roughly speaking it should mean that changes should be made only in cases where they do not change anything for the ordinary cases (common codecs in use, e.g. the whole keyframes thing) or where a serious bug/mistake/limitation is discovered during testing. This whole broadcast issue has brought up a lot of "new requirements" with no legitimate argument for how they fulfill any need that NUT does not already meet. "Because MPEG does it" is NOT A REASON!! Moreover, folks with MPEG broadcast experience (and who ACTUALLY LIKE MPEG) are not qualified to make recommendations about what's needed! NUT's goal was never to copy MPEG but to redo things and do them correctly from the ground up. "People in the industry do it this way" is not an argument. As far as I can tell, everything done with MPEG broadcast can be done much simpler, for instance the time synchronization issue. Due to NUT's extremely strict interleaving rules, frame dts ALREADY serves as a synchronization timestamp!! There's no need for separate timestamps! Please stop the insanity! NUT IS FINISHED. If anyone claims not, please show real, demonstratable bugs, not differences from MPEG. A difference from MPEG is not a bug but a sign of good design. Rich

4 9

[nut]: r590 - docs/nut.txt
by michael 06 Feb '08

06 Feb '08

Author: michael Date: Wed Feb 6 01:10:33 2008 New Revision: 590 Log: better wording from mans Modified: docs/nut.txt Modified: docs/nut.txt ============================================================================== --- docs/nut.txt (original) +++ docs/nut.txt Wed Feb 6 01:10:33 2008 @@ -684,7 +684,7 @@ match_time_delta of previous frames (that is the frames are virtually identical no matter if decoding started from the very first frame or from this keyframe). If its value is 1-(1<<62) then match_time_delta is unspecified, that is - the muxer did not have enough knowledge to set it. + the muxer lacked sufficient information to set it. A muxer MUST only set it to 1-(1<<62) if it does not know the correct value. That is, it is not allowed to randomly discard known values.

1 0

[nut]: r588 - docs/nut.txt
by michael 06 Feb '08

06 Feb '08

Author: michael Date: Tue Feb 5 15:54:07 2008 New Revision: 588 Log: Add a unknown match_time_delta value and use this as default. That way "old" nut files will have unspecified match_time_delta instead of all 0. And new muxers arent forced to know all match_time_delta. Comments welcome. An alternative would be to require specified match_time_delta but that could be tricky to find out in some cases, and no other container contains this info. Modified: docs/nut.txt Modified: docs/nut.txt ============================================================================== --- docs/nut.txt (original) +++ docs/nut.txt Tue Feb 5 15:54:07 2008 @@ -247,7 +247,7 @@ main_header: tmp_pts=0 tmp_mul=1 tmp_stream=0 - tmp_match=0 + tmp_match=1-(1<<62) for(i=0; i<256; ){ tmp_flag v tmp_fields v @@ -683,6 +683,10 @@ match_time_delta the output from the decoder has converged independent from the availability of previous frames (that is the frames are virtually identical no matter if decoding started from the very first frame or from this keyframe). + If its value is 1-(1<<62) then match_time_delta is unspecified, that is + the muxer didnt have enough knowledge to set it. + A muxer MUST only set it to 1-(1<<62) if it does not know the correct + value. That is, it is not allowed to randomly discard known values. lsb_pts Least significant bits of the pts in time_base precision.

3 2