February 2008 - NUT-devel - lists.mplayerhq.hu

[nut]: r591 - src/trunk/nututils/nutparse.c
by cladisch 07 Feb '08

07 Feb '08

Author: cladisch Date: Thu Feb 7 09:03:36 2008 New Revision: 591 Log: add FLAG_MATCH_TIME support Modified: src/trunk/nututils/nutparse.c Modified: src/trunk/nututils/nutparse.c ============================================================================== --- src/trunk/nututils/nutparse.c (original) +++ src/trunk/nututils/nutparse.c Thu Feb 7 09:03:36 2008 @@ -27,6 +27,7 @@ #define FLAG_SIZE_MSB 0x0020 #define FLAG_CHECKSUM 0x0040 #define FLAG_RESERVED 0x0080 +#define FLAG_MATCH_TIME 0x0800 #define FLAG_CODED 0x1000 #define FLAG_INVALID 0x2000 @@ -182,6 +183,11 @@ static int64_t read_svar(void) return convert_to_signed(read_var()); } +static int64_t read_svar_restricted(void) +{ + return convert_to_signed(read_var_restricted()); +} + static uint64_t convert_ts(uint64_t t, int tb_from, int tb_to) { uint64_t ln = time_bases[tb_from].num * time_bases[tb_to].denom; @@ -214,6 +220,7 @@ static void parse_main_header(void) int64_t tmp_pts; uint64_t tmp_mul; uint64_t tmp_stream; + int64_t tmp_match; uint64_t tmp_size; uint64_t tmp_res; uint64_t count; @@ -264,6 +271,7 @@ static void parse_main_header(void) tmp_pts = 0; tmp_mul = 1; tmp_stream = 0; + tmp_match = 1 - (INT64_C(1) << 62); for (i = 0; i < 256; ) { tmp_flag = read_var(); printf(" tmp_flag: %"PRIu64"\n", tmp_flag); @@ -313,7 +321,11 @@ static void parse_main_header(void) if (!count) error("count is zero"); } - for (j64 = 6; j64 < tmp_fields; ++j64) + if (tmp_fields > 6) { + tmp_match = read_svar(); + printf(" tmp_match: %"PRId64"\n", tmp_match); + } + for (j64 = 7; j64 < tmp_fields; ++j64) printf(" tmp_reserved[%"PRIu64"]: %"PRIu64"\n", j64, read_var()); for (j = 0; j < count && i < 256; ++j, ++i) { if (i == 'N') { @@ -337,6 +349,8 @@ static void parse_main_header(void) printf(" checksum"); if (tmp_flag & FLAG_RESERVED) printf(" reserved"); + if (tmp_flag & FLAG_MATCH_TIME) + printf(" match_time"); if (tmp_flag & FLAG_CODED) printf(" flag_coded"); if (tmp_flag & FLAG_INVALID) @@ -706,6 +720,7 @@ static void parse_frame(int frame_type) uint64_t coded_pts = 0; int64_t pts_delta; uint64_t data_size_msb; + int64_t match_time_delta; uint64_t reserved_count; uint64_t data_size; uint64_t old_last_pts; @@ -740,6 +755,10 @@ static void parse_frame(int frame_type) data_size_msb = read_var_restricted(); printf(" data_size_msb: %"PRIu64"\n", data_size_msb); } + if (frame_flags & FLAG_MATCH_TIME) { + match_time_delta = read_svar_restricted(); + printf(" match_time_delta: %"PRId64"\n", match_time_delta); + } if (frame_flags & FLAG_RESERVED) { reserved_count = read_var_restricted(); printf(" reserved_count: %"PRIu64"\n", reserved_count);

1 0

revisions for nut-english.txt?
by Rich Felker 07 Feb '08

07 Feb '08

A few things I came across reading it now: Keyframes - update definition from Michael? Representation of Time - "single contiguous interval of time" failes to mention EOR. It's mentioned later though. Should this be improved? Also it might be worth mentioning that multiple 'tracks' could be stored as chapters using info packets to label them, but if so they obviously need to be timed one after another rather than all starting at time=0. Rich

5 24

Broadcast sufficiency - my assumptions/reasoning/design for it
by Rich Felker 07 Feb '08

07 Feb '08

OK since there seems to be a disagreement (to put it lightly :) as to what the current NUT framework can do for synchronizing timing between a transmitter and receiver, I'd like to lay down the hypotheses and reasoning I'm working from. Maybe others disagree on the basic assumptions or haven't though of them. So, here it goes: Transmitter side: Key assumption is that frames are actually transmitted according to dts. In a VBR transmission with immensely excessive bandwidth, or a pure CBR (e.g. uncompressed, single-stream) transmission, this can be done almost exactly. In CBR with buffer restraints, obviously it will not be exact. It is sufficient that the mean difference between dts and actual transmission time be constant over intervals of time whose length is on the order of the maximal length of time in which transmitter and receiver clocks can be expected to remain within an acceptable error relative to one another. Ensuring that this mean difference requirement can be met is a matter of bitrate constraints on the codecs in use. Unless my casual estimates are horribly mistaken, it is much easier to meet these requirements than to meet most buffering requirements, i.e. any broadcast setup already meeting moderate to strict buffer size constraints will automatically meets the requirements for the transmitter to be able to ensure the mean difference requirement. Note that if content is created live and transmitted with minimal delay, the constraint will automatically be met. Only non-live transmissions need to make particular efforts to synchronize transmit times with dts, and even then if bitrate/buffer constraints are strict it may be trivial. Many years ago I wrote a non-live streaming server based on ogg container (eew!) which synchronized transmission of ogg pages to the system clock and it involved very little effort. Receiver side: Upon receiving each frame, the receiver stamps it with the actual time the frame was received, according to the local clock. Drift in the mean difference between receive-timestamps and dts is tracked and used to adjust the local clock as it diverges from the transmitter's clock. Note that there's nothing NUT-specific about this design. It will work with any existing container, and it can be used in place of the redundant timestamps in MPEG if one desires. In the interests of improving the media software/hardware landscape in general, it's much more worthwhile to have designs that work with any container, and to push developers to move towards supporting those rather than highly container-specific or application-specific ("application" in the sense of "broadcast tv" or "video on demand", not "firefox" or "ms word") implementations of synchronization. I'd be very happy to see (maybe even help write) an RFC on synchronization of unidirectional broadcast streams for arbitrary streamable containers. Is anyone willing to consider that as a constructive alternative to arguing about putting MPEG-type stuff in NUT? Rich

3 10

Some sanity...
by Rich Felker 06 Feb '08

06 Feb '08

I think we need a reality check here, as NUT is a FROZEN spec. I'm not sure if the term frozen was ever clearly defined, but roughly speaking it should mean that changes should be made only in cases where they do not change anything for the ordinary cases (common codecs in use, e.g. the whole keyframes thing) or where a serious bug/mistake/limitation is discovered during testing. This whole broadcast issue has brought up a lot of "new requirements" with no legitimate argument for how they fulfill any need that NUT does not already meet. "Because MPEG does it" is NOT A REASON!! Moreover, folks with MPEG broadcast experience (and who ACTUALLY LIKE MPEG) are not qualified to make recommendations about what's needed! NUT's goal was never to copy MPEG but to redo things and do them correctly from the ground up. "People in the industry do it this way" is not an argument. As far as I can tell, everything done with MPEG broadcast can be done much simpler, for instance the time synchronization issue. Due to NUT's extremely strict interleaving rules, frame dts ALREADY serves as a synchronization timestamp!! There's no need for separate timestamps! Please stop the insanity! NUT IS FINISHED. If anyone claims not, please show real, demonstratable bugs, not differences from MPEG. A difference from MPEG is not a bug but a sign of good design. Rich

4 9

[nut]: r590 - docs/nut.txt
by michael 06 Feb '08

06 Feb '08

Author: michael Date: Wed Feb 6 01:10:33 2008 New Revision: 590 Log: better wording from mans Modified: docs/nut.txt Modified: docs/nut.txt ============================================================================== --- docs/nut.txt (original) +++ docs/nut.txt Wed Feb 6 01:10:33 2008 @@ -684,7 +684,7 @@ match_time_delta of previous frames (that is the frames are virtually identical no matter if decoding started from the very first frame or from this keyframe). If its value is 1-(1<<62) then match_time_delta is unspecified, that is - the muxer did not have enough knowledge to set it. + the muxer lacked sufficient information to set it. A muxer MUST only set it to 1-(1<<62) if it does not know the correct value. That is, it is not allowed to randomly discard known values.

1 0

[nut]: r588 - docs/nut.txt
by michael 05 Feb '08

05 Feb '08

Author: michael Date: Tue Feb 5 15:54:07 2008 New Revision: 588 Log: Add a unknown match_time_delta value and use this as default. That way "old" nut files will have unspecified match_time_delta instead of all 0. And new muxers arent forced to know all match_time_delta. Comments welcome. An alternative would be to require specified match_time_delta but that could be tricky to find out in some cases, and no other container contains this info. Modified: docs/nut.txt Modified: docs/nut.txt ============================================================================== --- docs/nut.txt (original) +++ docs/nut.txt Tue Feb 5 15:54:07 2008 @@ -247,7 +247,7 @@ main_header: tmp_pts=0 tmp_mul=1 tmp_stream=0 - tmp_match=0 + tmp_match=1-(1<<62) for(i=0; i<256; ){ tmp_flag v tmp_fields v @@ -683,6 +683,10 @@ match_time_delta the output from the decoder has converged independent from the availability of previous frames (that is the frames are virtually identical no matter if decoding started from the very first frame or from this keyframe). + If its value is 1-(1<<62) then match_time_delta is unspecified, that is + the muxer didnt have enough knowledge to set it. + A muxer MUST only set it to 1-(1<<62) if it does not know the correct + value. That is, it is not allowed to randomly discard known values. lsb_pts Least significant bits of the pts in time_base precision.

3 2

[nut]: r589 - docs/nut.txt
by diego 05 Feb '08

05 Feb '08

Author: diego Date: Wed Feb 6 00:32:34 2008 New Revision: 589 Log: typo fixes Modified: docs/nut.txt Modified: docs/nut.txt ============================================================================== --- docs/nut.txt (original) +++ docs/nut.txt Wed Feb 6 00:32:34 2008 @@ -497,7 +497,7 @@ stream_class (v) 0 video 1 audio 2 subtitles - 3 userdata + 3 user data Note: The remaining values are reserved and MUST NOT be used. A demuxer MUST ignore streams with reserved classes. @@ -576,7 +576,7 @@ stream_flags (v) 1 FLAG_FIXED_FPS indicates that the fps is fixed codec_specific_data (vb) - Private global data for a codec (could be huffman tables or ...). + Private global data for a codec (could be Huffman tables or ...). If a codec has a global header it SHOULD be placed in here instead of at the start of every keyframe. The exact format is specified in the codec specification. @@ -684,7 +684,7 @@ match_time_delta of previous frames (that is the frames are virtually identical no matter if decoding started from the very first frame or from this keyframe). If its value is 1-(1<<62) then match_time_delta is unspecified, that is - the muxer didnt have enough knowledge to set it. + the muxer did not have enough knowledge to set it. A muxer MUST only set it to 1-(1<<62) if it does not know the correct value. That is, it is not allowed to randomly discard known values.

1 0

[nut]: r587 - docs/nut.txt
by michael 05 Feb '08

05 Feb '08

Author: michael Date: Tue Feb 5 15:30:26 2008 New Revision: 587 Log: Fixes to make match_time_delta storeable in the framecode. Modified: docs/nut.txt Modified: docs/nut.txt ============================================================================== --- docs/nut.txt (original) +++ docs/nut.txt Tue Feb 5 15:30:26 2008 @@ -619,7 +619,7 @@ flags[frame_code], frame_flags (v) 6 FLAG_CHECKSUM If set, the frame header contains a checksum. 7 FLAG_RESERVED If set, reserved_count is coded in the frame header. 11 FLAG_MATCH_TIME If set, match_time_delta is coded in the frame - header, otherwise its 0. + header 12 FLAG_CODED If set, coded_flags are stored in the frame header. 13 FLAG_INVALID If set, frame_code is invalid. @@ -679,7 +679,6 @@ coded_pts (v) pts = ( (pts_lsb - delta) & mask ) + delta match_time_delta - If FLAG_MATCH_TIME is not set, then this is 0 This is the time difference in stream timebase units from the pts at which the output from the decoder has converged independent from the availability of previous frames (that is the frames are virtually identical no matter

1 0

[nut]: r586 - docs/nut.txt
by michael 05 Feb '08

05 Feb '08

Author: michael Date: Tue Feb 5 15:14:01 2008 New Revision: 586 Log: Add match_time_delta to the framecode table. Modified: docs/nut.txt Modified: docs/nut.txt ============================================================================== --- docs/nut.txt (original) +++ docs/nut.txt Tue Feb 5 15:14:01 2008 @@ -247,6 +247,7 @@ main_header: tmp_pts=0 tmp_mul=1 tmp_stream=0 + tmp_match=0 for(i=0; i<256; ){ tmp_flag v tmp_fields v @@ -259,7 +260,8 @@ main_header: else tmp_res=0 if(tmp_fields>5) count v else count= tmp_mul - tmp_size - for(j=6; j<tmp_fields; j++){ + if(tmp_fields>6) tmp_match s + for(j=7; j<tmp_fields; j++){ tmp_reserved[i] v } for(j=0; j<count && i<256; j++, i++){ @@ -274,6 +276,7 @@ main_header: data_size_lsb[i]= tmp_size + j; pts_delta[i]= tmp_pts; reserved_count[i]= tmp_res; + match_time_delta[i]= tmp_match; } } reserved_bytes

2 1

[nut]: r585 - docs/nut.txt
by diego 05 Feb '08

05 Feb '08

Author: diego Date: Tue Feb 5 14:27:54 2008 New Revision: 585 Log: spelling/wording Modified: docs/nut.txt Modified: docs/nut.txt ============================================================================== --- docs/nut.txt (original) +++ docs/nut.txt Tue Feb 5 14:27:54 2008 @@ -678,7 +678,7 @@ coded_pts (v) match_time_delta If FLAG_MATCH_TIME is not set, then this is 0 This is the time difference in stream timebase units from the pts at which - the output from the decoder has converged independant of the availability + the output from the decoder has converged independent from the availability of previous frames (that is the frames are virtually identical no matter if decoding started from the very first frame or from this keyframe). @@ -888,15 +888,15 @@ info packet types of decoding it. "Depends" v (stream id) - This stream depends on the specified stream. That is if its - unavailable then this stream cannot be decoded. This is used to store - dependancies for scalability for example. + This stream depends on the specified stream. That is if the specified + stream is unavailable then this stream cannot be decoded. This is used + to store dependencies for scalability for example. "Uses" v (stream id) - This stream can use the specified stream. But its existence is not - mandatory for decoding this stream. Although if its unavailable - quality might suffer. Examples are font attachments or higher quality - samples from music instruments. + This stream can use the specified stream. But the existence of the + specified stream is not mandatory for decoding this stream, although + quality might suffer if it is unavailable. Examples: font attachments + or higher quality samples from musical instruments "UsesFont" "Times New Roman", "Arial", ... FIXME should we use the filename instead? is there some other unique way to identify fonts? @@ -942,12 +942,12 @@ in a file). There MUST be a syncpoint immediately before the first frame after any headers. -The main header is mandatory, so are all stream headers except thouse for -redundant and enhanchement streams. So for example if the file contains 2 -independant video streams encoding the same content at 2 different bitrates. +The main header is mandatory, so are all stream headers except those for +redundant and enhancement streams. So for example if the file contains 2 +independent video streams encoding the same content at 2 different bitrates. Then only one is mandatory, which that is, is at the discretion of the muxer. -Enhancement streams like font attachments are also non mandatory. -The muxer SHOULD repeat non mandatory headers like mandatory ones if +Enhancement streams like font attachments are also non-mandatory. +The muxer SHOULD repeat non-mandatory headers like mandatory ones if the overhead is not excessive.

1 0