- NUT-devel - lists.mplayerhq.hu

[nut]: r172 - trunk/docs/nut-english.txt
by ods15 28 Oct '06

28 Oct '06

Author: ods15 Date: Sat Oct 28 22:19:43 2006 New Revision: 172 Modified: trunk/docs/nut-english.txt Log: typos Modified: trunk/docs/nut-english.txt ============================================================================== --- trunk/docs/nut-english.txt (original) +++ trunk/docs/nut-english.txt Sat Oct 28 22:19:43 2006 @@ -46,7 +46,7 @@ Variable Length Coding Almost all fields in NUT are coded using VLCs. VLCs allow for compact -storage of small integers, which still being extendible to infinitely +storage of small integers, while still being extendible to infinitely large integers. The syntax is of a VLC is, per byte, a 1 bit flag stating if there are more bits to the integer, and 7 bits to be prepended to the lsb of the integer, shifting the previous value to the left. @@ -55,7 +55,7 @@ NUT packet. The only such fields are forward_ptr and fields in frame headers. This is to prevent demuxers from looping on a large of amount of damaged bytes of 0x80. Fields inside a NUT packet are protected by a CRC -whcih can be checked before decoding. +which can be checked before decoding. Header Structure

1 0

[nut]: r171 - trunk/nututils/nutindex.c
by ods15 28 Oct '06

28 Oct '06

Author: ods15 Date: Sat Oct 28 22:14:19 2006 New Revision: 171 Modified: trunk/nututils/nutindex.c Log: add a note so I don't forget this myself Modified: trunk/nututils/nutindex.c ============================================================================== --- trunk/nututils/nutindex.c (original) +++ trunk/nututils/nutindex.c Sat Oct 28 22:14:19 2006 @@ -332,6 +332,8 @@ fprintf(stderr, "bleh\n"); return 1; } + printf("Note! This program produces less error resiliant files by moving the main headers!\n"); + in = new_input_buffer(&iin, fin); out = new_output_buffer(&oout, fout); CHECK(find_copy_index(in, mem, &end));

1 0

[nut]: r170 - trunk/libnut/demuxer.c
by ods15 28 Oct '06

28 Oct '06

Author: ods15 Date: Sat Oct 28 22:03:14 2006 New Revision: 170 Modified: trunk/libnut/demuxer.c Log: add useful comment, remove outdated irrelavent line Modified: trunk/libnut/demuxer.c ============================================================================== --- trunk/libnut/demuxer.c (original) +++ trunk/libnut/demuxer.c Sat Oct 28 22:03:14 2006 @@ -1035,11 +1035,11 @@ off_t back_ptr = stopper->pos - stopper->back_ptr; for (i = 1; i < sl->len; i++) { if (sl->s[i].pos > back_ptr + 15) { + // no need to worry about this being exact or not - if we're using stopper, that means we don't have an index if (sl->s[i-1].seen_next) stopper_syncpoint = sl->s[i].pos; break; } } - if (back_ptr > (nut->seek_status>>1)) stopper = NULL; // bad stopper, it points to a different back_ptr FIXME so?.. if (stopper_syncpoint > bctello(nut->i)) stopper_syncpoint = 0; // don't premature }

1 0

[nut]: r169 - trunk/docs/nut-english.txt
by ods15 28 Oct '06

28 Oct '06

Author: ods15 Date: Sat Oct 28 21:27:00 2006 New Revision: 169 Modified: trunk/docs/nut-english.txt Log: I'm too lazy to explain seeking again :) Modified: trunk/docs/nut-english.txt ============================================================================== --- trunk/docs/nut-english.txt (original) +++ trunk/docs/nut-english.txt Sat Oct 28 21:27:00 2006 @@ -246,7 +246,14 @@ to reset the last_pts context of all streams, and to find the appropriate syncpoint when seeking. Demuxing can only begin at syncpoints for proper last_pts context across all streams, including after seeking. -back_ptr is explained in greater detail in the Seeking section. +A back_ptr points to a previous syncpoint in the file. The area between +the previous syncpoint and this one must contain a keyframe for every +stream, with a pts lower than or equal to the timestamp of this syncpoint. +This back_ptr is used for optimal seeking in files without an index. +For compression, the back_ptr is relative to this syncpoint, and is +divided by 16. The reason for this is that the minimum size for a +syncpoint is 16 bytes: +8 startcode + 1 forward_ptr + 1 timestamp + 1 back_ptr + 4 checksum + 1 frame_code Error Checking @@ -280,3 +287,8 @@ closest syncpoint startcode found, and resume demuxing from there. If possible, before the linear search, rewind to the last syncpoint seen, in case a syncpoint was already skipped due to demuxing damaged data. + +Seeking + +An in depth explanation of an optimal seeking algorithm can be found in +http://wiki.multimedia.cx/index.php?title=NUT

1 0

[nut]: r168 - trunk/docs/nut-english.txt
by ods15 28 Oct '06

28 Oct '06

Author: ods15 Date: Sat Oct 28 21:15:41 2006 New Revision: 168 Modified: trunk/docs/nut-english.txt Log: error recovery Modified: trunk/docs/nut-english.txt ============================================================================== --- trunk/docs/nut-english.txt (original) +++ trunk/docs/nut-english.txt Sat Oct 28 21:15:41 2006 @@ -272,3 +272,11 @@ buffer a large amount of data from a file because of damaged data. Also max_pts_distance prevents an overly large pts caused by damaged data to cause a player to get stuck. + +Error Recovery + +The recommended method for recovering from errors once damage has been +detected, is to linear search the file from current position to the +closest syncpoint startcode found, and resume demuxing from there. If +possible, before the linear search, rewind to the last syncpoint seen, in +case a syncpoint was already skipped due to demuxing damaged data.

1 0

[nut]: r167 - trunk/docs/nut-english.txt
by ods15 28 Oct '06

28 Oct '06

Author: ods15 Date: Sat Oct 28 21:08:46 2006 New Revision: 167 Modified: trunk/docs/nut-english.txt Log: add error checking Modified: trunk/docs/nut-english.txt ============================================================================== --- trunk/docs/nut-english.txt (original) +++ trunk/docs/nut-english.txt Sat Oct 28 21:08:46 2006 @@ -247,3 +247,28 @@ syncpoint when seeking. Demuxing can only begin at syncpoints for proper last_pts context across all streams, including after seeking. back_ptr is explained in greater detail in the Seeking section. + +Error Checking + +There are several ways to detect a damaged stream in NUT during demuxing: +1. Invalid framecode - If a framecode which has been marked as invalid in + the main header is found as the framecode in a frame header, then the + stream is damaged. For this reason, 0x00 and 0xFF are recommended to be + set as invalid frame codes in NUT. +2. Bad CRC on a NUT packet, packet_header, or frame header - fairly + obvious. +3. Decoded frame size causes a distance from the last syncpoint to be + bigger than max_distance, and frame does not follow a syncpoint. +4. Decoded frame size is bigger than max_distance*2, and frame header does + not have a CRC. +5. Decoded frame pts is more than max_pts_distance higher than last_pts, + and frame header does not have a CRC. +6. A VLC is found with more than 8 bytes of stuffing in a frame header or + forward_ptr. +7. Streams are found to not be strictly interleaved by comparing dts and + pts - a precise formula for this check can be found in the spec. + +All these conditions make it impossible for a demuxer to read, skip or +buffer a large amount of data from a file because of damaged data. Also +max_pts_distance prevents an overly large pts caused by damaged data to +cause a player to get stuck.

1 0

[nut]: r166 - trunk/docs/nut-english.txt
by ods15 28 Oct '06

28 Oct '06

Author: ods15 Date: Sat Oct 28 20:42:49 2006 New Revision: 166 Modified: trunk/docs/nut-english.txt Log: more on last_pts Modified: trunk/docs/nut-english.txt ============================================================================== --- trunk/docs/nut-english.txt (original) +++ trunk/docs/nut-english.txt Sat Oct 28 20:42:49 2006 @@ -137,6 +137,11 @@ the opaque global header object) are constant for the duration of the stream. +Each stream has a last_pts context. For compression, every frame's +pts is coded relatively to the last_pts. In order for demuxing to resume +from arbitrary points in the file, all last_pts contexes are reset by +syncpoints. + Frames NUT is built on the model that video, audio, and subtitle streams all @@ -239,5 +244,6 @@ (except those at end of file). The timestamp coded in the syncpoint is a global timestamp, which is used to reset the last_pts context of all streams, and to find the appropriate -syncpoint when seeking. back_ptr is explained in greater detail in the -Seeking section. +syncpoint when seeking. Demuxing can only begin at syncpoints for proper +last_pts context across all streams, including after seeking. +back_ptr is explained in greater detail in the Seeking section.

1 0

[nut]: r165 - trunk/docs/nut-english.txt
by ods15 28 Oct '06

28 Oct '06

Author: ods15 Date: Sat Oct 28 20:29:06 2006 New Revision: 165 Modified: trunk/docs/nut-english.txt Log: Explain VLC's... Modified: trunk/docs/nut-english.txt ============================================================================== --- trunk/docs/nut-english.txt (original) +++ trunk/docs/nut-english.txt Sat Oct 28 20:29:06 2006 @@ -43,6 +43,20 @@ reserved fields which can be skipped by old demuxers, and a CRC covering the packet from after the packet_header until the CRC itself. +Variable Length Coding + +Almost all fields in NUT are coded using VLCs. VLCs allow for compact +storage of small integers, which still being extendible to infinitely +large integers. The syntax is of a VLC is, per byte, a 1 bit flag stating +if there are more bits to the integer, and 7 bits to be prepended to the +lsb of the integer, shifting the previous value to the left. +Stuffing is allowed in VLCs by adding 0x80 bytes before the actual value, +but a maximum of 8 bytes of stuffing is allowed in any VLC outside of a +NUT packet. The only such fields are forward_ptr and fields in frame +headers. This is to prevent demuxers from looping on a large of amount of +damaged bytes of 0x80. Fields inside a NUT packet are protected by a CRC +whcih can be checked before decoding. + Header Structure A NUT file must begin with a magic identification string, followed by

1 0

[nut]: r164 - trunk/docs/nut-english.txt
by ods15 28 Oct '06

28 Oct '06

Author: ods15 Date: Sat Oct 28 20:14:40 2006 New Revision: 164 Modified: trunk/docs/nut-english.txt Log: explanation of NUT packets Modified: trunk/docs/nut-english.txt ============================================================================== --- trunk/docs/nut-english.txt (original) +++ trunk/docs/nut-english.txt Sat Oct 28 20:14:40 2006 @@ -29,6 +29,20 @@ beginning from 0. This allows simple and compact reference to streams in packet types where overhead must be kept to a minimum. +Packet Structure + +Every NUT packet has a packet_header and a packet_footer. +The packet_header consists of a 64-bit startcode, a forward_ptr, and an +optional CRC on the packet header if the forward_ptr is larger than 4k. +The forward_ptr gives the size of the packet from after the packet_header +until the end of the packet_footer. The optional CRC is to prevent a +demuxer from seeing a damaged startcode and forward_ptr with a high size, +causing it to skip or buffer a large part of the file, only to find it is +not really a NUT packet. +The packet_footer consists of reserved_bytes, which is room for any +reserved fields which can be skipped by old demuxers, and a CRC covering +the packet from after the packet_header until the CRC itself. + Header Structure A NUT file must begin with a magic identification string, followed by

1 0

[nut]: r163 - trunk/docs/nut-english.txt
by ods15 28 Oct '06

28 Oct '06

Author: ods15 Date: Sat Oct 28 20:02:37 2006 New Revision: 163 Modified: trunk/docs/nut-english.txt Log: explain syncpoints, could use a polish and more explanation... Modified: trunk/docs/nut-english.txt ============================================================================== --- trunk/docs/nut-english.txt (original) +++ trunk/docs/nut-english.txt Sat Oct 28 20:02:37 2006 @@ -202,4 +202,14 @@ Syncpoints -... +Syncpoints are mini-NUT packets, which serve for seeking, error recovery, +and error checking. They contain a startcode like all NUT packets, a +timestamp, a back_ptr, and a CRC on the packet itself. Syncpoints must be +placed every 32kb (or whatever max_distance is set to in the main header, +64kb at most), unless between the 2 syncpoints is a single frame. +Syncpoints must be followed by a frame, and must be placed after headers +(except those at end of file). +The timestamp coded in the syncpoint is a global timestamp, which is used +to reset the last_pts context of all streams, and to find the appropriate +syncpoint when seeking. back_ptr is explained in greater detail in the +Seeking section.

1 0