July 2006 - NUT-devel - lists.mplayerhq.hu

codec_specific_data
by Michael Niedermayer 18 Jul '06

18 Jul '06

Hi each stream has one global header in nut, matroska and avi too have exactly one global header APIs like the ones from mplayer and ffmpeg and directshow AFAIK are also designed based on a single global header but a very small number of codecs are designed so that they have several global headers but no standarized way (that i know of) of converting them to a single header, the "ogg codecs" fall in this category several solutions exist: 1. store several headers at nut level 2. leave it unspecified 3. specify how these multi packet codecs should be stored be it in nut.txt or in a seperate file if we do specify how these codecs should be stored then we can either A. invent our own system (this is for all practical proposes is identical to storing them at nut level) B. borrow the system from another container format leaving it unspecified will lead to a mess of different systems used by different muxers storing them at nut level will need them to be converted to a single packet format during demuxing and to a multi packet format during nut muxing in most cases borrowing another system and using that throughout an application will lead to the least converting around -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is

3 3

r19128 - trunk/DOCS/tech/nut.txt
by michael 17 Jul '06

17 Jul '06

Author: michael Date: Mon Jul 17 23:15:27 2006 New Revision: 19128 Modified: trunk/DOCS/tech/nut.txt Log: remove vorbis and theora support by request of ivan Modified: trunk/DOCS/tech/nut.txt ============================================================================== --- trunk/DOCS/tech/nut.txt (original) +++ trunk/DOCS/tech/nut.txt Mon Jul 17 23:15:27 2006 @@ -515,16 +515,6 @@ if a codec has a global header it SHOULD be placed in here instead of at the start of every keyframe the exact format is specified in the codec spec - codecs which dont specify it in their spec are specified below - for ogg based codecs (vorbis, theora) the following format shall be used - number_of_headers_minus_1 u(8) - for(i=0; i<number_of_headers; i++){ - -1 u(8*(size[i]/255)) - size[i] % 255 u(8) - } - for(i=0; i<number_of_headers; i++) - header[i] - Note, this is the same format these codecs use in matroska for H.264 the NAL units MUST be formated as in a bytestream (with 00 00 01 prefixes)

1 0

r19126 - trunk/DOCS/tech/nut.txt
by michael 17 Jul '06

17 Jul '06

Author: michael Date: Mon Jul 17 01:47:12 2006 New Revision: 19126 Modified: trunk/DOCS/tech/nut.txt Log: clarify H.264 global header format Modified: trunk/DOCS/tech/nut.txt ============================================================================== --- trunk/DOCS/tech/nut.txt (original) +++ trunk/DOCS/tech/nut.txt Mon Jul 17 01:47:12 2006 @@ -525,6 +525,8 @@ for(i=0; i<number_of_headers; i++) header[i] Note, this is the same format these codecs use in matroska + for H.264 the NAL units MUST be formated as in a bytestream + (with 00 00 01 prefixes) frame_code the meaning of this byte is stored in the main header

4 6

The codecs which have no fourcc in AVI
by Michael Niedermayer 16 Jul '06

16 Jul '06

Hi To make ivan happy, this thread is dedicated to the issue of what to do if our selected codec id system doesnt have a id for a specific codec an issue which many other container specs dont seem to care about ... so what can be done ... * IMO we should make a list of codec ids for codecs which are common but which arent stored in avi normally, well acually we already did make that list (see mailinglist, it was done with the sligtly different goal of avoiding implementation specific ids but when we drop these the rest is what we need here), also note my proposal here is just a small <10 entry list nothing bigger so whats left are "rare" codecs which arent stored in avi i think theres just one thing we can do here, and that is to tell people to contact us to discuss/ register a id for their codec whats the difference from that to a nut specific id system? makng and maintaining a list of all codecs-implementations-ids vs. maintaining a list of very few ids, just thouse which are not supported by avi and which people want to use in nut and where the designers of the codec didnt specify a recommanded fourcc already in their spec a few random notes ... * people dont contact microsoft/iso/apple to register their ids or did anyone? i dunno ... * thouse who will attempt to store codec X in nut for the first time will likely not be the ones who designed the codec and they will very likely not have read the nut spec and as such will not know how to select a codec id -> they will not contact us -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is

2 2

Re: [Ffmpeg-devel] Re: integrating AVS decoding into MPlayer
by Michael Niedermayer 16 Jul '06

16 Jul '06

Hi On Sat, Jul 15, 2006 at 10:49:02PM +0200, Baptiste Coudurier wrote: [...] > > [...] > > > > Care to explain how these rules will cease to apply? > > Like they changed in those years, when compression appeared, then VBR, > who knows... > > NUT system is the best one around. Still I persist to say that wraping > specs are needed, and even more for ambiguous codecs wraping formats, > and a fourcc for each codec recognized must be written in the specs, > that means standardized. for ambiguous cases we certainly should say something in the spec but most arent ambiguous, if you follow the spec of a codec and the nut spec and use a little common sense ... [...] > > >> I persist to say that a container SHALL standardize a codec wraping > >> (even just saying "default wraping rule applies" and its fourcc is ...) > >> if it needs to support it. It should just be a matter of one line per > >> codec, as NUT is very generic ;) > > > > The one line would be the same for every codec so it's pointless. > > IMHO not for H264, what exactly is unclear in the h.264 spec? > > > What about timecode ? Subtitles ? > > There is no specifications in NUT specs about wraping those kind of > data. A stream type identifier is not sufficient IMHO. > > I just read the specs again and I also do not see and decent standard to > pack extradata for specific codec. Just put the MOV atoms in NUT ? which codec spec is unclear about how to store it? ogg based codecs yes, that needs a line to clarify > > Btw this thread now belongs to NUT devel IMHO. yes [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is

3 3

Re: integrating AVS decoding into MPlayer
by Baptiste Coudurier 16 Jul '06

16 Jul '06

Rich Felker wrote: > On Sat, Jul 15, 2006 at 10:49:02PM +0200, Baptiste Coudurier wrote: >> NUT system is the best one around. Still I persist to say that wraping >> specs are needed, and even more for ambiguous codecs wraping formats, >> and a fourcc for each codec recognized must be written in the specs, >> that means standardized. > > Let's propose a compromise: additional information on the NUT website > or with the spec that's labelled "informative" as opposed to > "normative", clarifying what the general rules mean for particular > codecs. I totally agree. It will avoid fourcc collision which might occur according to the specs. Even in AVI some codecs use different fourcc... Im also thinking about DV, DVCPRO, DV50, DVCPRO100 which need a clearly specified fourcc. > What do you mean by timecodes? I assumed you meant timestamps but > maybe something else like chapter info..? > > As for subtitles, the subtitle codec_id specifies the format of the > data. Many format supports Timecode track which defines the timing reference not starting at 0 and reference GOP headers timecode in MPEG-2 for example. They allows non contiguous timing, and some other features. IMHO those tracks might need another stream identifier and some flags like drop frame flag, negative (reverse playing). > >> I just read the specs again and I also do not see and decent standard to >> pack extradata for specific codec. Just put the MOV atoms in NUT ? > > Extradata is a single binary data block whose format is specified by > the codec. For most codecs its obvious what it should be. For Xiph > codecs a 'supplemental codec spec' is needed, as has been discussed on > the nut devel list. For example QDM2 uses 2 atoms. It might be necessary to mention correct and clean formating. Btw all, sorry for all those points I am talking about now, Im just a bit late and I really want to participate in NUT revolution to the container world. -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA SMARTJOG S.A. http://www.smartjog.com Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA Phone: +33 1 49966312

1 0

nut-np (nut based network protocol)
by Michael Niedermayer 13 Jul '06

13 Jul '06

Hi is there a non bloated non broken protocol around for streaming multimedia over the net? maybe we should invent one? what about the following very simple one? nut-np packet: sequence_num v while(not end){ type v if(type == 0){ // one complete nut packet (main header, stream header, info packet, // syncpoint, normal packet) data vb or raw? }else if(type == 1){ //nut packet fragment fragment_count v //whole number of fragments fragment_num v //which fragment we have here fragment vb }else if(type == 2 || type == 3){ // acknowledge(2) / retransmit req(3) sequence_num_diff v }else if(type == 4 || type == 5 ){//server side seek stream_id v timestamp v // type=4 seek <=timestamp, type=5 >= }else if(type >=6 && type <=9){ // debug(6), warning(7) // error(8) (unsupported or failed command) // fatal(9) (cant recover / connection closed) sequence_num_diff v //sequence number this error referes to sub_seq_num v //which type+data part of the packet this refers to message vb }else if(type == 10 || type == 11){ // discard stream flag stream_id v // type=10 -> dont send stream / type=11 send }else if(type==12){ // keyframe request stream_id v }else if(type==13){ // set channel channel_id v }else if(type==14){ auto_retransmit_delay v }else if(type==15){ // forward error correction sequence_num_diff v // the number of packets this covers ecc_type v ecc vb or raw? } } optional packets: keyframe request, server side seek, set channel, auto_retransmit, forward error correction ecc_type MUST be 0, others are reserved ecc (type 0) simply the interleaved XOR of the bits of all covered packets so a single lost packet can be recovered without retransmit fragmentation stuff: * the decoder is generally able to find the start of slices as they have startcodes so we dont need to take care of that * the decoder can not use a slice if any part of that slice prior to it is missing ->encoders/nut-np servers should fragment packets so that the amount of data lost if a packet is lost, (that is everything inside that packet + all parts of slices where a earlier part of that slice has been lost) is minimized duplicate packets duplicate packets (same seq num) should be discarded and a acknowledge be sent retransmits (in case a backchannel is available and the outer protocol is unreliable) * the server can optionally retransmit a packet if no acknowledge has been received within auto_retransmit_delay and the packet is important (a frame used as reference an error message or similar) * the client and server MUST acknowledge important packets it received and ask for retransmits if something is missing and needed in which outer layer protocol this all is used is outside the scope of this spec -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is

4 8

r19062 - in trunk/DOCS/tech: mpcf.txt nut.txt
by michael 13 Jul '06

13 Jul '06

Author: michael Date: Thu Jul 13 12:29:21 2006 New Revision: 19062 Removed: trunk/DOCS/tech/mpcf.txt Changes in other areas also in this revision: Added: trunk/DOCS/tech/nut.txt - copied unchanged from r19061, /trunk/DOCS/tech/mpcf.txt Log: rename mpcf.txt to nut.txt

1 0

r19061 - trunk/DOCS/tech/mpcf.txt
by michael 13 Jul '06

13 Jul '06

Author: michael Date: Thu Jul 13 12:26:33 2006 New Revision: 19061 Modified: trunk/DOCS/tech/mpcf.txt Log: freeze (this is pretty much oded shimons patch from a few month ago) Modified: trunk/DOCS/tech/mpcf.txt ============================================================================== --- trunk/DOCS/tech/mpcf.txt (original) +++ trunk/DOCS/tech/mpcf.txt Thu Jul 13 12:26:33 2006 @@ -1,6 +1,6 @@ -======================================== -NUT Open Container Format DRAFT 20060322 -======================================== +================================== +NUT Open Container Format 20060713 +================================== @@ -31,6 +31,8 @@ damaged files can be played back with minimal data loss and fast resync times +The spec is frozen. All files following spec will be compatible unless the +spec is unfrozen. Definitions: @@ -390,7 +392,7 @@ 0xAB68B596BA78ULL + (((uint64_t)('N'<<8) + 'I')<<48) version - NUT version. The current value is 2. + NUT version. The current value is 3. All lower values are pre-freeze forward_ptr size of the packet data (exactly the distance from the first byte

1 0

Freezing nut
by Michael Niedermayer 07 Jul '06

07 Jul '06

Hi can we freeze nut or finally make it official nut 1.0 ? if noone objects within 7 days then ill do that, we can always do wording changes or further additions to nut afterwards as long as compatibility is maintained IMHO the question about what to do if a codec has no fourcc in avi or other such questions really is about as unrelated to the design of nut as is possible ... yes iam somewhat offensive here but if i where not then nothing would happen and most important if there is a problem with the current design then everyone _HAD_ half a year time to complain -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is

5 7