- NUT-devel - lists.mplayerhq.hu

Re: integrating AVS decoding into MPlayer
by Baptiste Coudurier 15 Jul '06

15 Jul '06

Rich Felker wrote: > On Sat, Jul 15, 2006 at 10:49:02PM +0200, Baptiste Coudurier wrote: >> NUT system is the best one around. Still I persist to say that wraping >> specs are needed, and even more for ambiguous codecs wraping formats, >> and a fourcc for each codec recognized must be written in the specs, >> that means standardized. > > Let's propose a compromise: additional information on the NUT website > or with the spec that's labelled "informative" as opposed to > "normative", clarifying what the general rules mean for particular > codecs. I totally agree. It will avoid fourcc collision which might occur according to the specs. Even in AVI some codecs use different fourcc... Im also thinking about DV, DVCPRO, DV50, DVCPRO100 which need a clearly specified fourcc. > What do you mean by timecodes? I assumed you meant timestamps but > maybe something else like chapter info..? > > As for subtitles, the subtitle codec_id specifies the format of the > data. Many format supports Timecode track which defines the timing reference not starting at 0 and reference GOP headers timecode in MPEG-2 for example. They allows non contiguous timing, and some other features. IMHO those tracks might need another stream identifier and some flags like drop frame flag, negative (reverse playing). > >> I just read the specs again and I also do not see and decent standard to >> pack extradata for specific codec. Just put the MOV atoms in NUT ? > > Extradata is a single binary data block whose format is specified by > the codec. For most codecs its obvious what it should be. For Xiph > codecs a 'supplemental codec spec' is needed, as has been discussed on > the nut devel list. For example QDM2 uses 2 atoms. It might be necessary to mention correct and clean formating. Btw all, sorry for all those points I am talking about now, Im just a bit late and I really want to participate in NUT revolution to the container world. -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA SMARTJOG S.A. http://www.smartjog.com Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA Phone: +33 1 49966312

1 0

nut-np (nut based network protocol)
by Michael Niedermayer 13 Jul '06

13 Jul '06

Hi is there a non bloated non broken protocol around for streaming multimedia over the net? maybe we should invent one? what about the following very simple one? nut-np packet: sequence_num v while(not end){ type v if(type == 0){ // one complete nut packet (main header, stream header, info packet, // syncpoint, normal packet) data vb or raw? }else if(type == 1){ //nut packet fragment fragment_count v //whole number of fragments fragment_num v //which fragment we have here fragment vb }else if(type == 2 || type == 3){ // acknowledge(2) / retransmit req(3) sequence_num_diff v }else if(type == 4 || type == 5 ){//server side seek stream_id v timestamp v // type=4 seek <=timestamp, type=5 >= }else if(type >=6 && type <=9){ // debug(6), warning(7) // error(8) (unsupported or failed command) // fatal(9) (cant recover / connection closed) sequence_num_diff v //sequence number this error referes to sub_seq_num v //which type+data part of the packet this refers to message vb }else if(type == 10 || type == 11){ // discard stream flag stream_id v // type=10 -> dont send stream / type=11 send }else if(type==12){ // keyframe request stream_id v }else if(type==13){ // set channel channel_id v }else if(type==14){ auto_retransmit_delay v }else if(type==15){ // forward error correction sequence_num_diff v // the number of packets this covers ecc_type v ecc vb or raw? } } optional packets: keyframe request, server side seek, set channel, auto_retransmit, forward error correction ecc_type MUST be 0, others are reserved ecc (type 0) simply the interleaved XOR of the bits of all covered packets so a single lost packet can be recovered without retransmit fragmentation stuff: * the decoder is generally able to find the start of slices as they have startcodes so we dont need to take care of that * the decoder can not use a slice if any part of that slice prior to it is missing ->encoders/nut-np servers should fragment packets so that the amount of data lost if a packet is lost, (that is everything inside that packet + all parts of slices where a earlier part of that slice has been lost) is minimized duplicate packets duplicate packets (same seq num) should be discarded and a acknowledge be sent retransmits (in case a backchannel is available and the outer protocol is unreliable) * the server can optionally retransmit a packet if no acknowledge has been received within auto_retransmit_delay and the packet is important (a frame used as reference an error message or similar) * the client and server MUST acknowledge important packets it received and ask for retransmits if something is missing and needed in which outer layer protocol this all is used is outside the scope of this spec -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is

4 8

r19062 - in trunk/DOCS/tech: mpcf.txt nut.txt
by michael 13 Jul '06

13 Jul '06

Author: michael Date: Thu Jul 13 12:29:21 2006 New Revision: 19062 Removed: trunk/DOCS/tech/mpcf.txt Changes in other areas also in this revision: Added: trunk/DOCS/tech/nut.txt - copied unchanged from r19061, /trunk/DOCS/tech/mpcf.txt Log: rename mpcf.txt to nut.txt

1 0

r19061 - trunk/DOCS/tech/mpcf.txt
by michael 13 Jul '06

13 Jul '06

Author: michael Date: Thu Jul 13 12:26:33 2006 New Revision: 19061 Modified: trunk/DOCS/tech/mpcf.txt Log: freeze (this is pretty much oded shimons patch from a few month ago) Modified: trunk/DOCS/tech/mpcf.txt ============================================================================== --- trunk/DOCS/tech/mpcf.txt (original) +++ trunk/DOCS/tech/mpcf.txt Thu Jul 13 12:26:33 2006 @@ -1,6 +1,6 @@ -======================================== -NUT Open Container Format DRAFT 20060322 -======================================== +================================== +NUT Open Container Format 20060713 +================================== @@ -31,6 +31,8 @@ damaged files can be played back with minimal data loss and fast resync times +The spec is frozen. All files following spec will be compatible unless the +spec is unfrozen. Definitions: @@ -390,7 +392,7 @@ 0xAB68B596BA78ULL + (((uint64_t)('N'<<8) + 'I')<<48) version - NUT version. The current value is 2. + NUT version. The current value is 3. All lower values are pre-freeze forward_ptr size of the packet data (exactly the distance from the first byte

1 0

Freezing nut
by Michael Niedermayer 07 Jul '06

07 Jul '06

Hi can we freeze nut or finally make it official nut 1.0 ? if noone objects within 7 days then ill do that, we can always do wording changes or further additions to nut afterwards as long as compatibility is maintained IMHO the question about what to do if a codec has no fourcc in avi or other such questions really is about as unrelated to the design of nut as is possible ... yes iam somewhat offensive here but if i where not then nothing would happen and most important if there is a problem with the current design then everyone _HAD_ half a year time to complain -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is

5 7

r18904 - trunk/DOCS/tech/mpcf.txt
by michael 06 Jul '06

06 Jul '06

Author: michael Date: Tue Jul 4 23:14:55 2006 New Revision: 18904 Modified: trunk/DOCS/tech/mpcf.txt Log: trying to end the codec id battle Modified: trunk/DOCS/tech/mpcf.txt ============================================================================== --- trunk/DOCS/tech/mpcf.txt (original) +++ trunk/DOCS/tech/mpcf.txt Tue Jul 4 23:14:55 2006 @@ -443,6 +443,8 @@ example: "H264" MUST contain 2 or 4 bytes, note, this might be increased in the future if needed + the id values used are the same as in avi, so if a codec uses a specific + fourcc in avi then the same fourcc MUST be used here time_base_nom / time_base_denom = time_base the length of a timer tick in seconds, this MUST be equal to the 1/fps @@ -716,6 +718,11 @@ "Source" "DVD", "VCD", "CD", "MD", "FM radio", "VHS", "TV", "LD" Optional: appended PAL, NTSC, SECAM, ... in parentheses + "SourceContainer" + "nut", "mkv", "mov", "avi", "ogg", "rm", "mpeg-ps", "mpeg-ts", "raw" + "SourceCodecTag" + the source codec id like a fourcc which was used to store a specific + stream in its SourceContainer "CaptureDevice" "BT878", "BT848", "webcam", ... (more exact names are fine too) "CreationTime"

5 6

Codec id - random notes
by Luca Barbato 01 Jul '06

01 Jul '06

0 a player (sane) has a physical stream layer (I/O), a demuxer layer, upper layers till the users. We are discussing about a container so we should care about that happened at the demuxer layer, nothing more higher. 1 why we need codec id (tag) and we cannot just go tagless and let the decoders try to parse the stream and tell if they can decode it or not? - Raw formats - takes much more time - Isn't something the container level should do, players willing to play guess games can just ignoring the proposed tag. - the player could just use the tag, lookup which codec could fit and play it. 2 why 32bit? - It's a nice number for the cpu. - enough space for now - we can just take another 32bit tag system and reduce to our needs, so current players won't have to convert to their internal presentation in complex ways (hash lists, multicolum lookups, whatever). 3 why not a fixed field? - 640k are enough for everybody (learn for other people errors) 4 why not taking JUST one previous tag system then? - most of them aren't bijective 5 why just alphanumeric/Ascii for the tags? - already present in the established systems summary: we could restrict our field to "just" 32bit for now since suits our needs, we just take quicktime ids and clean them up so we have just a 1:1 mappings. Side notes: a- what about workarounds you need to apply for certain encoders? - usually such hints could be guessed from the stream itself. - a proper decoder should conform to the standard completely and not need hints - they could be replicated in the info packet w/out many issues. b- what about new codecs/formats? - we provide some guidelines about new tags - we provide a lint tool to spot files breaking such guidelines. - new tags will be accepted either by webform or ml submission if the codec developers want the format officially supported before nut devels will notice themselves an add it to the list. - non official stuff won't be nut compliant. That's all for now, more structured reasoning will appear once I've slept enough. lu -- Luca Barbato Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero

6 32

codec id nonsense, please end it
by Rich Felker 27 Jun '06

27 Jun '06

we were on the verge of finishing nut entirely when this whole flamewar started and i'd like to see it end. while i agree with ivan's sentiment that we do not want the huge mess of random nonsensical fourcc's creeping into nut, i'm also quite confident that ivan's proposal will not solve the problems he claims it will. as long as a "new style" codec id is required (and if it's not required it's useless and stupid like matroska's dual system) then whenever users want to use a codec that's not in the official list they'll just copy a random fourcc or other string to use as the codec identifier. on the other hand, the disadvantages of making a new codec id system are significant. most importantly, it requires every player that wants to support nut to either switch to using the new codec id system (nut-centric) and map all old fourcc's from other containers to the new system, or to map all nut codec ids to fourccs. both of these procedures require pointless large conversion tables. while the fourcc situation historically is ugly, the proposal i've been mostly agreeing with (i don't call it mine because iirc it was put together from the ideas of many people) seems the best to remedy the situation. like all legitimate standardization processes in which the party making the standard is providing rules whose influence extend outside their scope, our process for standardizing codec ids should: 1. abstract common characteristics of historical implementations 2. eliminate characteristics which are vendor-specific and/or provide duplicate or ambiguous functionality 3. only as a last resort, fill in new definitions where steps 1 and 2 have left any gaps. in our case, this means looking at existing fourcc values and other fourcc-compatible codec identifiers used in avi, quicktime, nsv, etc. and choosing among them the ones which are sane (human-readable and vendor-neutral). this resulted in a list like "mp4v", "mp3 ", "h264", "mpg1", ... the benefit of using existing fourcc's is significant because any multi-format player can lookup a working decoder for a stream with its existing fourcc->decoder tables rather than requiring a whole new set of tables. while designing a new codec id system could be useful, it is beyond the scope of a container format simply because of big-O considerations. if each universal (any-codec) container has its own nonsense codec id system, then the size of the tables grows as O(num_containers*num_codecs) instead of just O(num_codecs). i do not object at all to commenting in the nut spec that, in the event of a new future codec id standard that's well-designed, a future nut version may support using the new id system instead of fourcc. however, making a new codec id system and getting all containers to adopt it is WAAAY to ambitious of a project right now and WAAAAY outside the scope of what nut is trying to do. as for ivan's concern that lamers will just use crap fourccs (like DIVX, XVID, X264, ...) instead of the correct/standardized ones, there are lots of other things lamers can do too, and nutlint and/or the reference demuxer should warn about or discourage their stupidity. actually i would like to see a list of "illegal fourccs" in the reference muxer so it knows all the classic vendor-specific/ambiguous fourccs and rejects them as permenantly invalid. this would be a fairly small table, just 4 bytes per fourcc. other ideas welcome, but i really hope we can put this issue aside asap and get on with release.. rich

6 12

FourCC vs format_id
by Ivan Kalvachev 18 Jun '06

18 Jun '06

First few word: Codec identification is actually most important part of the container, because all other so technically great features are "under the hood". Fourcc is among the few things that users (even windows one!) are aware these days . Second: The real issue is not how we will save and name codecs. The real issue is how to prevent propagation of existing fourcc schemes into nut, with all their drawbacks. What I think we may need: format_id - mandatory field. identification of the format, by standard or bitstream syntax. e.g. "mpeg2","h264","theora", "vorbis". profile - optional field. depends on the specifications (e.g. main simple, etc...). This field informs that full possibilities of the format are not used. encoder - optional. Identifier of the encoder used to create the stream. Could be used to workaround bugs. Or blame for poor quality. fourcc - optional. Backward compatible code if the format have such. Same as the file transcoded or one from the "clean list". Used only by decoders that don't have format_id table. --- At first decoders will use only fourcc, because that's what they know. When nut is widely accepted there will be more decoders that use format_id, and not fourcc. When other format dies, so will do fourcc. You may wonder what's the difference between format_id and fourcc. Think of format_id as trademark. Lusers may load fourcc with all the crap from avi and qt, but format_id is reserved only for official standards and bitstreams. Of course anybody could try to make up new format_id for existing standards. However giving that there are other means for having clean separations (profile,encoder), it could not happen by accident. The profile field is needed to prevent spamming and spawning of new format_id's for newer versions of the stream (e.g. theroa3.4.12). It could also be used by hardware players/stream servers to pick up streams they can play. The encoder field is the field that is supposed to take all the spam from the clone codec makers. It should describe exactly the encoder and to even allow workarounding the know encoder bugs.(e.g. xvid id and bitstream version that is increased every time encoder bug is found and fixed) P.S. Even if you don't accept format_id and leave only the fourcc, I would be happy if you think a little about profile and encoder_id.

3 4

Re: [NUT-devel] Freeze NUT, Flame FourCC
by Ivan Kalvachev 18 Jun '06

18 Jun '06

2006/6/3, Michael Niedermayer <michaelni(a)gmx.at>: > Hi > > On Sat, Jun 03, 2006 at 10:00:02AM +0300, Ivan Kalvachev wrote: > > 2006/6/1, Michael Niedermayer <michaelni(a)gmx.at>: > > >Hi > > > > > >On Thu, Jun 01, 2006 at 06:37:10PM +0300, Oded Shimon wrote: > > >> On Thu, Jun 01, 2006 at 05:13:34PM +0200, Luca Barbato wrote: > > >> > Michael Niedermayer wrote: > > >> > > > > >> > > some random ideas (dunno if good or not) > > >> > > allow multiple codec identifers: > > >> > > source fourcc (what the source in case of remuxing had) > > >> > > long name ("Vorbis", "ITU H.264", ...) > > >> > > simplifed fourcc (XVID,DIVX,FFMP4->M4V; ...) > > >> > > and make one of these mandatory and put the other 2 in an info > > >packet maybe I would really like if we work in this direction. I think I even proposed something like this in the previous thread. > > >> > could be useful in certain case, how much complex it could become? > > >> > > >> ... i don't like it > > >> I say go back to 4 byte method, and explicitly say to the spec it must be > > >> 4 bytes, and even remove the 'length' field... > > > > > >i am not against a 32bit fixed field ... > > > > I am against. > > > > If we are going to use 32bit fields. If we are going to use this for > > compatibility. > > Then I say to use the AVI fourcc and don't mind to make our own > > incompatible with everything fourcc table. > > > > However I really would NOT like to criple nut in such horribly ugly hackish > > way. > > Especally when everything else in nut is variable size... > > well almost everything might be variable size but these things also ensure > that values stay small if possible (width/height/sample_rate/...) fit in > 32bit for videos todays players support, (stream_id, ...) must be a small > number, having a 2 stream file with 1 stream with id 1<<99 isnt allowed > > now with fourcc/codec_id its different, its not that >32bit can be used if > we run out of 32bit values while we wouldnt need to bother today but that > it must be supported no matter if we ever run out of 32bit values ... > > so there are several choices: > 1. store as 32bit or store as variable length but limit to 32bit in the spec > fits in int > "only" 1<<32 codecs > if the codec/source/destination stream doesnt support a 32bit id then > we must convert > > 2. store arbitrary length > needs malloc()/free()/memcmp(), wont fit in int > unlimited number of codecs > if the codec/destination stream doesnt support variable length id then > we must convert > > there are further possibilities like storing both or requireing the first 4 > letters to uniquely identify the codec but these seem to be silly somehow ... > > theres no doubt that 2. is much more complex, so i vote for 1. if you > dissagree id like to hear _technical_ arguments (not hackish, var size is > better then fixed philosophy with no explanation why this applies here) I'll try. 1. We don't need limit on the codecs that could be used with nut. 2. It is generally good idea the codec_id to have some human understandable and google-able meaning, like codec name. e.g. "Unknown format 'vorbis' " 3. We won't use the full range of 32bit. This means we have more like 26^4 (or 36^4 or 40^4). We are also colliding with AVI FourCC (avi4cc). 3.1 Using full range means something like 0x0001->mpeg1, 0x0002->mpeg2. It would be entirely cryptic if player reports "Unknown fourcc 0xFAC0" (hum... i've seen this somewhere;) 3.2. We could cause mass confusion with defining our own 4cc. If somebody can't find his codec in our list, he will just use the same 4cc as in avi. Then they will start using avi4cc even for formats that we have different codecs. The real fun would come when/if same 4cc is used for different codecs in the different containers. 3.3. Even now the avi4cc is causing problems. Same format occupies many 4cc. Same 4cc are used for different codecs. Worse, in time codecs will try to use any available combination that even may not have anything in common with codec name. "Unknown fourcc 'JQWZ' " 4. malloc/free memcpy/memcmp is not really an issue. It is not time critical, nor it allocates megabytes of memory. Comparing speed isn't issue here, as it is done only once. (not even talking about realloc or stream_header raw buffer hacks). 5. > if the codec/destination stream doesnt support variable length id then > we must convert I'm really having problem understanding that. I'll assume you mean that if player doesn't know the codec_id string we will have to convert to 4cc it knows. Well this logic is totally flawed. 5.1. We cannot know what 4cc the player knows. 5.2. We are going to use our own nut4cc codes. We even started making a list. If player doesn't know our codec_id then it won't know and our nut4cc. Making nut4cc->avi4cc totally beats the reason of having nut4cc. 5.3. Burdening the demuxer with code matching tables is counter effective. These tables should be updated on regular basis....

6 10