TotalTime is unecessary thanks to max_pts in index no more null termination of info packets stream_id coded sperately seperate to 4 basic types for extendibility, now we can add new fields without breaking old demuxers more: maybe use null terminated strings instead of vb? The reason I gave 'NULL' as an option for type for basic codes, maybe those 1 or 2 vlc's can take on a different meaning with some other type... Also I consider collapsing rational and float into a single code, with different types. They decode the same, being 2 vlc's... - ods15
On Wed, Feb 15, 2006 at 10:34:04AM +0200, Oded Shimon wrote:
TotalTime is unecessary thanks to max_pts in index no more null termination of info packets stream_id coded sperately seperate to 4 basic types for extendibility, now we can add new fields without breaking old demuxers
more: maybe use null terminated strings instead of vb?
never mind...
The reason I gave 'NULL' as an option for type for basic codes, maybe those 1 or 2 vlc's can take on a different meaning with some other type...
Also I consider collapsing rational and float into a single code, with different types. They decode the same, being 2 vlc's...
what about this?... time changed to rational, not integer/msecs. float fixed.. Also, any objections to this cleanup patch?.. - ods15
Hi On Wed, Feb 15, 2006 at 10:34:04AM +0200, Oded Shimon wrote:
TotalTime is unecessary thanks to max_pts in index
ok
no more null termination of info packets
ok
stream_id coded sperately
ok if multiple stream_ids per info packet are possible
seperate to 4 basic types for extendibility, now we can add new fields without breaking old demuxers
please elaborate [...]
+ } else if(code==2) { + mantissa s + exponent s + value= abs(mantissa)^exp + value*= sign(mantissa)
interresting ;) [...] -- Michael
On Wed, Feb 15, 2006 at 11:25:13AM +0100, Michael Niedermayer wrote:
Hi
On Wed, Feb 15, 2006 at 10:34:04AM +0200, Oded Shimon wrote:
TotalTime is unecessary thanks to max_pts in index
ok
no more null termination of info packets
ok
stream_id coded sperately
ok if multiple stream_ids per info packet are possible
Hmm, no such possilbity, it's all or one... What's the usefulness?
seperate to 4 basic types for extendibility, now we can add new fields without breaking old demuxers
please elaborate
If someone makes up a new header, and we decide we like it and add it to info table, using the old method, old demuxers not knowing the new info entry would crap out on the entire info packet because of the v/vb stuff. With the type stuff, it can be done in backwards compatible way, lacking the demuxer just understand the new header... Also it's done in a way where we can force muxers to use rational and not float for some values or similar... Or force utf-8, etc.
[...]
+ } else if(code==2) { + mantissa s + exponent s + value= abs(mantissa)^exp + value*= sign(mantissa)
interresting ;)
I'm assuming you mean the totally incorrect way I did it, sorry, I was confused. :) Already sent a fixed patch. Commit... ? - ods15
On Wed, Feb 15, 2006 at 12:59:50PM +0200, Oded Shimon wrote:
On Wed, Feb 15, 2006 at 11:25:13AM +0100, Michael Niedermayer wrote:
On Wed, Feb 15, 2006 at 10:34:04AM +0200, Oded Shimon wrote:
seperate to 4 basic types for extendibility, now we can add new fields without breaking old demuxers
please elaborate
If someone makes up a new header, and we decide we like it and add it to info table, using the old method, old demuxers not knowing the new info entry would crap out on the entire info packet because of the v/vb stuff. With the type stuff, it can be done in backwards compatible way, lacking the demuxer just understand the new header...
Also it's done in a way where we can force muxers to use rational and not float for some values or similar... Or force utf-8, etc.
We just noticed the problem with demuxer not knowing if type or name are NULL, in which case if to read a vb or not. So I suggest adding the restriction - no future headers will have NULL fields. I don't think it's a big problem, I doubt any ever will... Only the existing fields can have NULL value.. - ods15
Hi On Wed, Feb 15, 2006 at 12:59:50PM +0200, Oded Shimon wrote:
On Wed, Feb 15, 2006 at 11:25:13AM +0100, Michael Niedermayer wrote:
Hi
On Wed, Feb 15, 2006 at 10:34:04AM +0200, Oded Shimon wrote:
TotalTime is unecessary thanks to max_pts in index
ok
no more null termination of info packets
ok
stream_id coded sperately
ok if multiple stream_ids per info packet are possible
Hmm, no such possilbity, it's all or one... What's the usefulness?
just a factor of 10 reduction of overhead for example take a movie with a few audio and subtitle streams, then lets assume it does contain some info packets which apply to all the streams like the movie name, and various other metadata ... now add another audio stream of your favorite music as background music, none of the "global" info applies so you must duplicate all the info packets for every stream or add a fansubbed sutitle stream, many things like studio, producer, director and so on wont apply to it while they apply to all other streams
seperate to 4 basic types for extendibility, now we can add new fields without breaking old demuxers
please elaborate
If someone makes up a new header, and we decide we like it and add it to info table, using the old method, old demuxers not knowing the new info entry would crap out on the entire info packet because of the v/vb stuff. With the type stuff, it can be done in backwards compatible way, lacking the demuxer just understand the new header...
just store the f* table in the main header :) [...] -- Michael
On Wed, Feb 15, 2006 at 03:23:30PM +0100, Michael Niedermayer wrote:
Hi
On Wed, Feb 15, 2006 at 12:59:50PM +0200, Oded Shimon wrote:
On Wed, Feb 15, 2006 at 11:25:13AM +0100, Michael Niedermayer wrote:
Hi
On Wed, Feb 15, 2006 at 10:34:04AM +0200, Oded Shimon wrote:
TotalTime is unecessary thanks to max_pts in index
ok
no more null termination of info packets
ok
stream_id coded sperately
ok if multiple stream_ids per info packet are possible
Hmm, no such possilbity, it's all or one... What's the usefulness?
just a factor of 10 reduction of overhead for example take a movie with a few audio and subtitle streams, then lets assume it does contain some info packets which apply to all the streams like the movie name, and various other metadata ... now add another audio stream of your favorite music as background music, none of the "global" info applies so you must duplicate all the info packets for every stream or add a fansubbed sutitle stream, many things like studio, producer, director and so on wont apply to it while they apply to all other streams
This may be a question of opinion, but IMO "producor/author/title" etc. still apply to all streams/entire file, and then maybe you have a packet applied to the subtitle stream. Infact, "director" makes no sense to apply to any stream, it's not a video thing or audio thing, it's a movie (as in, entire file) thing. But I do see your point, do you have a suggestion for how to code it? (in stream, using vlc's. not in C code :) Maybe a bitmask? 0 meaning all streams?
seperate to 4 basic types for extendibility, now we can add new fields without breaking old demuxers
please elaborate
If someone makes up a new header, and we decide we like it and add it to info table, using the old method, old demuxers not knowing the new info entry would crap out on the entire info packet because of the v/vb stuff. With the type stuff, it can be done in backwards compatible way, lacking the demuxer just understand the new header...
just store the f* table in the main header :)
Bleh. :) It's a nice solution, but it's somewhat bad in that it adds info stuff in essential, small, main header (slightly defeating the purpose). Also it makes it harder for us to enforce our premade headers... (utf-8, etc.) - ods15
Hi On Wed, Feb 15, 2006 at 05:14:25PM +0200, Oded Shimon wrote:
On Wed, Feb 15, 2006 at 03:23:30PM +0100, Michael Niedermayer wrote:
Hi
On Wed, Feb 15, 2006 at 12:59:50PM +0200, Oded Shimon wrote:
On Wed, Feb 15, 2006 at 11:25:13AM +0100, Michael Niedermayer wrote:
Hi
On Wed, Feb 15, 2006 at 10:34:04AM +0200, Oded Shimon wrote:
TotalTime is unecessary thanks to max_pts in index
ok
no more null termination of info packets
ok
stream_id coded sperately
ok if multiple stream_ids per info packet are possible
Hmm, no such possilbity, it's all or one... What's the usefulness?
just a factor of 10 reduction of overhead for example take a movie with a few audio and subtitle streams, then lets assume it does contain some info packets which apply to all the streams like the movie name, and various other metadata ... now add another audio stream of your favorite music as background music, none of the "global" info applies so you must duplicate all the info packets for every stream or add a fansubbed sutitle stream, many things like studio, producer, director and so on wont apply to it while they apply to all other streams
This may be a question of opinion, but IMO "producor/author/title" etc. still apply to all streams/entire file, and then maybe you have a packet applied to the subtitle stream. Infact, "director" makes no sense to apply to any stream, it's not a video thing or audio thing, it's a movie (as in, entire file) thing. But I do see your point, do you have a suggestion for how to code it? (in stream, using vlc's. not in C code :) Maybe a bitmask? 0 meaning all streams?
hmmmmmm, what about decompress_element(a, i, count) type v x= type>>1; if(type&1){ x>>=1 while(x--) a[i++] = !!(type&2) a[i++] = !(type&2); }else{ while(x != 1){ a[i++] = x&1; x>>=1; } } return i and then use this for both the index (we aleady do ...) and info packets info_packet for(i=0; i<stream_count; ) i= decompress_element(streams, i, stream_count)
seperate to 4 basic types for extendibility, now we can add new fields without breaking old demuxers
please elaborate
If someone makes up a new header, and we decide we like it and add it to info table, using the old method, old demuxers not knowing the new info entry would crap out on the entire info packet because of the v/vb stuff. With the type stuff, it can be done in backwards compatible way, lacking the demuxer just understand the new header...
just store the f* table in the main header :)
Bleh. :) It's a nice solution, but it's somewhat bad in that it adds info stuff in essential, small, main header (slightly defeating the purpose). Also it makes it harder for us to enforce our premade headers... (utf-8, etc.)
we cant enforce that anyway, just becaue it has utf-8 as type doensnt mean it is utf8, if someone decides to ignore a rule in the spec which says only utf-8 allowed, then he will care little about the type being enforced to utf-8 actually later is worse as its undetectable, so IMHO better give "them" an option to specify that they violate the rules instead of making it impossible to specify the type while obviously not preventing them from storing things in their native encoding ... [...] -- Michael
On Wed, Feb 15, 2006 at 05:46:37PM +0100, Michael Niedermayer wrote:
On Wed, Feb 15, 2006 at 05:14:25PM +0200, Oded Shimon wrote:
This may be a question of opinion, but IMO "producor/author/title" etc. still apply to all streams/entire file, and then maybe you have a packet applied to the subtitle stream. Infact, "director" makes no sense to apply to any stream, it's not a video thing or audio thing, it's a movie (as in, entire file) thing. But I do see your point, do you have a suggestion for how to code it? (in stream, using vlc's. not in C code :) Maybe a bitmask? 0 meaning all streams?
hmmmmmm, what about
decompress_element(a, i, count) type v x= type>>1; if(type&1){ x>>=1 while(x--) a[i++] = !!(type&2) a[i++] = !(type&2); }else{ while(x != 1){ a[i++] = x&1; x>>=1; } } return i
Warning: `count' unused parameter. :) Maybe you meant to put the loop here?
and then use this for both the index (we aleady do ...) and info packets
info_packet for(i=0; i<stream_count; ) i= decompress_element(streams, i, stream_count)
I dislike it. For this purpose, a pure bitmask with special case 0 is good enough... It's not quite a complication but it is simply not worth it... (bitmask has the slight disadvantage of overflowing, but seriously, 64 streams... Worst pathological scenario I can think of is 30 streams, 20 subtitles and 10 audio tracks)
just store the f* table in the main header :)
Bleh. :) It's a nice solution, but it's somewhat bad in that it adds info stuff in essential, small, main header (slightly defeating the purpose). Also it makes it harder for us to enforce our premade headers... (utf-8, etc.)
we cant enforce that anyway, just becaue it has utf-8 as type doensnt mean it is utf8, if someone decides to ignore a rule in the spec which says only utf-8 allowed, then he will care little about the type being enforced to utf-8 actually later is worse as its undetectable, so IMHO better give "them" an option to specify that they violate the rules instead of making it impossible to specify the type while obviously not preventing them from storing things in their native encoding ...
That's somewhat a good point, but still not excuse enough IMO to put the info types in the main header, it literally defeats the purpose - info stuff goes in info packets, not in main headers... And, well, let's hope no-one breaks the utf-8 rule.. :) Actually, it's better - they can still violate with our "enforcing", but it will be expensive (overhead). So they will do it at their own expenses... - ods15
Hi On Wed, Feb 15, 2006 at 08:05:19PM +0200, Oded Shimon wrote:
On Wed, Feb 15, 2006 at 05:46:37PM +0100, Michael Niedermayer wrote:
On Wed, Feb 15, 2006 at 05:14:25PM +0200, Oded Shimon wrote:
This may be a question of opinion, but IMO "producor/author/title" etc. still apply to all streams/entire file, and then maybe you have a packet applied to the subtitle stream. Infact, "director" makes no sense to apply to any stream, it's not a video thing or audio thing, it's a movie (as in, entire file) thing. But I do see your point, do you have a suggestion for how to code it? (in stream, using vlc's. not in C code :) Maybe a bitmask? 0 meaning all streams?
hmmmmmm, what about
decompress_element(a, i, count) type v x= type>>1; if(type&1){ x>>=1 while(x--) a[i++] = !!(type&2) a[i++] = !(type&2); }else{ while(x != 1){ a[i++] = x&1; x>>=1; } } return i
Warning: `count' unused parameter. :) Maybe you meant to put the loop here?
and then use this for both the index (we aleady do ...) and info packets
info_packet for(i=0; i<stream_count; ) i= decompress_element(streams, i, stream_count)
I dislike it. For this purpose, a pure bitmask with special case 0 is good enough... It's not quite a complication but it is simply not worth it... (bitmask has the slight disadvantage of overflowing, but seriously, 64 streams... Worst pathological scenario I can think of is 30 streams, 20 subtitles and 10 audio tracks)
a limitation to 30-60 streams is not acceptable
just store the f* table in the main header :)
Bleh. :) It's a nice solution, but it's somewhat bad in that it adds info stuff in essential, small, main header (slightly defeating the purpose). Also it makes it harder for us to enforce our premade headers... (utf-8, etc.)
we cant enforce that anyway, just becaue it has utf-8 as type doensnt mean it is utf8, if someone decides to ignore a rule in the spec which says only utf-8 allowed, then he will care little about the type being enforced to utf-8 actually later is worse as its undetectable, so IMHO better give "them" an option to specify that they violate the rules instead of making it impossible to specify the type while obviously not preventing them from storing things in their native encoding ...
That's somewhat a good point, but still not excuse enough IMO to put the info types in the main header, it literally defeats the purpose - info stuff goes in info packets, not in main headers...
then my vote is to simply not allow extending the table, its not needed anyway to add new fields to info packets you either have a fixed table or you store the table in the file [...] -- Michael
On Thu, Feb 16, 2006 at 12:27:22AM +0100, Michael Niedermayer wrote:
Hi
On Wed, Feb 15, 2006 at 08:05:19PM +0200, Oded Shimon wrote:
On Wed, Feb 15, 2006 at 05:46:37PM +0100, Michael Niedermayer wrote:
On Wed, Feb 15, 2006 at 05:14:25PM +0200, Oded Shimon wrote:
This may be a question of opinion, but IMO "producor/author/title" etc. still apply to all streams/entire file, and then maybe you have a packet applied to the subtitle stream. Infact, "director" makes no sense to apply to any stream, it's not a video thing or audio thing, it's a movie (as in, entire file) thing. But I do see your point, do you have a suggestion for how to code it? (in stream, using vlc's. not in C code :) Maybe a bitmask? 0 meaning all streams?
hmmmmmm, what about
decompress_element(a, i, count) type v x= type>>1; if(type&1){ x>>=1 while(x--) a[i++] = !!(type&2) a[i++] = !(type&2); }else{ while(x != 1){ a[i++] = x&1; x>>=1; } } return i
Warning: `count' unused parameter. :) Maybe you meant to put the loop here?
and then use this for both the index (we aleady do ...) and info packets
info_packet for(i=0; i<stream_count; ) i= decompress_element(streams, i, stream_count)
I dislike it. For this purpose, a pure bitmask with special case 0 is good enough... It's not quite a complication but it is simply not worth it... (bitmask has the slight disadvantage of overflowing, but seriously, 64 streams... Worst pathological scenario I can think of is 30 streams, 20 subtitles and 10 audio tracks)
a limitation to 30-60 streams is not acceptable
It's just a implementation limitation, not a spec one, but anyway, see new patch.
just store the f* table in the main header :)
Bleh. :) It's a nice solution, but it's somewhat bad in that it adds info stuff in essential, small, main header (slightly defeating the purpose). Also it makes it harder for us to enforce our premade headers... (utf-8, etc.)
we cant enforce that anyway, just becaue it has utf-8 as type doensnt mean it is utf8, if someone decides to ignore a rule in the spec which says only utf-8 allowed, then he will care little about the type being enforced to utf-8 actually later is worse as its undetectable, so IMHO better give "them" an option to specify that they violate the rules instead of making it impossible to specify the type while obviously not preventing them from storing things in their native encoding ...
That's somewhat a good point, but still not excuse enough IMO to put the info types in the main header, it literally defeats the purpose - info stuff goes in info packets, not in main headers...
then my vote is to simply not allow extending the table, its not needed anyway to add new fields to info packets
you either have a fixed table or you store the table in the file
Is the problem old programs not understanding new fields? It's not a big issue IMO... Main difference in this patch is "type" is only for binary data. I couldn't decide an elegant way to design the table to accomodate this though... - ods15
On Thu, Feb 16, 2006 at 08:01:38AM +0200, Oded Shimon wrote:
I dislike it. For this purpose, a pure bitmask with special case 0 is good enough... It's not quite a complication but it is simply not worth it... (bitmask has the slight disadvantage of overflowing, but seriously, 64 streams... Worst pathological scenario I can think of is 30 streams, 20 subtitles and 10 audio tracks)
a limitation to 30-60 streams is not acceptable
It's just a implementation limitation, not a spec one, but anyway, see new patch.
Agree. It's easier to have a custom v-reader/writer for arbitrary length bitflags than to have some huge mechanism to use ordinary read to 64bit integer types for this purpose.
That's somewhat a good point, but still not excuse enough IMO to put the info types in the main header, it literally defeats the purpose - info stuff goes in info packets, not in main headers...
then my vote is to simply not allow extending the table, its not needed anyway to add new fields to info packets
you either have a fixed table or you store the table in the file
Is the problem old programs not understanding new fields? It's not a big issue IMO...
Yes, I thought it was an issue before, but then realized it's probably not. In any case I'm against forbidding this in the design, in case we decide later that there are important fields we want to add.
Main difference in this patch is "type" is only for binary data. I couldn't decide an elegant way to design the table to accomodate this though...
Design of the table is an _implementation_ issue.
[Note: strings MUST be encoded in UTF-8] + [Note: strings MUST NOT be encoded with their terminating NUL]
IMO this should read "the character NUL (U+0000) is not legal within or at the end of a string."
+ code= id&3 + index= id>>2 + name= info_table[code][index][0] + if(name==NULL) name vb + if(code==0) { value s - else + } else if(code==1) { + nom s + denom v + value= nom/denom + } else if(code==2) { + mantissa s + exponent s + value= mantissa*pow(2,exp) + } else { + if(index) type= info_table[code][index][1] + else type vb value vb + }
OK IMO.
+info_table[4][][2]={ + { + {NULL , "i"}, // integer + }, + { + {NULL , "r"}, // rational + {"TrackTime" , "r"}, + }, + { + {NULL , "f"}, // float + },
IMO this part is silly/misleading, since "i", "r", and "f" are not "types".
+ { + {NULL , NULL }, + {NULL , "UTF8"}, + {"Author" , "UTF8"}, + {"Description" , "UTF8"}, + {"Copyright" , "UTF8"}, + {"Encoder" , "UTF8"}, + {"Title" , "UTF8"}, + {"Cover" , "JPEG"}, + {"Cover" , "PNG"}, + {"Source" , "UTF8"}, + {"CaptureDevice" , "UTF8"}, + {"CreationTime" , "UTF8"}, + {"Keywords" , "UTF8"}, + {"Language" , "UTF8"}, + {"Disposition" , "UTF8"}, + } }; + Note: No future entries will have NULL as a value.
Two comments: 1. Is it really necessary to have entry 1 ({NULL, "UTF8"}) ? It would be simpler if type were always coded when a custom name is. 2. Maybe this is stupid/pedantic, but the correct name is UTF-8, not UTF8. Is it worth wasting a byte to correct this? Rich
On Thu, Feb 16, 2006 at 01:29:32AM -0500, Rich Felker wrote:
On Thu, Feb 16, 2006 at 08:01:38AM +0200, Oded Shimon wrote:
Main difference in this patch is "type" is only for binary data. I couldn't decide an elegant way to design the table to accomodate this though...
Design of the table is an _implementation_ issue.
I meant the design just here in the spec, you said it yourself, it is misleading that the int types have a "type".
+info_table[4][][2]={ + { + {NULL , "i"}, // integer + }, + { + {NULL , "r"}, // rational + {"TrackTime" , "r"}, + }, + { + {NULL , "f"}, // float + },
IMO this part is silly/misleading, since "i", "r", and "f" are not "types".
exactly what I meant....
+ { + {NULL , NULL }, + {NULL , "UTF8"}, + {"Author" , "UTF8"}, + {"Description" , "UTF8"}, + {"Copyright" , "UTF8"}, + {"Encoder" , "UTF8"}, + {"Title" , "UTF8"}, + {"Cover" , "JPEG"}, + {"Cover" , "PNG"}, + {"Source" , "UTF8"}, + {"CaptureDevice" , "UTF8"}, + {"CreationTime" , "UTF8"}, + {"Keywords" , "UTF8"}, + {"Language" , "UTF8"}, + {"Disposition" , "UTF8"}, + } }; + Note: No future entries will have NULL as a value.
Two comments:
1. Is it really necessary to have entry 1 ({NULL, "UTF8"}) ? It would be simpler if type were always coded when a custom name is.
I think it is worth it...
2. Maybe this is stupid/pedantic, but the correct name is UTF-8, not UTF8. Is it worth wasting a byte to correct this?
It's not wasting a byte because we have entry 1... I still have one major issue left with info packets - chapters... We need to decide a sane way to do them and say so in the spec... But that's after we all agree on this patch. Does anyone have objections left... - ods15
Hi On Sat, Feb 18, 2006 at 10:02:48AM +0200, Oded Shimon wrote: [...]
I still have one major issue left with info packets - chapters... We need to decide a sane way to do them and say so in the spec... But that's after we all agree on this patch. Does anyone have objections left...
i am against it, my oppinion is either store the table in the file or make it constant, or propose to drop "extendible" from the goals first [...] -- Michael
On Sat, Feb 18, 2006 at 12:28:59PM +0100, Michael Niedermayer wrote:
Hi
On Sat, Feb 18, 2006 at 10:02:48AM +0200, Oded Shimon wrote: [...]
I still have one major issue left with info packets - chapters... We need to decide a sane way to do them and say so in the spec... But that's after we all agree on this patch. Does anyone have objections left...
i am against it, my oppinion is either store the table in the file or make it constant, or propose to drop "extendible" from the goals first
:/ Putting the tables in the main header is very very weird. It doesn't belong there, it makes it more difficult to mux because even for info streams you have to know all your fields before hand (or, i guess you always have the option of using the fields directly in the info packets), it's also tricky when combining 2 nut files or something... Locking the table is the opposite of extendiblity, it prevents us from ever improoving... Though admittedly there is some non-extendiblity in growing the tables as well, as it "breaks" old demuxers. But IMO no - ALL reserved fields everywhere in NUT "break" old demuxers in that old demuxers do not understand them - the same goes for new info table entries, they only break in that they fail to understand themselves, but still keep everything else intact. - ods15
On Sat, Feb 18, 2006 at 01:57:25PM +0200, Oded Shimon wrote:
On Sat, Feb 18, 2006 at 12:28:59PM +0100, Michael Niedermayer wrote:
Hi
On Sat, Feb 18, 2006 at 10:02:48AM +0200, Oded Shimon wrote: [...]
I still have one major issue left with info packets - chapters... We need to decide a sane way to do them and say so in the spec... But that's after we all agree on this patch. Does anyone have objections left...
i am against it, my oppinion is either store the table in the file or make it constant, or propose to drop "extendible" from the goals first
:/
Putting the tables in the main header is very very weird. It doesn't belong there, it makes it more difficult to mux because even for info streams you have to know all your fields before hand (or, i guess you always have the option of using the fields directly in the info packets), it's also tricky when combining 2 nut files or something...
Locking the table is the opposite of extendiblity, it prevents us from ever improoving... Though admittedly there is some non-extendiblity in growing the tables as well, as it "breaks" old demuxers. But IMO no - ALL reserved fields everywhere in NUT "break" old demuxers in that old demuxers do not understand them - the same goes for new info table entries, they only break in that they fail to understand themselves, but still keep everything else intact.
BTW, following your philosophy - the info tables in the main header can triple the size of the main header (or even more), and a single byte of damage in this non essential info would break the entire main header and make the file unplayable... And no, making another packet type just for info tables is a bad idea. :/ - ods15
Hi On Sat, Feb 18, 2006 at 02:19:00PM +0200, Oded Shimon wrote:
On Sat, Feb 18, 2006 at 01:57:25PM +0200, Oded Shimon wrote:
On Sat, Feb 18, 2006 at 12:28:59PM +0100, Michael Niedermayer wrote:
Hi
On Sat, Feb 18, 2006 at 10:02:48AM +0200, Oded Shimon wrote: [...]
I still have one major issue left with info packets - chapters... We need to decide a sane way to do them and say so in the spec... But that's after we all agree on this patch. Does anyone have objections left...
i am against it, my oppinion is either store the table in the file or make it constant, or propose to drop "extendible" from the goals first
:/
Putting the tables in the main header is very very weird. It doesn't belong
thats my favorite reasoning why something is bad (its weird, philosophical wrong, not unixish, ...) btw, you can also put them in an info packet of course if you prefer ...
there, it makes it more difficult to mux because even for info streams you have to know all your fields before hand (or, i guess you always have the option of using the fields directly in the info packets), it's also tricky when combining 2 nut files or something...
huh what?! a muxer would probably have a fixed table which it writes ... _this_ is the equivalent to your suggestion, not designing a optimal table which isnt needed at all and has negligable gain the info packets are string name + type + value triplets, the table is a optimization to reduce the size, i vote for droping the table completely way before i agree to allowing changing the table without storing it in the file, the table was never intended to be changable, if you insist on changing it it must be stored in the file [...]
BTW, following your philosophy - the info tables in the main header can triple the size of the main header (or even more), and a single byte of damage in this non essential info would break the entire main header and make the file unplayable...
you do remember that nut files have a minimum of 3 duplicate main headers? [...] -- Michael
On Sat, Feb 18, 2006 at 03:14:44PM +0100, Michael Niedermayer wrote:
Hi
On Sat, Feb 18, 2006 at 02:19:00PM +0200, Oded Shimon wrote:
On Sat, Feb 18, 2006 at 01:57:25PM +0200, Oded Shimon wrote:
On Sat, Feb 18, 2006 at 12:28:59PM +0100, Michael Niedermayer wrote:
Hi
On Sat, Feb 18, 2006 at 10:02:48AM +0200, Oded Shimon wrote: [...]
I still have one major issue left with info packets - chapters... We need to decide a sane way to do them and say so in the spec... But that's after we all agree on this patch. Does anyone have objections left...
i am against it, my oppinion is either store the table in the file or make it constant, or propose to drop "extendible" from the goals first
:/
Putting the tables in the main header is very very weird. It doesn't belong
thats my favorite reasoning why something is bad (its weird, philosophical wrong, not unixish, ...)
Well, it is a reason... Things should be logical/sense-making...
btw, you can also put them in an info packet of course if you prefer ...
This seems to be the only solution I just barely like...
there, it makes it more difficult to mux because even for info streams you have to know all your fields before hand (or, i guess you always have the option of using the fields directly in the info packets), it's also tricky when combining 2 nut files or something...
huh what?! a muxer would probably have a fixed table which it writes ... _this_ is the equivalent to your suggestion, not designing a optimal table which isnt needed at all and has negligable gain
the info packets are string name + type + value triplets, the table is a optimization to reduce the size, i vote for droping the table completely way before i agree to allowing changing the table without storing it in the file, the table was never intended to be changable, if you insist on changing it it must be stored in the file
Many containers have ID-to-metadata mappings, and they are fairly static. If we have found a way to do this while still being able to, in the far future, add new fields, then why not... I think the only gripe I have with writing the table in some header, is, besides the complexity, the fact that we don't have a pre-made table with common fields and prepared with "UTF-8" and etc. Id's are more consistent than names (some muxer might goof up and use Titel instead of Title, and then demuxers won't understand it...).
BTW, following your philosophy - the info tables in the main header can triple the size of the main header (or even more), and a single byte of damage in this non essential info would break the entire main header and make the file unplayable...
you do remember that nut files have a minimum of 3 duplicate main headers?
I'm downloading the file with bittorrent, I only have one copy of the main headers, and because of its size it just ended up being on a block boundry.. I'm surprised, you were very high strong about all other packets '1 byte should cause minimal data loss, even in essential repeated headers', which is why we still have stream_id in stream headers.. - ods15
On Sat, Feb 18, 2006 at 04:34:35PM +0200, Oded Shimon wrote:
I think the only gripe I have with writing the table in some header, is, besides the complexity, the fact that we don't have a pre-made table with common fields and prepared with "UTF-8" and etc. Id's are more consistent than names (some muxer might goof up and use Titel instead of Title, and then demuxers won't understand it...).
Nutlint will catch it. :) IIRC, all nonstandard names are required to start with X- Rich
On Sat, Feb 18, 2006 at 12:56:24PM -0500, Rich Felker wrote:
On Sat, Feb 18, 2006 at 04:34:35PM +0200, Oded Shimon wrote:
I think the only gripe I have with writing the table in some header, is, besides the complexity, the fact that we don't have a pre-made table with common fields and prepared with "UTF-8" and etc. Id's are more consistent than names (some muxer might goof up and use Titel instead of Title, and then demuxers won't understand it...).
Nutlint will catch it. :) IIRC, all nonstandard names are required to start with X-
Just "should" (not even in caps). That's enough for a warning though... BTW, Matroska have a rediculous huge amount of tags... I'm wondering how many standard tags we should have in NUT. - ods15
Hi On Sat, Feb 18, 2006 at 04:34:35PM +0200, Oded Shimon wrote: [...]
there, it makes it more difficult to mux because even for info streams you have to know all your fields before hand (or, i guess you always have the option of using the fields directly in the info packets), it's also tricky when combining 2 nut files or something...
huh what?! a muxer would probably have a fixed table which it writes ... _this_ is the equivalent to your suggestion, not designing a optimal table which isnt needed at all and has negligable gain
the info packets are string name + type + value triplets, the table is a optimization to reduce the size, i vote for droping the table completely way before i agree to allowing changing the table without storing it in the file, the table was never intended to be changable, if you insist on changing it it must be stored in the file
Many containers have ID-to-metadata mappings, and they are fairly static. If we have found a way to do this while still being able to, in the far future, add new fields, then why not...
I think the only gripe I have with writing the table in some header, is, besides the complexity, the fact that we don't have a pre-made table with common fields and prepared with "UTF-8" and etc. Id's are more consistent than names (some muxer might goof up and use Titel instead of Title, and then demuxers won't understand it...).
right, now if only they made the C standard use random ids instead of english words the world would be a better place ... and please keep in mind C is written by people who make mistakes, nut files are written by muxers which are less likely to mess up, and such messup is detectable and last thing, why exactly should title vs. titel be a more likely messup then 5 vs. 6 ?! not to mention later is not detectable while a unknown name with no X- infront is detectable
BTW, following your philosophy - the info tables in the main header can triple the size of the main header (or even more), and a single byte of damage in this non essential info would break the entire main header and make the file unplayable...
you do remember that nut files have a minimum of 3 duplicate main headers?
I'm downloading the file with bittorrent, I only have one copy of the main headers, and because of its size it just ended up being on a block boundry..
in that case i suspect your solution will put one stream header at that block bouandary
I'm surprised, you were very high strong about all other packets '1 byte should cause minimal data loss, even in essential repeated headers', which is why we still have stream_id in stream headers..
index packets are 100kb while the mainheader is <1kb, the mainheader must be there at least 3 times the index in most cases will be in the file just once [...] -- Michael
On Sun, Feb 19, 2006 at 06:40:47AM +0200, Oded Shimon wrote:
On Sun, Feb 19, 2006 at 01:08:41AM +0100, Michael Niedermayer wrote:
[...]
Rich and I vote for throwing the tables alltogether, they are unnecessary complexity in reducing size of already redundant data... Michael, what's your vote on this?
Well this is my preference. In the absence of that, I'm willing to have a small list of essential key names, but I'd strongly prefer only including the essential, universal ones if the list will be locked forever, rather than arbitrarily including whatever we think of right now.. Rich
Hi On Sun, Feb 19, 2006 at 02:17:13AM -0500, Rich Felker wrote:
On Sun, Feb 19, 2006 at 06:40:47AM +0200, Oded Shimon wrote:
On Sun, Feb 19, 2006 at 01:08:41AM +0100, Michael Niedermayer wrote:
[...]
Rich and I vote for throwing the tables alltogether, they are unnecessary complexity in reducing size of already redundant data... Michael, what's your vote on this?
Well this is my preference. In the absence of that, I'm willing to have a small list of essential key names, but I'd strongly prefer only including the essential, universal ones if the list will be locked forever, rather than arbitrarily including whatever we think of right now..
i agree with removing the tables completely [...] -- Michael
On Sat, Feb 18, 2006 at 03:14:44PM +0100, Michael Niedermayer wrote:
Hi
On Sat, Feb 18, 2006 at 02:19:00PM +0200, Oded Shimon wrote:
On Sat, Feb 18, 2006 at 01:57:25PM +0200, Oded Shimon wrote:
On Sat, Feb 18, 2006 at 12:28:59PM +0100, Michael Niedermayer wrote:
Hi
On Sat, Feb 18, 2006 at 10:02:48AM +0200, Oded Shimon wrote: [...]
I still have one major issue left with info packets - chapters... We need to decide a sane way to do them and say so in the spec... But that's after we all agree on this patch. Does anyone have objections left...
i am against it, my oppinion is either store the table in the file or make it constant, or propose to drop "extendible" from the goals first
:/
Putting the tables in the main header is very very weird. It doesn't belong
thats my favorite reasoning why something is bad (its weird, philosophical wrong, not unixish, ...)
btw, you can also put them in an info packet of course if you prefer ...
If you prefer, weird means it adds to the number of data structures a demuxer must maintain, along with seeming unnatural.
the info packets are string name + type + value triplets, the table is a optimization to reduce the size, i vote for droping the table completely way before i agree to allowing changing the table without storing it in the file, the table was never intended to be changable, if you insist on changing it it must be stored in the file
Again we don't insist on changing it any more than we insist on adding new nonsense fields to the main headers. But it should still be possible anyway unless it turns out to be needed (e.g. some really important new tag people want when NUT 2.0 comes around and it's wasting lots of space by storing the name over and over..) If you insist on extensions being in the global headers, I'd prefer that we remove the table from the spec entirely and put all codes in the global headers. At least then it's consistent. The spec can still list the names/types of standard info items, and we can expand this list without breaking anything.. Rich
Hi On Thu, Feb 16, 2006 at 08:01:38AM +0200, Oded Shimon wrote: [...]
just store the f* table in the main header :)
Bleh. :) It's a nice solution, but it's somewhat bad in that it adds info stuff in essential, small, main header (slightly defeating the purpose). Also it makes it harder for us to enforce our premade headers... (utf-8, etc.)
we cant enforce that anyway, just becaue it has utf-8 as type doensnt mean it is utf8, if someone decides to ignore a rule in the spec which says only utf-8 allowed, then he will care little about the type being enforced to utf-8 actually later is worse as its undetectable, so IMHO better give "them" an option to specify that they violate the rules instead of making it impossible to specify the type while obviously not preventing them from storing things in their native encoding ...
That's somewhat a good point, but still not excuse enough IMO to put the info types in the main header, it literally defeats the purpose - info stuff goes in info packets, not in main headers...
then my vote is to simply not allow extending the table, its not needed anyway to add new fields to info packets
you either have a fixed table or you store the table in the file
Is the problem old programs not understanding new fields? It's not a big issue IMO...
if the id->string table is in the mainheader, every demuxer new and old will support it, if its not in the main header and we dont force muxers to use litteral strings as id then the values are not useable by old demuxers and there simply is no need to be able to parse them, just skip the rest of the info packet and require muxers to order fields by "age" also note that such unknown metadata would be not transcodeable to other containers, and even nut->nut might be hard over some (de)muxer APIs also keep in mind not everone will update their sw every month to support the new info types [...] -- Michael
On Thu, Feb 16, 2006 at 04:28:44PM +0100, Michael Niedermayer wrote:
Hi
On Thu, Feb 16, 2006 at 08:01:38AM +0200, Oded Shimon wrote: [...]
> just store the f* table in the main header :)
Bleh. :) It's a nice solution, but it's somewhat bad in that it adds info stuff in essential, small, main header (slightly defeating the purpose). Also it makes it harder for us to enforce our premade headers... (utf-8, etc.)
we cant enforce that anyway, just becaue it has utf-8 as type doensnt mean it is utf8, if someone decides to ignore a rule in the spec which says only utf-8 allowed, then he will care little about the type being enforced to utf-8 actually later is worse as its undetectable, so IMHO better give "them" an option to specify that they violate the rules instead of making it impossible to specify the type while obviously not preventing them from storing things in their native encoding ...
That's somewhat a good point, but still not excuse enough IMO to put the info types in the main header, it literally defeats the purpose - info stuff goes in info packets, not in main headers...
then my vote is to simply not allow extending the table, its not needed anyway to add new fields to info packets
you either have a fixed table or you store the table in the file
Is the problem old programs not understanding new fields? It's not a big issue IMO...
if the id->string table is in the mainheader, every demuxer new and old will support it, if its not in the main header and we dont force muxers to use litteral strings as id then the values are not useable by old demuxers and there simply is no need to be able to parse them, just skip the rest of the info packet and require muxers to order fields by "age"
I'm against additional requirements like this when we already have a good solution whereby all keys are parsable even if their contents are unknown.
also note that such unknown metadata would be not transcodeable to other containers, and even nut->nut might be hard over some (de)muxer APIs
also keep in mind not everone will update their sw every month to support the new info types
As I said this is not an "every month" thing. It's just in case we eventually find a need for new types. Aside from a few important ones, info is always an "optional" thing. It's metadata, not part of the content itself, and it's no big tragedy if an old demuxer can't demux part of it. (In fact there's probably no use to having an old demuxer support it.) Rich
participants (3)
-
Michael Niedermayer -
Oded Shimon -
Rich Felker