Author: michael Date: Sat Jul 15 23:49:47 2006 New Revision: 19110 Modified: trunk/DOCS/tech/nut.txt Log: clarify "extradata" comments welcome! if anyone disagrees ill reverse this and we can disscuss it, i just thought there wont be any disagreement and was too lazy to send a patch ... Modified: trunk/DOCS/tech/nut.txt ============================================================================== --- trunk/DOCS/tech/nut.txt (original) +++ trunk/DOCS/tech/nut.txt Sat Jul 15 23:49:47 2006 @@ -512,6 +512,19 @@ codec_specific_data private global data for a codec (could be huffman tables or ...) + if a codec has a global header it SHOULD be placed in here instead of + at the start of every keyframe + the exact format is specified in the codec spec + codecs which dont specify it in their spec are specified below + for ogg based codecs (vorbis, theora) the following format shall be used + number_of_headers_minus_1 u(8) + for(i=0; i<number_of_headers; i++){ + -1 u(8*(size[i]/255)) + size[i] % 255 u(8) + } + for(i=0; i<number_of_headers; i++) + header[i] + Note, this is the same format these codecs use in matroska frame_code the meaning of this byte is stored in the main header
michael <subversion@mplayerhq.hu> writes:
Author: michael Date: Sat Jul 15 23:49:47 2006 New Revision: 19110
Modified: trunk/DOCS/tech/nut.txt
Log: clarify "extradata" comments welcome! if anyone disagrees ill reverse this and we can disscuss it, i just thought there wont be any disagreement and was too lazy to send a patch ...
Modified: trunk/DOCS/tech/nut.txt ============================================================================== --- trunk/DOCS/tech/nut.txt (original) +++ trunk/DOCS/tech/nut.txt Sat Jul 15 23:49:47 2006 @@ -512,6 +512,19 @@
codec_specific_data private global data for a codec (could be huffman tables or ...) + if a codec has a global header it SHOULD be placed in here instead of + at the start of every keyframe + the exact format is specified in the codec spec + codecs which dont specify it in their spec are specified below + for ogg based codecs (vorbis, theora) the following format shall be used + number_of_headers_minus_1 u(8) + for(i=0; i<number_of_headers; i++){ + -1 u(8*(size[i]/255)) + size[i] % 255 u(8) + } + for(i=0; i<number_of_headers; i++) + header[i] + Note, this is the same format these codecs use in matroska
If I didn't already know how what that format looks like, I'd have a hard time deciphering that description. Could we perhaps generalize that format to include all codecs with several chunks of extradata, not only ogg based codecs? I'm not aware of any others, but for all I know they might exist. -- Måns Rullgård mru@inprovide.com
Hi On Sat, Jul 15, 2006 at 11:02:52PM +0100, Måns Rullgård wrote:
michael <subversion@mplayerhq.hu> writes:
Author: michael Date: Sat Jul 15 23:49:47 2006 New Revision: 19110
Modified: trunk/DOCS/tech/nut.txt
Log: clarify "extradata" comments welcome! if anyone disagrees ill reverse this and we can disscuss it, i just thought there wont be any disagreement and was too lazy to send a patch ...
Modified: trunk/DOCS/tech/nut.txt ============================================================================== --- trunk/DOCS/tech/nut.txt (original) +++ trunk/DOCS/tech/nut.txt Sat Jul 15 23:49:47 2006 @@ -512,6 +512,19 @@
codec_specific_data private global data for a codec (could be huffman tables or ...) + if a codec has a global header it SHOULD be placed in here instead of + at the start of every keyframe + the exact format is specified in the codec spec + codecs which dont specify it in their spec are specified below + for ogg based codecs (vorbis, theora) the following format shall be used + number_of_headers_minus_1 u(8) + for(i=0; i<number_of_headers; i++){ + -1 u(8*(size[i]/255)) + size[i] % 255 u(8) + } + for(i=0; i<number_of_headers; i++) + header[i] + Note, this is the same format these codecs use in matroska
If I didn't already know how what that format looks like, I'd have a hard time deciphering that description.
feel free to write a less obfuscated one ...
Could we perhaps generalize that format to include all codecs with several chunks of extradata, not only ogg based codecs? I'm not aware of any others, but for all I know they might exist.
ok, but the following needs disscussion: A always choose ogg style lacing B always choose v + packet data (nut style) C use whatever matroska uses + one of the above for the cases not supported by matroska and there are many codecs which have several chunks all the mpeg variants do, but they have startcodes so its a non issue h.264 does too but with, the startcode prefixes being optional, IMHO we should just require the startcode prefixes to be in there ... then there are some mov based codecs like qdm2 which maybe need several chunks, they currently simply copy a whole set of mov chunks blindly into extradata in ffmpeg, that of course is a terrible mess which someone should fix in ffmpeg ... [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is
2006/7/16, Michael Niedermayer <michaelni@gmx.at>:
Hi
On Sat, Jul 15, 2006 at 11:02:52PM +0100, Måns Rullgård wrote:
michael <subversion@mplayerhq.hu> writes:
Author: michael Date: Sat Jul 15 23:49:47 2006 New Revision: 19110
Modified: trunk/DOCS/tech/nut.txt
Log: clarify "extradata" comments welcome! if anyone disagrees ill reverse this and we can disscuss it, i just thought there wont be any disagreement and was too lazy to send a patch ...
Modified: trunk/DOCS/tech/nut.txt ============================================================================== --- trunk/DOCS/tech/nut.txt (original) +++ trunk/DOCS/tech/nut.txt Sat Jul 15 23:49:47 2006 @@ -512,6 +512,19 @@
codec_specific_data private global data for a codec (could be huffman tables or ...) + if a codec has a global header it SHOULD be placed in here instead of + at the start of every keyframe + the exact format is specified in the codec spec + codecs which dont specify it in their spec are specified below + for ogg based codecs (vorbis, theora) the following format shall be used + number_of_headers_minus_1 u(8) + for(i=0; i<number_of_headers; i++){ + -1 u(8*(size[i]/255)) + size[i] % 255 u(8) + } + for(i=0; i<number_of_headers; i++) + header[i] + Note, this is the same format these codecs use in matroska
If I didn't already know how what that format looks like, I'd have a hard time deciphering that description.
I still have no idea what it should do. At least header[i] is not associated with any type. My guess b(size[i]) If I had decoded the above correctly, then 65536 bytes of header would have about 257 bytes of 0xFF.... It's not acceptable to have that thing in nut. We have our own much better vlc system.
feel free to write a less obfuscated one ...
Could we perhaps generalize that format to include all codecs with several chunks of extradata, not only ogg based codecs? I'm not aware of any others, but for all I know they might exist.
ok, but the following needs disscussion: A always choose ogg style lacing B always choose v + packet data (nut style) C use whatever matroska uses + one of the above for the cases not supported by matroska
I don't think there is place for discussion. Always use nut style.
and there are many codecs which have several chunks all the mpeg variants do, but they have startcodes so its a non issue h.264 does too but with, the startcode prefixes being optional, IMHO we should just require the startcode prefixes to be in there ... then there are some mov based codecs like qdm2 which maybe need several chunks, they currently simply copy a whole set of mov chunks blindly into extradata in ffmpeg, that of course is a terrible mess which someone should fix in ffmpeg ...
How about one of the following: num_headers v for(i=0;i<num_headers;i++){ header[i] vb } --- or --- for(i=0; show_v() != 0; i++){ header[i] vb } 0x00 (aka vb with size 0); --- or --- (inside the codec_specific_data.value) for(i=0; i+show_v() <= codec_specific_data.length; i+= show_v() ){ header[i] vb } #2 is sensitive to data corruption, #3 could interpret older header in wrong way. #1 changes the frozen bitstream (hope nobody looks ;) I'd say KISS and use #1.
On Sun, Jul 16, 2006 at 03:43:11PM +0300, Ivan Kalvachev wrote:
Could we perhaps generalize that format to include all codecs with several chunks of extradata, not only ogg based codecs? I'm not aware of any others, but for all I know they might exist.
ok, but the following needs disscussion: A always choose ogg style lacing B always choose v + packet data (nut style) C use whatever matroska uses + one of the above for the cases not supported by matroska
I don't think there is place for discussion. Always use nut style.
No, read my post about the intent... The demuxer has nothing to do with parsing this information. It's part of the codec layer. If it were part of the demuxer layer, we would have a whole huge bloated elaborate "multi-packet-extradata-database" layer in NUT which is obviously an abomination. Rich
2006/7/17, Rich Felker <dalias@aerifal.cx>:
On Sun, Jul 16, 2006 at 03:43:11PM +0300, Ivan Kalvachev wrote:
Could we perhaps generalize that format to include all codecs with several chunks of extradata, not only ogg based codecs? I'm not aware of any others, but for all I know they might exist.
ok, but the following needs disscussion: A always choose ogg style lacing B always choose v + packet data (nut style) C use whatever matroska uses + one of the above for the cases not supported by matroska
I don't think there is place for discussion. Always use nut style.
No, read my post about the intent... The demuxer has nothing to do with parsing this information. It's part of the codec layer. If it were part of the demuxer layer, we would
If that was true then what is this commit doing in nut.txt? What the demuxer does is parsing information. If the codec expect 3 headers it should get 3 headers. If it expect 1 atom, it should get 1 atom.
have a whole huge bloated elaborate "multi-packet-extradata-database" layer in NUT which is obviously an abomination.
It is abomination to set hacks both in codec layer and nut de/muxer to do what could be done directly. If libavcodec expect extradata to be in mkv form then it's libavcodec problem, not nut one. Michael, You say that if somebody objects you will revert this. I don't think that agreement would need 22 mails in comments. So please revert and get it back to drawing board.
On Mon, Jul 17, 2006 at 10:39:08PM +0300, Ivan Kalvachev wrote:
2006/7/17, Rich Felker <dalias@aerifal.cx>:
On Sun, Jul 16, 2006 at 03:43:11PM +0300, Ivan Kalvachev wrote:
Could we perhaps generalize that format to include all codecs with several chunks of extradata, not only ogg based codecs? I'm not aware of any others, but for all I know they might exist.
ok, but the following needs disscussion: A always choose ogg style lacing B always choose v + packet data (nut style) C use whatever matroska uses + one of the above for the cases not supported by matroska
I don't think there is place for discussion. Always use nut style.
No, read my post about the intent... The demuxer has nothing to do with parsing this information. It's part of the codec layer. If it were part of the demuxer layer, we would
If that was true then what is this commit doing in nut.txt? [..]
Michael, You say that if somebody objects you will revert this. I don't think that agreement would need 22 mails in comments. So please revert and get it back to drawing board.
I say just revert but make a codec documents giving details about several codecs in nut where the wrapping isn't obvious (like vorbis). Ivan, vorbis doesn't expect 3 extradata according to the spec, it expect the first 3 audio packet to be the headers. I think this is insane enough that it allows us to break their spec instead of ours. - ods15
Hi Oded Shimon wrote:
[...]
Ivan, vorbis doesn't expect 3 extradata according to the spec, it expect the first 3 audio packet to be the headers. I think this is insane enough that it allows us to break their spec instead of ours.
Huh. How come a container can surpasses codec specs ? IMHO that's not acceptable. -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA SMARTJOG S.A. http://www.smartjog.com Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA Phone: +33 1 49966312
Baptiste Coudurier <baptiste.coudurier@smartjog.com> writes:
Hi
Oded Shimon wrote:
[...]
Ivan, vorbis doesn't expect 3 extradata according to the spec, it expect the first 3 audio packet to be the headers. I think this is insane enough that it allows us to break their spec instead of ours.
The vorbis spec states that when stored in ogg, the first three packets are headers. These three packets are passed to libvorbis using a function (vorbis_synthesis_headerin()) different from the one used for the remaining packets (vorbis_synthesis()). The vorbis spec itself treats these headers quite distinctly from the main bitstream. It was a bad move of the vorbis designers to choose three separate header formats, seeing as many container formats allow for one piece of codec extradata, and ogg doesn't have any dedicated space at all for this. Packing the three pieces of data into one would have made no difference for ogg, and would have made storing vorbis in other containers easier. Actually, ogg processing would also be simpler if you knew that the first packet was header data, rather than the first N packets.
Huh. How come a container can surpasses codec specs ? IMHO that's not acceptable.
Indeed. The problem here is that ffmpeg, like most other apps, only handles passing one piece of extradata from the demuxer to the decoder (and from the encoder to the muxer). This is enough for most codecs and formats, but falls short when a codec uses several extradata blocks (like vorbis). It is also a problem when using these codecs in formats that only handle one extradata block. Now we have a chance, by specifying storage of multiple extradata blocks, to create a container that can store *any* codec without resorting to hacks outside the spec. Most apps will probably have to somehow pack these multiple blocks into a single block that is passed around inside the app, and unpack/repack it to suit whatever codec libs they pass it to. The advantage over storing this single block directly in the container file is that there is no need to agree among apps on the format to be used. Hypothetical apps that can handle multiple extradata pieces will need no hacks at all. I'll state my main point again. Allowing multiple extradata blocks in nut will allow any codec to be used with nut, *without* any need for agreements outside the codec specs. Apps that use some common format for packing extradata can continue to pack nut extradata into that format internally, so there will be no added complexity for those. -- Måns Rullgård mru@inprovide.com
On Tue, Jul 18, 2006 at 09:17:01AM +0100, Måns Rullgård wrote:
The problem here is that ffmpeg, like most other apps, only handles passing one piece of extradata from the demuxer to the decoder (and from the encoder to the muxer). This is enough for most codecs and formats, but falls short when a codec uses several extradata blocks (like vorbis). It is also a problem when using these codecs in formats that only handle one extradata block.
Now we have a chance, by specifying storage of multiple extradata blocks, to create a container that can store *any* codec without resorting to hacks outside the spec. Most apps will probably have to somehow pack these multiple blocks into a single block that is passed around inside the app, and unpack/repack it to suit whatever codec libs they pass it to.
The advantage over storing this single block directly in the container file is that there is no need to agree among apps on the format to be used. Hypothetical apps that can handle multiple extradata pieces will need no hacks at all.
I'll state my main point again. Allowing multiple extradata blocks in nut will allow any codec to be used with nut, *without* any need for agreements outside the codec specs. Apps that use some common format
No it will not. All Xiph has to do is make their new latest-and-greatest container have a hierarchical filesystem of header blocks, or a relational database of header blocks, or whatever other bloated crap they can come up with, and then we cannot store that without hacks! A codec having multiple header chunks without any in-band signalling of their boundaries is a _pathology_. It is not the norm and it is not something NUT should account for or that players should have to deal with. Rich
Rich Felker said:
On Tue, Jul 18, 2006 at 09:17:01AM +0100, Måns Rullgård wrote:
The problem here is that ffmpeg, like most other apps, only handles passing one piece of extradata from the demuxer to the decoder (and from the encoder to the muxer). This is enough for most codecs and formats, but falls short when a codec uses several extradata blocks (like vorbis). It is also a problem when using these codecs in formats that only handle one extradata block.
Now we have a chance, by specifying storage of multiple extradata blocks, to create a container that can store *any* codec without resorting to hacks outside the spec. Most apps will probably have to somehow pack these multiple blocks into a single block that is passed around inside the app, and unpack/repack it to suit whatever codec libs they pass it to.
The advantage over storing this single block directly in the container file is that there is no need to agree among apps on the format to be used. Hypothetical apps that can handle multiple extradata pieces will need no hacks at all.
I'll state my main point again. Allowing multiple extradata blocks in nut will allow any codec to be used with nut, *without* any need for agreements outside the codec specs. Apps that use some common format
No it will not. All Xiph has to do is make their new latest-and-greatest container have a hierarchical filesystem of header blocks, or a relational database of header blocks, or whatever other bloated crap they can come up with, and then we cannot store that without hacks!
They're not doing it now, and we have no reason to believe they ever will, so let's worry about that if it ever happens. They (and others) *do* specify multiple extradata blocks, which is something we can easily handle if we want to.
A codec having multiple header chunks without any in-band signalling of their boundaries is a _pathology_. It is not the norm and it is not something NUT should account for or that players should have to deal with.
Xiph does not rule the world, and neither do we. Pretending otherwise is a big mistake. Codecs with multiple blocks of extradata are a reality. We can either accept that reality, and deal with it (which is trivial), or we can all go and hide in your academic utopia and pretend everything is just the way we want it. -- Måns Rullgård mru@inprovide.com
On Tue, Jul 18, 2006 at 10:52:37AM +0100, Måns Rullgård wrote:
I'll state my main point again. Allowing multiple extradata blocks in nut will allow any codec to be used with nut, *without* any need for agreements outside the codec specs. Apps that use some common format
No it will not. All Xiph has to do is make their new latest-and-greatest container have a hierarchical filesystem of header blocks, or a relational database of header blocks, or whatever other bloated crap they can come up with, and then we cannot store that without hacks!
They're not doing it now, and we have no reason to believe they ever will, so let's worry about that if it ever happens.
No. "Let's worry about that if it ever happens" is the philosophy of a bad container. Surely _something_ that does not fit the "N consecutive packets" model will eventually happen in some codec. The question is a matter of being clean and simple. Do we support _one_ rare and already-ugly case for strange header data, or do we draw the line before that and say no, headers must be in the one universal format. (Binary data block is the minimal universal format that works for absolutely any codec.)
They (and others)
Which others? You have provided no examples. H.264 does not count because they already have startcodes separating the parts.
*do* specify multiple extradata blocks, which is something we can easily handle if we want to.
But we _don't_ want to. It makes things more difficult for implementations which will always want to take the simple, sane approach and pack everything into one binary block.
A codec having multiple header chunks without any in-band signalling of their boundaries is a _pathology_. It is not the norm and it is not something NUT should account for or that players should have to deal with.
Xiph does not rule the world, and neither do we. Pretending otherwise is a big mistake.
Codecs with multiple blocks of extradata are a reality. We can either accept that reality, and deal with it (which is trivial), or we can all go and hide in your academic utopia and pretend everything is just the way we want it.
Hiding something broken is always the correct solution. It has nothing to do with academic utopias but practical benefits. NUT is already complex enough and having to wrap the NUT demuxer with a header-repacker to get the headers into the sane, ALREADY STANDARDIZED single binary block format will just make it more annoying to use! Rich
On Tue, Jul 18, 2006 at 01:37:53PM -0400, Rich Felker wrote:
They (and others)
Which others? You have provided no examples. H.264 does not count because they already have startcodes separating the parts.
I'd like to add on top of that, that a NUT framework for multiple headers would greatly _complicate_ the question of how to store sane formats like H.264. Do you store them all as one block separated by the startcodes? Or in N different blocks? If storing them in N different blocks, do you keep the startcodes or strip them?? See how nasty this all gets? Just having one format, the single binary block with a spec for what goes in it, eliminates all the ambiguity and complexity. Rich
Hi On Tue, Jul 18, 2006 at 01:37:53PM -0400, Rich Felker wrote:
On Tue, Jul 18, 2006 at 10:52:37AM +0100, Måns Rullgård wrote:
I'll state my main point again. Allowing multiple extradata blocks in nut will allow any codec to be used with nut, *without* any need for agreements outside the codec specs. Apps that use some common format
No it will not. All Xiph has to do is make their new latest-and-greatest container have a hierarchical filesystem of header blocks, or a relational database of header blocks, or whatever other bloated crap they can come up with, and then we cannot store that without hacks!
They're not doing it now, and we have no reason to believe they ever will, so let's worry about that if it ever happens.
No. "Let's worry about that if it ever happens" is the philosophy of a bad container.
the alternative was mov, the container which supports everything the designers could think off, "sadly" the designers thoughts didnt became reality, and what did become reality wasnt supported (b frames for example) not to mention no mov demuxer or muxer supports even just half of the spec AFIAK
Surely _something_ that does not fit the "N consecutive packets" model will eventually happen in some codec. The question is a
this depends strongly on why multiple header packet but no standarized "serialization" codecs exist do they exist because the designers where unaware of the prblems this would cause or did they design it that way with the intention to make usage of their codec difficult in containers other then their own if later then yes no matter what you add, such people will do something to make storage hard now i dont know why xiph designed things like that, but i do know that they are _now_ aware of the issues (many people ask on their mailinglists about vorbis in non-ogg) and they dont specify how to turn the 3 packets into 1 ... so IMHO, if a codec uses multiple packets because the designers wherent aware of the issues this has with most containers and APIs then they will probably not mind adding a few lines to their spec to specify how to store these headers in a one-header API/container, and thouse who design codecs with multiple header packets to make storeage in "unwanted" containers hard will always find a way to reach that goal so IMHO supporting multiple header packets at nut level might solve vorbis & theora but its usefullness with future codecs is uncertain [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is
On Tue, Jul 18, 2006 at 10:54:04PM +0200, Michael Niedermayer wrote:
Surely _something_ that does not fit the "N consecutive packets" model will eventually happen in some codec. The question is a
this depends strongly on why multiple header packet but no standarized "serialization" codecs exist do they exist because the designers where unaware of the prblems this would cause or did they design it that way with the intention to make usage of their codec difficult in containers other then their own if later then yes no matter what you add, such people will do something to make storage hard
it's my belief that as the interest in storing codecs in arbitrary containers increases, everyone will realize that single binary block is the natural/"portable" way of storing headers. if this happens, then the only reason they would come up with more complicated arrangements would be to thwart interoperability.
now i dont know why xiph designed things like that, but i do know that they are _now_ aware of the issues (many people ask on their mailinglists about vorbis in non-ogg) and they dont specify how to turn the 3 packets into 1 ...
hmm. if they're "aware" of the issue now, do you think they'd be amenable to publishing a xiph-sanctioned spec for how to store vorbis in non-ogg containers. preferably using either the existing mkv format or the pure concatenation which you demonstrated is possible. (these two formats only differ by the existence of a few bytes of junk at the beginning of the former.
so IMHO, if a codec uses multiple packets because the designers wherent aware of the issues this has with most containers and APIs then they will probably not mind adding a few lines to their spec to specify how to store these headers in a one-header API/container, and thouse who design codecs with multiple header packets to make storeage in "unwanted" containers hard will always find a way to reach that goal
agree!
so IMHO supporting multiple header packets at nut level might solve vorbis & theora but its usefullness with future codecs is uncertain
imnsho it creates a lot more opportunity for ambiguity about how the headers should be stored than its worth. if we added something like this, suddenly there would be questions about how h.264, etc. should be stored. avoiding such ambiguities which lead to countless different combinations that an implementation must support seems to be a very important goal. rich
Hi On Wed, Jul 19, 2006 at 12:04:56AM -0400, Rich Felker wrote: [...]
now i dont know why xiph designed things like that, but i do know that they are _now_ aware of the issues (many people ask on their mailinglists about vorbis in non-ogg) and they dont specify how to turn the 3 packets into 1 ...
hmm. if they're "aware" of the issue now, do you think they'd be amenable to publishing a xiph-sanctioned spec for how to store vorbis in non-ogg containers. preferably using either the existing mkv format or the pure concatenation which you demonstrated is possible. (these two formats only differ by the existence of a few bytes of junk at the beginning of the former.
well, ill subscribe & ask on vorbis-dev unless there are other ideas or some more diplomatic volunteer for it [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is
On Wed, Jul 19, 2006 at 09:05:58AM +0200, Michael Niedermayer wrote:
Hi
On Wed, Jul 19, 2006 at 12:04:56AM -0400, Rich Felker wrote: [...]
now i dont know why xiph designed things like that, but i do know that they are _now_ aware of the issues (many people ask on their mailinglists about vorbis in non-ogg) and they dont specify how to turn the 3 packets into 1 ...
hmm. if they're "aware" of the issue now, do you think they'd be amenable to publishing a xiph-sanctioned spec for how to store vorbis in non-ogg containers. preferably using either the existing mkv format or the pure concatenation which you demonstrated is possible. (these two formats only differ by the existence of a few bytes of junk at the beginning of the former.
well, ill subscribe & ask on vorbis-dev unless there are other ideas or some more diplomatic volunteer for it
imo you're a good spokesperson. i don't think they like me very much these days... :) and you're well-respected for technical competence and as one of the main developers of ffmpeg (albeit a project which xiph doesn't much respect). rich
On Tue, Jul 18, 2006 at 09:02:07AM +0200, Baptiste Coudurier wrote:
Hi
Oded Shimon wrote:
[...]
Ivan, vorbis doesn't expect 3 extradata according to the spec, it expect the first 3 audio packet to be the headers. I think this is insane enough that it allows us to break their spec instead of ours.
Huh. How come a container can surpasses codec specs ? IMHO that's not acceptable.
The vorbis spec conflicts with a fundamental part of the NUT spec (and any sane container spec) that absolutely cannot be broken. There is no possible way to do what the vorbis spec says because then you would not have the headers if you started playing mid-file or if you cut part of the file to make a new file. This kind of brokenness was the intent of the Xiph developers. They want every program that deals with vorbis audio to have to use their ogg container, and they want every program that deals with vorbis audio to specifically know it's vorbis and have vorbis-specific code for dealing with it rather than being able to treat it as an abstract codec. They are completely and utterly insane. However, they did succeed in making one (exactly one) good codec. As far as I'm concerned, the approach that Matroska has taken (and that we take) is completely reasonable given Xiph's unreasonableness. The alternative would be to just fork the vorbis spec entirely, stop calling it vorbis, fix the bitstream to be more efficient, etc... Rich
Baptiste Coudurier wrote:
Huh. How come a container can surpasses codec specs ? IMHO that's not acceptable.
let rephrase the spec: " you need 3 parts of configuration in order to make the decoder happy: Info Comment Setup if the decoder doesn't get those three chunks in this order it cannot decode the stream. " Now the decoder won't care if those are packed together, put in a gpg enclosure or delivered by quantum singularity, all it cares is just that it can get them in order. (see the current implementation to feel the irony). the codec should not tell much about it's configuration data beside that is it, packing is up to the muxer in the end. lu -- Luca Barbato Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero
Michael Niedermayer <michaelni@gmx.at> writes:
and there are many codecs which have several chunks all the mpeg variants do, but they have startcodes so its a non issue
Hmm... I've never heard of MPEG1/2 stored with OOB data. Have I been looking in the wrong places?
h.264 does too but with, the startcode prefixes being optional, IMHO we should just require the startcode prefixes to be in there ...
Makes sense.
then there are some mov based codecs like qdm2 which maybe need several chunks, they currently simply copy a whole set of mov chunks blindly into extradata in ffmpeg, that of course is a terrible mess which someone should fix in ffmpeg ...
Whoever designs codecs like that should be shot. -- Måns Rullgård mru@inprovide.com
Hi On Sun, Jul 16, 2006 at 02:04:48PM +0100, Måns Rullgård wrote:
Michael Niedermayer <michaelni@gmx.at> writes:
and there are many codecs which have several chunks all the mpeg variants do, but they have startcodes so its a non issue
Hmm... I've never heard of MPEG1/2 stored with OOB data. Have I been looking in the wrong places?
no, ive never heard of it either ... [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is
Michael Niedermayer <michaelni@gmx.at> writes:
Hi
On Sun, Jul 16, 2006 at 02:04:48PM +0100, Måns Rullgård wrote:
Michael Niedermayer <michaelni@gmx.at> writes:
and there are many codecs which have several chunks all the mpeg variants do, but they have startcodes so its a non issue
Hmm... I've never heard of MPEG1/2 stored with OOB data. Have I been looking in the wrong places?
no, ive never heard of it either ...
So I should just ignore what you said? -- Måns Rullgård mru@inprovide.com
Hi On Sun, Jul 16, 2006 at 03:15:50PM +0100, Måns Rullgård wrote:
Michael Niedermayer <michaelni@gmx.at> writes:
Hi
On Sun, Jul 16, 2006 at 02:04:48PM +0100, Måns Rullgård wrote:
Michael Niedermayer <michaelni@gmx.at> writes:
and there are many codecs which have several chunks all the mpeg variants do, but they have startcodes so its a non issue
Hmm... I've never heard of MPEG1/2 stored with OOB data. Have I been looking in the wrong places?
no, ive never heard of it either ...
So I should just ignore what you said?
if you like ... mpeg1/2 do have several chunks which could be considered to be global headers but noone stores them out of band / in a global header furthermore they could be trivially stored as they are in a global header due to their startcodes delimiting the individual chunks [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is
Michael Niedermayer <michaelni@gmx.at> writes:
Hi
On Sun, Jul 16, 2006 at 03:15:50PM +0100, Måns Rullgård wrote:
Michael Niedermayer <michaelni@gmx.at> writes:
Hi
On Sun, Jul 16, 2006 at 02:04:48PM +0100, Måns Rullgård wrote:
Michael Niedermayer <michaelni@gmx.at> writes:
and there are many codecs which have several chunks all the mpeg variants do, but they have startcodes so its a non issue
Hmm... I've never heard of MPEG1/2 stored with OOB data. Have I been looking in the wrong places?
no, ive never heard of it either ...
So I should just ignore what you said?
if you like ...
mpeg1/2 do have several chunks which could be considered to be global headers but noone stores them out of band / in a global header furthermore they could be trivially stored as they are in a global header due to their startcodes delimiting the individual chunks
I guess you're referring to the sequence header and related extension headers here. Yes, those could be stored OOB, but the standard doesn't make any such provisions. -- Måns Rullgård mru@inprovide.com
Hi On Sun, Jul 16, 2006 at 04:03:45PM +0200, Michael Niedermayer wrote:
Hi
On Sun, Jul 16, 2006 at 02:04:48PM +0100, Måns Rullgård wrote:
Michael Niedermayer <michaelni@gmx.at> writes:
and there are many codecs which have several chunks all the mpeg variants do, but they have startcodes so its a non issue
Hmm... I've never heard of MPEG1/2 stored with OOB data. Have I been looking in the wrong places?
no, ive never heard of it either ...
see: http://haali.cs.msu.ru/mkv/codecs.pdf matroska specifies how to put mpeg1/2 seq headers out of band ... [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is
Michael Niedermayer <michaelni@gmx.at> writes:
Hi
On Sun, Jul 16, 2006 at 04:03:45PM +0200, Michael Niedermayer wrote:
Hi
On Sun, Jul 16, 2006 at 02:04:48PM +0100, Måns Rullgård wrote:
Michael Niedermayer <michaelni@gmx.at> writes:
and there are many codecs which have several chunks all the mpeg variants do, but they have startcodes so its a non issue
Hmm... I've never heard of MPEG1/2 stored with OOB data. Have I been looking in the wrong places?
no, ive never heard of it either ...
see: http://haali.cs.msu.ru/mkv/codecs.pdf
matroska specifies how to put mpeg1/2 seq headers out of band ...
Oh no... That is only asking for trouble. ISO 13818-2 requires them to be inband, and all decoders will be expecting them there anyway. A container should never do things that directly violates the specs of a codec. However stupidly designed a codec is, attempting to "fix" it at the container level is more stupid. -- Måns Rullgård mru@inprovide.com
On Sun, Jul 16, 2006 at 02:58:02AM +0200, Michael Niedermayer wrote:
Could we perhaps generalize that format to include all codecs with several chunks of extradata, not only ogg based codecs? I'm not aware of any others, but for all I know they might exist.
ok, but the following needs disscussion: A always choose ogg style lacing B always choose v + packet data (nut style) C use whatever matroska uses + one of the above for the cases not supported by matroska
Keep in mind that the _intent_ of specifying this is _NOT_ that the NUT muxer or demuxer would have to process such mess, but rather that implementations should treat the specified extradata format as the standard for the codec, and put whatever hacks are necessary to use legacy implementations (e.g. libvorbis) in the wrapper code for the library.
and there are many codecs which have several chunks all the mpeg variants do, but they have startcodes so its a non issue
Yes, for once sanity prevails..
h.264 does too but with, the startcode prefixes being optional, IMHO we should just require the startcode prefixes to be in there ...
Agree strongly!
then there are some mov based codecs like qdm2 which maybe need several chunks, they currently simply copy a whole set of mov chunks blindly into extradata in ffmpeg, that of course is a terrible mess which someone should fix in ffmpeg ...
Any good ideas? If the mov atoms aren't hard to parse I wonder if original Qt codecs should just be allowed to keep their mov atoms... Of course this is unacceptable for any standard codec that just happens to be stored in mov, including mpeg4/h264. Rich
Hi On Sun, Jul 16, 2006 at 05:52:25PM -0400, Rich Felker wrote: [...]
then there are some mov based codecs like qdm2 which maybe need several chunks, they currently simply copy a whole set of mov chunks blindly into extradata in ffmpeg, that of course is a terrible mess which someone should fix in ffmpeg ...
Any good ideas? If the mov atoms aren't hard to parse I wonder if original Qt codecs should just be allowed to keep their mov atoms...
hmm, ive no strong objections here ... iam also thinking that maybe we should simply ignore these closed propriatary mov based codecs, i really doubt anyone will want to store them in nut ever anyway ... [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is
Hi Michael Niedermayer wrote:
Hi
On Sun, Jul 16, 2006 at 05:52:25PM -0400, Rich Felker wrote: [...]
then there are some mov based codecs like qdm2 which maybe need several chunks, they currently simply copy a whole set of mov chunks blindly into extradata in ffmpeg, that of course is a terrible mess which someone should fix in ffmpeg ... Any good ideas? If the mov atoms aren't hard to parse I wonder if original Qt codecs should just be allowed to keep their mov atoms...
hmm, ive no strong objections here ...
iam also thinking that maybe we should simply ignore these closed propriatary mov based codecs, i really doubt anyone will want to store them in nut ever anyway ...
[...]
What do you mean "proprietary" mov ? Consider that now mov/isom/mp4/3gp is ISO file format reference. Im thinking of jpeg 2000 extradata, that has a chance to be in NUT some day, same for qclp (3g2 audio codec) and what about aac ? aac cant be stored as adts in nut it violates the specs (double container) -- Baptiste COUDURIER GnuPG Key Id: 0x5C1ABAAA SMARTJOG S.A. http://www.smartjog.com Key fingerprint 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA Phone: +33 1 49966312
On Mon, Jul 17, 2006 at 12:10:56AM +0100, Måns Rullgård wrote:
Baptiste Coudurier <baptiste.coudurier@smartjog.com> writes:
aac cant be stored as adts in nut it violates the specs (double container)
ADTS is IMHO no more of a container than mp3 is.
I really don't know what ADTS looks like, but I would say that if it doesn't multiplex streams and doesn't have timestamps or seek info, it's probably not a container.. However if it's stupid bloat it probably shouldn't be stored in NUT anyway. Rich
Rich Felker <dalias@aerifal.cx> writes:
On Mon, Jul 17, 2006 at 12:10:56AM +0100, Måns Rullgård wrote:
Baptiste Coudurier <baptiste.coudurier@smartjog.com> writes:
aac cant be stored as adts in nut it violates the specs (double container)
ADTS is IMHO no more of a container than mp3 is.
I really don't know what ADTS looks like, but I would say that if it doesn't multiplex streams and doesn't have timestamps or seek info, it's probably not a container..
The ADTS header is 7 bytes per frame containing a sync word, AAC type, sample rate, number of channels, frame size, and a few useless pieces of information. It is equivalent to the mp3 frame header. In a container with proper framing, this information can be stored in a global header instead. MOV does just that, but of course uses a different format for the global header. See libavcodec/parser.c and libavformat/adtsenc.c for the gory details. The reason ADTS exists is (I assume) to make possible raw AAC streams and headerless streams for broadcast.
However if it's stupid bloat it probably shouldn't be stored in NUT anyway.
Since nut has an extradata space, using it seems like the right choice. It does save a little overhead. -- Måns Rullgård mru@inprovide.com
Hi On Mon, Jul 17, 2006 at 12:58:47AM +0200, Baptiste Coudurier wrote:
Hi
Michael Niedermayer wrote:
Hi
On Sun, Jul 16, 2006 at 05:52:25PM -0400, Rich Felker wrote: [...]
then there are some mov based codecs like qdm2 which maybe need several chunks, they currently simply copy a whole set of mov chunks blindly into extradata in ffmpeg, that of course is a terrible mess which someone should fix in ffmpeg ... Any good ideas? If the mov atoms aren't hard to parse I wonder if original Qt codecs should just be allowed to keep their mov atoms...
hmm, ive no strong objections here ...
iam also thinking that maybe we should simply ignore these closed propriatary mov based codecs, i really doubt anyone will want to store them in nut ever anyway ...
[...]
What do you mean "proprietary" mov ? Consider that now mov/isom/mp4/3gp is ISO file format reference.
propriatary mov based _codecs_, stuff where you need to sign an NDA or worse to see the specs (SVQ3/...)
Im thinking of jpeg 2000 extradata, that has a chance to be in NUT some day,
jpeg 2000 has extradata? its a image based (=intra only) codec it shouldnt have much extradata ...
same for qclp (3g2 audio codec)
i cant comment on a codec for which i have no specs or source code [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB In the past you could go to a library and read, borrow or copy any book Today you'd get arrested for mere telling someone where the library is
On Sat, Jul 15, 2006 at 11:49:47PM +0200, michael wrote:
Author: michael Date: Sat Jul 15 23:49:47 2006 New Revision: 19110
Modified: trunk/DOCS/tech/nut.txt
Log: clarify "extradata" comments welcome! if anyone disagrees ill reverse this and we can disscuss it, i just thought there wont be any disagreement and was too lazy to send a patch ...
Modified: trunk/DOCS/tech/nut.txt ============================================================================== --- trunk/DOCS/tech/nut.txt (original) +++ trunk/DOCS/tech/nut.txt Sat Jul 15 23:49:47 2006 @@ -512,6 +512,19 @@
codec_specific_data private global data for a codec (could be huffman tables or ...) + if a codec has a global header it SHOULD be placed in here instead of + at the start of every keyframe + the exact format is specified in the codec spec + codecs which dont specify it in their spec are specified below + for ogg based codecs (vorbis, theora) the following format shall be used + number_of_headers_minus_1 u(8) + for(i=0; i<number_of_headers; i++){ + -1 u(8*(size[i]/255)) + size[i] % 255 u(8) + } + for(i=0; i<number_of_headers; i++) + header[i] + Note, this is the same format these codecs use in matroska
I like this change, but storing them all in the main spec file will get out of hand if there come to me many more codecs that require stupid formatting. Maybe we could put stuff like this in a supplemental file. I was talking with Haali (of windows fame) on #ffmpeg and he said Matroska already has a list of extradata formats, packed into a single binary data block. I have not read it yet but here's the file: http://haali.cs.msu.ru/mkv/codecs.pdf We might want to consider making a sort of "alliance" on this to strengthen the notion that "single binary block in header" is the standard for codec init data, and that all containers and codecs should conform to it. Rich
participants (8)
-
Baptiste Coudurier -
Ivan Kalvachev -
Luca Barbato -
michael -
Michael Niedermayer -
Måns Rullgård -
Oded Shimon -
Rich Felker