Should Join Discussion - Fwd: [whatwg] Give guidance about RFC 4281 codecs parameter
Hello, You guys might want to join this discussion. (And maybe some others too.) On the WhatWG mailing list, there's been a fury of e-mails about adding a <video> element to HTML5. They're talking about codecs and containers now. It might be helpful to have your representatives there. And help shape things. See ya ---------- Forwarded message ---------- From: Dave Singer <singer@apple.com> Date: Apr 9, 2007 10:15 AM Subject: Re: [whatwg] Give guidance about RFC 4281 codecs parameter To: Henri Sivonen <hsivonen@iki.fi>, WHAT working group <whatwg@lists.whatwg.org> Cc: Randall Gellens <randy@qualcomm.com>, "Per Fröjdh (KI/EAB)" <per.frojdh@ericsson.com> WARNING: I have CC'd the co-authors of the RFC, as I think they might like to see the discussion, comment on my answers, and possibly correct me. I also have a question whether there is a typo in the RFC... * * * * * Henry these are all great questions. Let me see how many I can answer. Overall, the RFC was struggling with the issue that there is no 'uniform' naming of codecs; the namespace for codecs is dependent on the container format, so products that do container conversion have to have tables of code matches. ugh. That's why the RFC is as it is. The RFC suggests that updated information would be done with RFCs, which is a little heavy. The RFC as written formally applies to 3GPP files and 3GPP2 files, but the definitions are applicable for all ISO-family files. As you'll see below, 3GPP has also defined it for avc1 in MP4-family containers, but no spec. or registration authority provides a pointer. We might want to ask IANA whether we could add something to the MIME registry. At 11:37 +0300 8/04/07, Henri Sivonen wrote: * Theora video and Vorbis audio in Ogg container. (application/ogg; .ogg) * Dirac video and Vorbis audio in Ogg container. (application/ogg; .ogg) * Theora video and Vorbis audio in Matroska container. (video/x-matroska; .mkv) * Dirac video and Vorbis audio in Matroska container. (video/x-matroska; .mkv) My understanding is that the Ogg container is 'specific' to these codecs, and therefore the codecs parameter is not needed. But I am not an Ogg or Matroska expert; perhaps they could chime in? We did have the discussion over profiles of these codecs; I understand profiles are not used, but I am still unclear as to which of the following is true: a) features are never added to the bitstreams, so any release of the decoder will decode bitstreams made by any release of the encoder (modulo bugs); b) the decoder release needed is signalled somehow in the bitstream ('need at least the April 2005 release or later to decode this file') c) neither of the above are true, it's left to the users to stay up to date, and if they don't, then, well, that's their problem. If there are profiles, release versions etc. signalled, then a parameter might be appropriate, and if the container formats are general, a codecs parameter might be appropriate. * H.264 Baseline profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4) The AAC is covered by the RFC; the example is there - mp4a.40.2. H.264 was recently covered by 3GPP. See 26.244 V7.1.0 section A.2.2, available from www.3gpp.org. When the first element of a value is 'avc1', indicating H.264 (AVC) video [29], the second element is the hexadecimal representation of the following three bytes in the sequence parameter set NAL unit specified in [29]: 1) profile_idc, 2) a byte composed of the values of constraint_set0_flag, constraint_set1_flag, constraint_set2_flag, constraint_set3_flag, and reserved_zero_4bits in bit-significance order, starting from the most significant bit, and 3) level_idc. Note that reserved_zero_4bits is required to be equal to 0 in [29], but other values for it may be specified in the future by ITU-T or ISO/IEC. You don't give me a level number, so I put xx for those hex digits below. Assuming we're simple baseline, and also main and extended compatible avc1.42E0xx * H.264 Extended profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4 Extended profile has the value (decimal) 88, and typically Extended profile streams would be marked as Baseline compatible, at least. Here is an example for this AVC avc1.58A0xx * H.264 Main profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4) Main profile is decimal 77, so something like avc1.4D40xx * H.264 High profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4) There are a number of high profiles, confusingly, though there is one called 'high', with value decimal 100. avc1.6400xx if it's not compatible with main, baseline, or extended profiles. * MPEG-4 Simple Profile profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4) Covered by the RFC: For example, MPEG-4 Visual Simple Profile Level 0 has the value 9, so a complete string for MPEG-4 Visual Simple Profile Level 0 would be "mp4v.20.9". Though when checking the next answer, I read in the spec. that we may have a typo here, it might be 8. Ooops, if so. Simple Profile/Level 0 00001000 Reserved 00001001 - 00001111 * MPEG-4 Advanced Simple Profile profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4) Advanced Simple Profile/Level 0 11110000 so, mp4v.20.240 * MPEG-4 Simple Profile profile video and AMR audio in 3GPP container. (video/3gpp; .3gp) Video we've covered. AMR is in 26.244, samr * WMV9 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv) * WMV8 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv) These would be up to Microsoft to define, I assume. I am not aware of a definition. * VC-1 video and WMA 10 Pro audio in ASF container. (video/x-ms-wmv; .wmv) VC-1 is standardized by SMPTE, but again this container format is Microsoft's. * Real Video 10 video and High-Efficiency AAC audio in Real Media container. (application/vnd.rn-realmedia; rm) We'd have to ask Real, but I don't think it is defined. * XviD video and MP3 audio in AVI container. (video/x-msvideo; .avi) * Motion-JPEG video and uncompressed PCM audio in AVI container. (video/x-msvideo; .avi) AVI I *think* is owned by Microsoft, and I *think* they deprecate it; it's up to the owner to define. Again, I am not aware of a definition, but the 4CC style from MP4 might be appropriate. * MPEG-1 video and MPEG-1 Audio Layer II audio in MPEG-1 program stream (video/mpeg; .mpg) MPEG has not defined the codecs parameter for these 'file' (stream) formats, as far as I know. (That's a lot of cases and, yet, none are contrived.) I tried to figure this out on my own with RFC 4281 and concluded that this is not something that authors or even browser implementors are likely to get right without an expert-created lookup table. I see that at least of the RFC authors reads this mailing list. :-) -- David Singer Apple Computer/QuickTime -- Charles Iliya Krempeaux, B.Sc. charles @ reptile.ca supercanadian @ gmail.com developer weblog: http://ChangeLog.ca/
Hello, Sorry... forgot to include this in the last e-mail. You can join the mailing list here... http://lists.whatwg.org/listinfo.cgi/whatwg-whatwg.org See ya On 4/9/07, Charles Iliya Krempeaux <supercanadian@gmail.com> wrote:
Hello,
You guys might want to join this discussion. (And maybe some others too.)
On the WhatWG mailing list, there's been a fury of e-mails about adding a <video> element to HTML5. They're talking about codecs and containers now. It might be helpful to have your representatives there. And help shape things.
See ya
---------- Forwarded message ---------- From: Dave Singer <singer@apple.com> Date: Apr 9, 2007 10:15 AM Subject: Re: [whatwg] Give guidance about RFC 4281 codecs parameter To: Henri Sivonen <hsivonen@iki.fi>, WHAT working group <whatwg@lists.whatwg.org> Cc: Randall Gellens <randy@qualcomm.com>, "Per Fröjdh (KI/EAB)" <per.frojdh@ericsson.com>
WARNING: I have CC'd the co-authors of the RFC, as I think they might like to see the discussion, comment on my answers, and possibly correct me. I also have a question whether there is a typo in the RFC...
* * * * *
Henry
these are all great questions. Let me see how many I can answer.
Overall, the RFC was struggling with the issue that there is no 'uniform' naming of codecs; the namespace for codecs is dependent on the container format, so products that do container conversion have to have tables of code matches. ugh. That's why the RFC is as it is.
The RFC suggests that updated information would be done with RFCs, which is a little heavy. The RFC as written formally applies to 3GPP files and 3GPP2 files, but the definitions are applicable for all ISO-family files.
As you'll see below, 3GPP has also defined it for avc1 in MP4-family containers, but no spec. or registration authority provides a pointer. We might want to ask IANA whether we could add something to the MIME registry.
At 11:37 +0300 8/04/07, Henri Sivonen wrote:
* Theora video and Vorbis audio in Ogg container. (application/ogg; .ogg) * Dirac video and Vorbis audio in Ogg container. (application/ogg; .ogg) * Theora video and Vorbis audio in Matroska container. (video/x-matroska; .mkv) * Dirac video and Vorbis audio in Matroska container. (video/x-matroska; .mkv)
My understanding is that the Ogg container is 'specific' to these codecs, and therefore the codecs parameter is not needed. But I am not an Ogg or Matroska expert; perhaps they could chime in?
We did have the discussion over profiles of these codecs; I understand profiles are not used, but I am still unclear as to which of the following is true: a) features are never added to the bitstreams, so any release of the decoder will decode bitstreams made by any release of the encoder (modulo bugs); b) the decoder release needed is signalled somehow in the bitstream ('need at least the April 2005 release or later to decode this file') c) neither of the above are true, it's left to the users to stay up to date, and if they don't, then, well, that's their problem.
If there are profiles, release versions etc. signalled, then a parameter might be appropriate, and if the container formats are general, a codecs parameter might be appropriate.
* H.264 Baseline profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4)
The AAC is covered by the RFC; the example is there - mp4a.40.2.
H.264 was recently covered by 3GPP. See 26.244 V7.1.0 section A.2.2, available from www.3gpp.org.
When the first element of a value is 'avc1', indicating H.264 (AVC) video [29], the second element is the hexadecimal representation of the following three bytes in the sequence parameter set NAL unit specified in [29]: 1) profile_idc, 2) a byte composed of the values of constraint_set0_flag, constraint_set1_flag, constraint_set2_flag, constraint_set3_flag, and reserved_zero_4bits in bit-significance order, starting from the most significant bit, and 3) level_idc. Note that reserved_zero_4bits is required to be equal to 0 in [29], but other values for it may be specified in the future by ITU-T or ISO/IEC.
You don't give me a level number, so I put xx for those hex digits below.
Assuming we're simple baseline, and also main and extended compatible avc1.42E0xx
* H.264 Extended profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4
Extended profile has the value (decimal) 88, and typically Extended profile streams would be marked as Baseline compatible, at least. Here is an example for this AVC
avc1.58A0xx
* H.264 Main profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4)
Main profile is decimal 77, so something like avc1.4D40xx
* H.264 High profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4)
There are a number of high profiles, confusingly, though there is one called 'high', with value decimal 100. avc1.6400xx
if it's not compatible with main, baseline, or extended profiles.
* MPEG-4 Simple Profile profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4)
Covered by the RFC:
For example, MPEG-4 Visual Simple Profile Level 0 has the value 9, so a complete string for MPEG-4 Visual Simple Profile Level 0 would be "mp4v.20.9".
Though when checking the next answer, I read in the spec. that we may have a typo here, it might be 8. Ooops, if so.
Simple Profile/Level 0 00001000 Reserved 00001001 - 00001111
* MPEG-4 Advanced Simple Profile profile video and Low-Complexity AAC audio in MP4 container. (video/mp4; .mp4)
Advanced Simple Profile/Level 0 11110000
so, mp4v.20.240
* MPEG-4 Simple Profile profile video and AMR audio in 3GPP container. (video/3gpp; .3gp)
Video we've covered. AMR is in 26.244,
samr
* WMV9 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv) * WMV8 video and WMA 2 audio in ASF container. (video/x-ms-wmv; .wmv)
These would be up to Microsoft to define, I assume. I am not aware of a definition.
* VC-1 video and WMA 10 Pro audio in ASF container. (video/x-ms-wmv; .wmv)
VC-1 is standardized by SMPTE, but again this container format is Microsoft's.
* Real Video 10 video and High-Efficiency AAC audio in Real Media container. (application/vnd.rn-realmedia; rm)
We'd have to ask Real, but I don't think it is defined.
* XviD video and MP3 audio in AVI container. (video/x-msvideo; .avi) * Motion-JPEG video and uncompressed PCM audio in AVI container. (video/x-msvideo; .avi)
AVI I *think* is owned by Microsoft, and I *think* they deprecate it; it's up to the owner to define. Again, I am not aware of a definition, but the 4CC style from MP4 might be appropriate.
* MPEG-1 video and MPEG-1 Audio Layer II audio in MPEG-1 program stream (video/mpeg; .mpg)
MPEG has not defined the codecs parameter for these 'file' (stream) formats, as far as I know.
(That's a lot of cases and, yet, none are contrived.)
I tried to figure this out on my own with RFC 4281 and concluded that this is not something that authors or even browser implementors are likely to get right without an expert-created lookup table. I see that at least of the RFC authors reads this mailing list. :-)
--
David Singer Apple Computer/QuickTime
-- Charles Iliya Krempeaux, B.Sc.
charles @ reptile.ca supercanadian @ gmail.com
developer weblog: http://ChangeLog.ca/
-- Charles Iliya Krempeaux, B.Sc. charles @ reptile.ca supercanadian @ gmail.com developer weblog: http://ChangeLog.ca/
participants (1)
-
Charles Iliya Krempeaux