nut: Add per frame sidedata and metadata

This patch adds frame sidedata and metadata support to nut It should be compatible with old demuxers so that they still can correctly demux new streams in most cases The design tries to reuse existing syntactical structures to keep things simple Please keep both ffmpeg-devel and nut-devel in CC if you reply

Signed-off-by: Michael Niedermayer <michaelni@gmx.at> --- docs/nut.txt | 111 +++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 83 insertions(+), 28 deletions(-) diff --git a/docs/nut.txt b/docs/nut.txt index 7969e3c..d2db40d 100644 --- a/docs/nut.txt +++ b/docs/nut.txt @@ -1,5 +1,5 @@ ================================== -NUT Open Container Format 20080202 +NUT Open Container Format 20130327 ================================== @@ -255,6 +255,7 @@ main_header: tmp_stream=0 tmp_match=1-(1<<62) tmp_head_idx= 0; + tmp_side_size = 0; for(i=0; i<256; ){ tmp_flag v tmp_fields v @@ -269,7 +270,8 @@ main_header: else count= tmp_mul - tmp_size if(tmp_fields>6) tmp_match s if(tmp_fields>7) tmp_head_idx v - for(j=8; j<tmp_fields; j++){ + if(tmp_fields>8) tmp_side_size v + for(j=9; j<tmp_fields; j++){ tmp_reserved[i] v } for(j=0; j<count && i<256; j++, i++){ @@ -286,6 +288,7 @@ main_header: reserved_count[i]= tmp_res; match_time_delta[i]= tmp_match; header_idx[i]= tmp_head_idx; + sidedata_size[i]= tmp_side_size; } } header_count_minus1 v @@ -344,13 +347,47 @@ frame: } if(frame_flags&FLAG_RESERVED) frame_res v - for(i=0; i<frame_res; i++) + if(frame_flags&FLAG_SIDEDATA) + sidedata_size v + for(i=0; i<frame_res - !(frame_flags&FLAG_SIDEDATA); i++) reserved v if(frame_flags&FLAG_CHECKSUM){ checksum u(32) } data +data: + frame_data + if (sidedata_size>0) + side_data + +side_data: + count v + for(i=0; i<count; i++){ + name vb + value s + if (value==-1){ + type= "UTF-8" + value vb + }else if (value==-2){ + type vb + value vb + }else if (value==-3){ + type= "s" + value s + }else if (value==-4){ + type= "t" + value t + }else if (value<-4){ + type= "r" + value.den= -value-4 + value.num s + }else{ + type= "v" + } + } + + index: max_pts t syncpoints v @@ -399,30 +436,7 @@ info_packet: until 2006-11-04.) chapter_start t chapter_len v - count v - for(i=0; i<count; i++){ - name vb - value s - if (value==-1){ - type= "UTF-8" - value vb - }else if (value==-2){ - type vb - value vb - }else if (value==-3){ - type= "s" - value s - }else if (value==-4){ - type= "t" - value t - }else if (value<-4){ - type= "r" - value.den= -value-4 - value.num s - }else{ - type= "v" - } - } + side_data reserved_bytes syncpoint: @@ -643,6 +657,7 @@ flags[frame_code], frame_flags (v) otherwise data_size_msb is 0. 6 FLAG_CHECKSUM If set, the frame header contains a checksum. 7 FLAG_RESERVED If set, reserved_count is coded in the frame header. + 8 FLAG_SIDEDATA If set, sidedata_size is coded in the frame header. 10 FLAG_HEADER_IDX If set, header_idx is coded in the frame header. 11 FLAG_MATCH_TIME If set, match_time_delta is coded in the frame header @@ -690,7 +705,47 @@ pts_delta[frame_code] (s) MUST be <16384 and >-16384. reserved_count[frame_code] (v) - MUST be <256. + Muxers conforming to this version of the specification MUST set it to + !!(flags[frame_code] & FLAG_SIDEDATA) + Demuxers MUST support any value from 0 to 255 + These rules are to ensure extensibility of the format. + +sidedata_size[frame_code] (v) + Size in bytes at the end inside data which represent frame sidedata and + frame metadata. + +side_data + This data structure is used both in frames for per frame side and metadata + as well as info tags for metadata covering the whole file, a stream + chapter or other. + Metadata is data that is about the actual data and generally not essential + for correct presentation + Sidedata is semantically part of the data and essential for its correct + presentation. The same syntax is used by both for simplicity. + The use of sidedata should be avoided and the information be put inside + the codec bitstream, if its syntax supports carrying such information. + Types of per frame side data: + "Channels", "ChannelLayout", "SampleRate", "Width", "Height" + This frame changes the number of channels, the channel layout, ... to + the given value (v) + If used in any frame of a stream then every keyframe of the stream + SHOULD carry such sidedata to allow seeking. + "Extradata", "Palette" + This frame changes the codec_specific_data or palette to the given + value (vb) + If used in any frame of a stream then every keyframe of the stream + SHOULD carry such sidedata to allow seeking. + "CodecSpecificSide<num>" + Codec specific side data, equivalent to matroskas BlockAdditional (vb) + the "<num>" should be replaced by a number identifying the type of + side data, it is equivalent/equal to BlockAddId in matroska. + "SkipStart", "SkipEnd" + The decoder should skip/drop the specified number of samples at the + start/end of this frame (v) + "UserData<identifer here>" + User specific side data, the "<identifer here>" should be replaced + by a globally unique identifer of the project that + uses/creates/understands the side data. For example "UserDataFFmpeg" data_size data_size = data_size_lsb + data_size_msb * data_size_mul ; -- 1.7.9.5

On 27/03/13 15:41, Michael Niedermayer wrote:
reserved_count[frame_code] (v) - MUST be <256. + Muxers conforming to this version of the specification MUST set it to + !!(flags[frame_code] & FLAG_SIDEDATA) + Demuxers MUST support any value from 0 to 255 + These rules are to ensure extensibility of the format.
Would enjoy a better explanation. Such as "muxer supporting FLAG_SIDEDATA MUST set the reserved_count to 1 if side data is in used in order to be compatible with older demuxers"
+ +sidedata_size[frame_code] (v) + Size in bytes at the end inside data which represent frame sidedata and + frame metadata.
metadata and sidedata are different items, I'm not exactly cool in lumping everything in the same serialized dictionary.
+ +side_data + This data structure is used both in frames for per frame side and metadata + as well as info tags for metadata covering the whole file, a stream + chapter or other. + Metadata is data that is about the actual data and generally not essential + for correct presentation + Sidedata is semantically part of the data and essential for its correct + presentation. The same syntax is used by both for simplicity. + The use of sidedata should be avoided and the information be put inside + the codec bitstream, if its syntax supports carrying such information.
This statement should be omitted.
+ Types of per frame side data: + "Channels", "ChannelLayout", "SampleRate", "Width", "Height" + This frame changes the number of channels, the channel layout, ... to + the given value (v) + If used in any frame of a stream then every keyframe of the stream + SHOULD carry such sidedata to allow seeking. + "Extradata", "Palette" + This frame changes the codec_specific_data or palette to the given + value (vb) + If used in any frame of a stream then every keyframe of the stream + SHOULD carry such sidedata to allow seeking. + "CodecSpecificSide<num>" + Codec specific side data, equivalent to matroskas BlockAdditional (vb) + the "<num>" should be replaced by a number identifying the type of + side data, it is equivalent/equal to BlockAddId in matroska. + "SkipStart", "SkipEnd" + The decoder should skip/drop the specified number of samples at the + start/end of this frame (v) + "UserData<identifer here>" + User specific side data, the "<identifer here>" should be replaced + by a globally unique identifer of the project that + uses/creates/understands the side data. For example "UserDataFFmpeg"
I'm not certain serializing type and enumeration this way would work that well. lu

On Wed, Mar 27, 2013 at 05:14:23PM +0100, Luca Barbato wrote:
On 27/03/13 15:41, Michael Niedermayer wrote:
reserved_count[frame_code] (v) - MUST be <256. + Muxers conforming to this version of the specification MUST set it to + !!(flags[frame_code] & FLAG_SIDEDATA) + Demuxers MUST support any value from 0 to 255 + These rules are to ensure extensibility of the format.
Would enjoy a better explanation. Such as "muxer supporting FLAG_SIDEDATA MUST set the reserved_count to 1 if side data is in used in order to be compatible with older demuxers"
ok, added
+ +sidedata_size[frame_code] (v) + Size in bytes at the end inside data which represent frame sidedata and + frame metadata.
metadata and sidedata are different items, I'm not exactly cool in lumping everything in the same serialized dictionary.
i see what you mean, ill post a different suggestion in a moment
+ +side_data + This data structure is used both in frames for per frame side and metadata + as well as info tags for metadata covering the whole file, a stream + chapter or other. + Metadata is data that is about the actual data and generally not essential + for correct presentation + Sidedata is semantically part of the data and essential for its correct + presentation. The same syntax is used by both for simplicity. + The use of sidedata should be avoided and the information be put inside + the codec bitstream, if its syntax supports carrying such information.
This statement should be omitted.
ok
+ Types of per frame side data: + "Channels", "ChannelLayout", "SampleRate", "Width", "Height" + This frame changes the number of channels, the channel layout, ... to + the given value (v) + If used in any frame of a stream then every keyframe of the stream + SHOULD carry such sidedata to allow seeking. + "Extradata", "Palette" + This frame changes the codec_specific_data or palette to the given + value (vb) + If used in any frame of a stream then every keyframe of the stream + SHOULD carry such sidedata to allow seeking. + "CodecSpecificSide<num>" + Codec specific side data, equivalent to matroskas BlockAdditional (vb) + the "<num>" should be replaced by a number identifying the type of + side data, it is equivalent/equal to BlockAddId in matroska. + "SkipStart", "SkipEnd" + The decoder should skip/drop the specified number of samples at the + start/end of this frame (v) + "UserData<identifer here>" + User specific side data, the "<identifer here>" should be replaced + by a globally unique identifer of the project that + uses/creates/understands the side data. For example "UserDataFFmpeg"
I'm not certain serializing type and enumeration this way would work that well.
its easy extendible and quite flexible, it also uses existing syntax Thanks [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB Many that live deserve death. And some that die deserve life. Can you give it to them? Then do not be too eager to deal out death in judgement. For even the very wise cannot see all ends. -- Gandalf
participants (2)
-
Luca Barbato
-
Michael Niedermayer