On Mon, Dec 23, 2013 at 01:28:16PM +0100, Reimar Döffinger wrote:
On 23.12.2013, at 03:21, Michael Niedermayer <michaelni@gmx.at> wrote:
This variant is simpler and has less overhead than the previous It is not compatible with existing demuxers though, the reasoning here is that as side data is essential for presentation of a stream demuxer support is neccessary anyway.
This seems to split things into "side data" which is defined as essential for decoding and metadata. But this seems a bit too limited to me. For example I believe we now have side data in FFmpeg that contains CC subtitle data. It is clearly not necessary for decoding and it clearly isn't metadata either.
its not neccessary for decoding the video but it is neccessary for presentation of the video where audio cannot be used, like in case of a loud environment, hearing impairment or insufficient language compatibility between the language used in the audio and the listener so i would argue that it qualifies as side data if its not in its own stream/track
I guess it maybe shouldn't be written in this structure at all, but would/will the FFmpeg muxer do something sensible with it?
Iam not sure which side data type you speak of exactly?
I think it would be good to make sure we have enough flexibility to represent stuff we did not yet think of,
Its hard to make sure that we support things that we did not think of. If one asks the question "what did we not think of" well maybe side data per slice or per macroblock or metadata for a subset of pixels in a picture These could be supprted by including the "where/what" in the "key/name" tough like "Person-RectXY[50,70-200,30]", "guy fawkes" "Slice56-FirstMV", "12.25,89" iam not sure how usefull that is though ...
though I fully approve of the "essential to decode" distinction in principle. It's just that we don't really have this information if its essential in FFmpeg, so I'd suspect we might end up generating lots of invalid files (due to them having non-essential data in side data for example).
id assume all metadata would use AV_PKT_DATA_STRINGS_METADATA or AV_PKT_DATA_METADATA_UPDATE and the rest would be side data. if a new field that represents metadata is added to ffmpeg but not to nut and if our nut muxer would store unknown types (it does not currently, but would with the proposed patch, but that could be changed) and the input would contain that new field then the muxer would create a file that contains a frame with a side data element that is "Junk" a demuxer either would ignore the side data or extract it back into the new metadata field opaquely. While this would be incorrect in some sense, as it should have been stored in metadata instead, i dont see how it could break anything. Also there are quite a few ifs in there and id consider such "storing in the wrong list" a bug in the implementation [...] -- Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB It is what and why we do it that matters, not just one of them.