[Libav-user] Recomended approach for a timed metadata track within an mp4 (ISOBMFF section 12.3.3.2)

[Brad Hards]

On Friday, 1 April 2022 7:58:18 PM AEDT Tom Vercauteren wrote:
> As far as I understand, MISB ST 1910.1 with EMSG box would be mostly
> important for web streaming purpose, right? In my case, I am moslty
> interested in being able to store (and read back) some metadata alongside
> each video frame of an mp4 file. From that article, it seems like adding
> KLV metadata would be sufficient for me and would be supported by ffmpeg /
> libav already.
It is intended for viewing in the browser, but nothing stops you processing it 
some other way. 
 
> In such a case, am I right in assuming that using a stream of type
> AV_MEDIA_TYPE_DATA that uses codec AV_CODEC_ID_SMPTE_KLV is a good way to
> go? Something similar to how it is suggested here?
> https://stackoverflow.com/q/68304330/17261462
Its hard to say, since I'm not sure what your functional need is. If the 
content isn't KLV (see below), then saying AV_CODEC_ID_SMPTE_KLV isn't 
correct.
That SO post looks to be more transport stream oriented. YMMV with ffmpeg and 
EMSG.
 
> Now, for the message itself, is there a recommended MISB compliant KLV
> library or is it OK to "hack" it by say using protobuf for serialising the
> metadata (similar to https://github.com/Parrot-Developers/libvideo-metadata
> )?
Depends on interoperability need. If you want to do something MISB compliant, 
then protobuf is not OK.  If its just for you, then you can package it to meet 
your needs.
 
> Also, is it ok to leave the interleaving to ffmpeg, i.e. using
> av_interleaved_write_frame rather than write_frame (which is shown in many
> of the questioned I read so far)?
If it works for your needs, I don't see a problem.

Brad