[Libav-user] Recomended approach for a timed metadata track within an mp4 (ISOBMFF section 12.3.3.2)

[Tom Vercauteren]

> > In such a case, am I right in assuming that using a stream of type
> > AV_MEDIA_TYPE_DATA that uses codec AV_CODEC_ID_SMPTE_KLV is a good way to
> > go? Something similar to how it is suggested here?
> > https://stackoverflow.com/q/68304330/17261462
> Its hard to say, since I'm not sure what your functional need is. If the
> content isn't KLV (see below), then saying AV_CODEC_ID_SMPTE_KLV isn't
> correct.
> That SO post looks to be more transport stream oriented. YMMV with ffmpeg
> and
> EMSG.
>
> > Now, for the message itself, is there a recommended MISB compliant KLV
> > library or is it OK to "hack" it by say using protobuf for serialising
> the
> > metadata (similar to
> https://github.com/Parrot-Developers/libvideo-metadata
> > )?
> Depends on interoperability need. If you want to do something MISB
> compliant,
> then protobuf is not OK.  If its just for you, then you can package it to
> meet
> your needs.
>

Thanks for the feedback. In terms of functional needs, I am interested in
storing some frame-level metadata in a fragmented mp4 file. The resulting
video should be readable in a standard media player. If the metadata stream
is compliant with an existing standard (e.g. MISB or ISOBMFF), that would
be best but otherwise, I am fine with a non-standard-compliant metadata
stream that I am able to write and read back with libav.

The reason I was referring to that SO post is that is while it focuses on
streaming, it seems to have the most detailed sample code for setting up a
KLV stream with ffmpeg / libav.

I tried adding a KLV track within the muxing.c example code from ffmpeg
https://github.com/FFmpeg/FFmpeg/blob/release/5.0/doc/examples/muxing.c
but haven't managed to do so yet. My initial attempts lead to the following
error message:
"Could not find tag for codec klv in stream #2, codec not currently
supported in container"

This led me to taking a step back and try from the ffmpeg command line.
Unfortunately ffmpeg is also throwing an error when trying to copy a
(suposedly) valid KLV stream (from the library of ffmpeg sample files) to
an mp4 or mpg file (despite the original file with KLV data being an mpg
file). Calling
ffmpeg -i https://samples.ffmpeg.org/MPEG2/mpegts-klv/Day%20Flight.mpg -c
copy -map 0 test.mp4
indeed also leads to
"Could not find tag for codec klv in stream #1, codec not currently
supported in container"

It works with a ts file though.
ffmpeg -i https://samples.ffmpeg.org/MPEG2/mpegts-klv/Day%20Flight.mpg -c
copy -map 0 test.ts

Does this mean that muxing a KLV stream in an mp4 is not yet supported by
libav or is there a workaround for this? If the former, would muxing a
datastream with a different codec be supported in an mp4?

It would be great if the muxing.c example could be expanded to include a
data stream.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20220403/4e70c2e7/attachment.htm>