[MPlayer-dev-eng] Embed Data in Video Stream

[Ivan Kalvachev]

On Jan 7, 2008 6:41 AM, Keith Chew <keith.chew at gmail.com> wrote:
> Hi
>
> I am looking for a way to embed data into video frames, like how data
> can be embedded in images using Exif standard.
>
> For example, location data (GPS) can be embedded into the video stream
> every second, while mencoder is encoding the video.
>
> During playback, would be great if I can add a onData() method hook,
> and decide how to process that information.
>
> Does such capabilities exist in mencoder/mplayer? Looking at the
> codebase, I cannot see anything so would be really good if someone can
> point me to the right direction. I don't need detailed explanation,
> just some starting points and I will try to figure it all out.

I'm afraid this channel is not dedicated on teaching how you can use
mplayer/mencoder, even if this involves coding. Come back when you
have patch to apply.

Now on the question. From the short look I took it looks like EXIF is
container format that is used to store the jpeg and the other
metadata.

The bad news is that the video already have well defined method for
storing in containers like avi,asf,rm,mpeg,ogg,mkv,nut. If you
encapsulate the video in exif and put it in one of these containers,
nothing but your version would be able to play it.

Now the good news.

There are many places where you can put some extra data into mpeg
streams, but you'd need to make critical changes in both muxer and
demuxer, be sure it won't have side effects like startcode-emulation
and stuff.

Some of the formats allow subtitles. If the data you want could be
represented as short text, this would be the most easier way to handle
it.

Some of the formats allow defining custom streams, it would be like
subtitles, but you can put any kind of binary data in it. Ogg is the
one who workaround its own lack of skeleton structure in that way.

The format MKV is based on binary XML, so you can theoretically extend
it to support whatever you need, however you still may need to code
muxer-demuxer changes.

The bad news is that mencoder primary output container is avi, that is
quite limited as standard, but the good news is that mencoder support
all libavformat (de)muxers, so you'd have to go to ffmpeg coding.