[FFmpeg-devel] [PATCH] attachments support in matroska demuxer

[Rich Felker]

On Wed, Jan 02, 2008 at 06:49:28AM +0200, Uoti Urpala wrote:
> >  That would be a serious
> > design flaw and hardly one upon which we should base our API.
> 
> API for doing what? Libavformat should provide access to the contents of
> Matroska attachments. What other things should the same API be
> generalized to do? Hypothetical attachments in other containers?
> Restrict it to less than general attachment handling for subtitle fonts
> only, but generalize that to hypothetical fonts in other containers?

I think Michael's concern is how you expect a decoder for ASS/SSA
subtitles to obtain the fonts. Having demuxer-specific code (or even
code which depends on a demuxer api at all) in the decoder is not
acceptable.

> > Also speaking of redundancy between streams, fonts are not special here
> > extradata will often be identical or similar between streams of the same
> > codec.
> 
> Extradata may happen to be identical, but subtitles can refer to "the
> same font" in a stricter sense. If you demux a .mkv file into separate
> files, a natural format is to have an audio file (or files), a video
> file, a .ssa/.ass file (or files) and a bunch of font files. Nobody
> splits video/audio tracks into separate "standard extradata" and "main
> content" parts.

I agree here too. Most subtitle 'codecs' already have an extradata
format of their own, text-based, separate from the binary ttf data.
Most likely it's the "header text" which would appear at the beginning
of the subtitle file if it were flat text. So it would be rather
problematic to try to munge the fonts together with this...

> Duplicating the fonts can be expensive if there are lots of subtitle
> tracks (which there can be as they're otherwise small). In addition to
> the memory directly used by the font files there are also whatever data
> structures parsing and rendering them requires - duplicating the fonts
> in the extradata of each track would force a decoder to do extra work
> comparing them to go back to the shared representation if it wants to

This makes sense.

> avoid that overhead. And what about other Matroska attachments?
> Shouldn't there be a general way to access them? I think it would be
> better to create an API for accessing Matroska attachments (generalized
> to attachments in other containers if some get such functionality), and
> leave issues specific to fonts and their relation to subtitle tracks
> outside libavformat for now. 

The whole idea of attachments is rather disgusting and something of a
hack... But I don't know any right way to deal with them, just that
existing ways are not the right way..

Rich