[FFmpeg-devel] [PATCH] attachments support in matroska demuxer

[Aurelien Jacobs]

On Sat, 29 Dec 2007 22:38:24 +0300
Evgeniy Stepanov <eugeni.stepanov at gmail.com> wrote:

> Hi,
> 
> this patch adds attachments support to mkv demuxer. They are collected in 
> AVFormatContext::attachments. This is needed to correctly display SSA/ASS 
> subtitles that often depend on fonts emdedded in matroska container.

Well, obviously, AVFormatContext.attachments is very controversial.

I will try to summarize things which were pointed out, and my ideas:

- Matroska has a feature called attachment. It consist in a kind
  of generalized archiver (think about a tar file concatenated to
  the Matroska file).
  (whether it's ugly or not is irrelevant. files exists in the wild
  so we need to support them)
- Attachment can be any kind of file. It has a filename and a mime-type.
  It's commonly used to store required ttf fonts, but also jpeg/png of
  the poster of the movie.
- Attachement may not be related to any other stream (poster of the
  movie), or related to one or more stream (ttf fonts for example)
- Streams might be "linked" to one of the attachment (subtitle stream
  linked to one ttf font for example), but I have not verified if
  it's used in practice.

We need a clean and generic solution to demux and remux this.
Specificially we should try to remux this in other containers
(avi, mov...) and we should be able to select which stream
and which attachments we want to remux.

Deductions:
- We can't rely on any link between streams and attachments to
  deduce which attachment needs to be demuxed/remuxed.
- We can't put attachments in extradata of a codec as they may not
  be related to any stream, and codec such as SSA alreay have
  some extradata.

Conclusions:
- Attachment really looks like a stream itself. You can select
  the one you want to remux. For example the film poster would
  just be a jpeg stream with a single picture. The specificity
  of those streams would be:
    * contain one single packet
    * not pts
    * has a filename (important so that an ASS decoder can
      select the proper ttf based on it's name)
    * always demuxed before other (normal) streams (IIRC)
  It seems to me that attachment would fit pretty well, each one
  in its own AVStream. Does this sound reasonable ?
- Here is how remuxing to other formats (avi, mov...) could work:
    * supported codec (jpeg poster) could be muxed as a normal
      JPEG stream with only one picture
    * ttf streams should be written in a separate file (that's
      the way various players expect to find ttf fonts when
      demuxing AVI, IIRC)
- I think the only required API modifications to support this
  would be:
    * adding AVStream.filename and maybe AVStream.id_of_link_stream.
    * adding CODEC_ID_TTF ? (ugly, but shouldn't cause much trouble)
    * adding a way to mux ttf font in a separate file (maybe use
      a separate muxer for this, ie. call 2 different muxers from
      ffmpeg. seems ugly, but I've no better idea right now).

What do you think about this proposal ? Does it sounds like a
reasonable base ?
If it's not, the only viable alternative seems to be
AVFormatContext.attachments just like in the original patch,
but with some additionnal code to allow "remuxing", etc...

Aurel