[FFmpeg-devel] [PATCH] attachments support in matroska demuxer

[Aurelien Jacobs]

On Thu, 3 Jan 2008 02:23:09 +0100
Michael Niedermayer <michaelni at gmx.at> wrote:

> On Thu, Jan 03, 2008 at 01:38:39AM +0100, Aurelien Jacobs wrote:
> > On Sat, 29 Dec 2007 22:38:24 +0300
> > Evgeniy Stepanov <eugeni.stepanov at gmail.com> wrote:
> [...]
> > Conclusions:
> > - Attachment really looks like a stream itself. You can select
> >   the one you want to remux. For example the film poster would
> >   just be a jpeg stream with a single picture. The specificity
> >   of those streams would be:
> 
> >     * contain one single packet
> >     * not pts
> 
> Contain no packets, just one global extradata, this was proposed already.
> If you use normal packets there will be problems with them disapearing
> if a stream is split timewise or if you seek before demuxing the
> first packet.

Hum... Indeed, using extradata has some advantages over normal packet...

> Iam not completely happy with this use of streams but we will need some
> solution and maybe its the least ugly, we will see ...
> 
> at least iam pretty sure extradata to be a better choice than 1 normal
> packet

Still it has at least one drawback: applications can't decode
attached pictures as a normal stream. They will need some special
handling.
But, well, extradata may indeed be the best solution.

> > - I think the only required API modifications to support this
> >   would be:
> >     * adding AVStream.filename and maybe AVStream.id_of_link_stream.
> >     * adding CODEC_ID_TTF ? (ugly, but shouldn't cause much trouble)
> >     * adding a way to mux ttf font in a separate file (maybe use
> >       a separate muxer for this, ie. call 2 different muxers from
> >       ffmpeg. seems ugly, but I've no better idea right now).
> 
> Well, there are a few more things i think
> We need name / (mime) type to remux attachments
> (CD Cover / ? / tiff)
> (Fan art / image/x-photoshop / psd)
> There also could be other things like Author, ...
> 
> Also we need a way to get the attachments to the decoders, yes we dont
> have a ASS/SSA decoder but lets assume we had.

I wonder if we really need this.
It depends on what you consider a SSA decoder should do.
Maybe a SSA decoder should just output some plain-text+formating
(so that it can be re-encoded to another textual format), and thus
it don't need the ttf file.
The subtitle rendering (ttf bliting) could be done in libvo (for
optimal speed) or in libavfilter (subtitle incrustation during
transcoding for example).
Still we need a way to get the attachments to vo/filters...

> And while we can just store the streams in avi/mov, where does the
> filename/name/type go? if its dropped you could end up with a huge
> number of images and not know what each is.

Indeed. But then we can either:
 - accept this limitation of avi/mov container
 - write those attachments in separate files
Or even better, be able to choose for each attachement if you want
it as a separate file or not.
In fact it would be as simple as: let the user remux the stream
he wants inside the avi, and then let him do some more invocation of
ffmpeg to extract other streams in separate files using raw muxer.

Aurel