[FFmpeg-devel] [PATCH] attachments support in matroska demuxer

[Michael Niedermayer]

On Wed, Jan 02, 2008 at 03:47:40AM +0200, Uoti Urpala wrote:
> On Tue, 2008-01-01 at 13:27 +0100, Michael Niedermayer wrote:
> > On Tue, Jan 01, 2008 at 01:17:57PM +0300, Evgeniy Stepanov wrote:
> > > On 12/31/07, Michael Niedermayer <michaelni at gmx.at> wrote:
> > > > such data belongs in AVCodecContext.extradata/extradata_size
> > > 
> > > They are not related to any stream. 
> > 
> > Then no stream needs them per definition, and they can be droped.
> 
> A decoder may need them to show the subtitles accurately if it does not
> already have the required fonts. However the resources are not
> associated with any particular stream on the container level.
> Libavformat can guess (but NOT know for sure) that attached files which
> have a font type are used by decoders of subtitle streams of type SSA or
> ASS and not by other decoders. The same font may be used by multiple
> subtitle streams.

Well, if the demuxer cannot know which streams need an attachment then so
the matroska file cannot be remuxed with a subset of the streams unless you
also include all (likely) unneeded attachments. That would be a serious
design flaw and hardly one upon which we should base our API.

Also speaking of redundancy between streams, fonts are not special here
extradata will often be identical or similar between streams of the same
codec.


> 
> > > Should I add a fake stream with
> > > attachments as extradata ? 
> > 
> > this does not sound reasonable ...
> 
> I think this would be the least bad fit if you want to force the extra
> resources into existing libavformat data structures.

Its not so much existing libavformat data structures, than existing container
formats. I can add a new data structure in libavformat, but how is the avi, 
asf, mov, ... file generated from it going to work?


[...]
> > The only way to see fonts as special relative to global data (=extradata)
> > would be if one argues that a subtitle decoder outputs UTF-8 + position
> > + font name + color + size + ...
> 
> You can't make an exhaustive list of properties easily. Position cannot
> be expressed with simple coordinates for some fixed part of rendered
> text unless the decoder knows the exact rendered size in advance. There
> can be rotated text, different kinds of border, and various special
> effects (more of which can appear in the future).

The data structure isnt cast in stone and can be extended as new things appear
in the future.
Any difficulty in making an exhaustive list exists for writing a (needed)
subtitle decoder/renderer already. If you cant do that you cant decode the
subtitle either way.


> 
> > anyway comments welcome!
> > but most important is that we need generic solutions, not something which
> > can only be used with mkv-ass (= no hacks)
> 
> No other container can properly handle SSA/ASS with fonts AFAIK, so this
> requirement is rather vague. In practice no way of handling fonts (or
> other extra resource files) can be used with other containers before
> there's a spec on how to store such data in those containers. I think
> trying to make an overly general solution is not a good idea as long as
> the only real use will be with mkv+SSA/ASS. Better make something which
> works for that particular use and worry about a more general solution
> later when there's a need for it. Trying to guess now what kind of extra
> generality will be needed later is unlikely to produce a good result.

Simply duplicating the fonts in extradata would work (with respect to fonts)
with most containers and a libavcodec based ASS/SSA decoder (->it also would
work in all libavcodec based players even ones not using libavformat)

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Let us carefully observe those good qualities wherein our enemies excel us
and endeavor to excel them, by avoiding what is faulty, and imitating what
is excellent in them. -- Plutarch
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080102/abd5c539/attachment.pgp>