[FFmpeg-devel] [PATCH] attachments support in matroska demuxer

[Uoti Urpala]

On Thu, 2008-01-03 at 03:18 +0100, Michael Niedermayer wrote:
> On Thu, Jan 03, 2008 at 02:47:09AM +0200, Uoti Urpala wrote:
> > On Thu, 2008-01-03 at 00:12 +0100, Michael Niedermayer wrote:
> > > On Wed, Jan 02, 2008 at 09:10:22PM +0200, Uoti Urpala wrote:
> > > > On Wed, 2008-01-02 at 11:21 +0100, Michael Niedermayer wrote:

> > you create a separate "generalized" hack for subtitles there are still
> > other attachment types. Do you think that lavf should not provide access
> > to Matroska attachments in general (at a level which does not care what
> > the application wants to use those attachments for)?
> 
> I do want attachments to be exported to the application in some _CLEAN_ way,
> hey this is even the topic of this thread.
> But i guess i can say whatever i want you will keep asking questions as if i
> said something i didnt say.

You did oppose exporting the attachments as is and wanted things like
copying font attachments to the extradata of every subtitle track
instead. If that was the only way to access the attachments it wouldn't
work for a program that wants to access the attachments for some other
reason.

> > > Could you please refrain from trolling.
> > 
> > My reply is no more trolling than your "the decoding is not done in lavc
> > so nobody needs to demux them either". Do you actually have a better
> > reason for your comment than those I suggested?
> 
> Iam just trying to make it clear that we need a clean solution. Iam not
> seriously planing to not support attachments.
> But threatening with that makes people think about the half baked mess they
> proposed, hey i even got you to write some comments.

Your own description of what you tried to do sounds pretty much like the
definition of trolling.

>  Even if they are
> 90% trollinga and flaming.

I think most of my comments in this thread are in no way trolling or
flaming.

> > And even if you think
> > you'll want a more general solution later that doesn't mean you should
> > do it that way now when you don't need it yet.
> 
> Many applications use lavc and lavf, changing the API continuously is quite
> annoying for most of their developers. Its also a lot of extra work for them 
> to repeatly change the code used to access lavc/lavf.

Any changes will only be needed after there is some actual use for more
general functionality. And generalizing the API in advance does not
guarantee you'll avoid changes. That would only happen if you get it
right before you know how other containers will actually implement the
functionality (if at all).

> > > > > Lets say if the system doesnt work with avi/mov/nut its rejected.
> > > > 
> > > > Work to give what result? What are you supposed to get when you try to
> > > > remux an mkv with attachments to avi?
> > > 
> > > a avi which when decoded results in bit identical video+subs
> > 
> > It's not possible to make an avi which would give identical results. To
> > get that you'd first need to extend specs and then change decoders.
> 
> let me simplify that
> "its not possible. To make it possible you would ..."
> 
> Is it just me or is that what you say selfcontradicting? And iam not claiming
> that would be limited to the above 2 sentences.

Here's a spelled out version:
Given current specs and existing decoders it is not possible to create
an avi file which would behave the way you want. So it is impossible to
create any system of exporting Matroska attachments that would make
remuxing to avi work (as such avi files simply cannot exist).

So to get something that could be called "avi" and has the desired
functionality you would first need to abandon the "given current specs
and existing decoders" premise, extend the specs and change decoders to
support the new specs. I think it is unreasonable to require extending
specs related to other container formats as a precondition to supporting
some Matroska feature.

> > Your reply makes no sense. Of course separating .ass does not differ
> > >from separating audio and video; the point is that it's natural to split
> > to an .ass file + font files, whereas nobody splits to a video extradata
> > file plus separate video main content file. And an .ass track without
> > (all) fonts attached still makes some sense, whereas nobody muxes video
> > tracks without the extradata.
> 
> Just compare fonts to some refinement streams (mpeg/jpeg/jpeg2k scalability)
> They both are optional and improve presentation if available.

Those refinement streams make no sense as independent objects, and are
very unlikely to be shared between tracks.

> Your arguments are based on the idea that mkv attachments and fonts (2
> distinct things implemented with the same syntax) are mkv specific and
> dont exist in other containers. This is false.

I'm not sure which arguments you're referring to. I've made several
arguments in this thread to which your claim obviously can't apply at
all.

> Still you skip over the presented facts and continue to claim we should
> ignore the other containers. And design a API based on how mkv handles
> fonts and attachments

Here's an overall description of my view:
- There should be a way to access Matroska attachments directly (the
same API could likely to be used for generic attachments in other
containers). This should still exist even if there was a separate way to
handle fonts, or other higher-level way(s) to access resources Matroska
can store in attachments.
- Such an attachment API would be reasonably easy to implement.
- Even if an attachment-specific API is not suitable for storing fonts
in every container the practical benefit of creating a different API for
fonts (and possibly other hypothetical functionality) is very limited as
long as fonts are not in practice stored in other containers.
- Getting fonts through an attachment-specific API does not require much
code. Nor is there any other obvious reason why it would be more
important to make sure no changes are needed in the future than to make
the current implementation simple and practical.
- Given the above it's best to add an attachment API to libavformat now
and leave the semantics of the attachments to the calling application.

So I'm not advocating creating a "font API" based on how mkv does it.
Rather I'm saying that a simple "just give the attachments" API is the
most practical way to do everything people want with currently existing
files, and a more abstracted (and complicated to implement) API should
only be added later if the situation changes so that such an API would
give some real practical benefit.