[FFmpeg-devel] [PATCH] [RFC] avformat/options_table: do not merge sidedata by default

[wm4]

On Wed, 20 Nov 2013 21:21:25 +0100
Reimar Döffinger <Reimar.Doeffinger at gmx.de> wrote:

> On Wed, Nov 20, 2013 at 08:53:14PM +0100, Reimar Döffinger wrote:
> > On Wed, Nov 20, 2013 at 08:43:03PM +0100, wm4 wrote:
> > > Side data seems like a nice way to communicate parameter changes and
> > > other out-of-band data to the decoder side. How else would you handle
> > > parameter changes?
> > 
> > By putting them in-band of course.

By the way, if you dislike side data as out-of-band mechanism so much,
maybe you should have a look at the OGG metadata patches by Ben
Boeckel, which uses side data to notify the libavformat user of
mid-stream tag updates.

> > Putting data critical for decoding out-of-band, in particular when
> > there is no standard for doing so and thus "out-of-band" is actually
> > "oops, it got lost somewhere" is pretty much what I consider braindead.

Exactly, this is used in cases where there is no standard. For raw
data, it wouldn't make much sense to invent "extra" headers, and for
compressed data it would be stupid too, because it simply isn't part of
the payload.

Besides, reallocating and copying all the packet data is going to be
inefficient too. (Though hopefully side data is actually rather rare.)

> > That it ended up getting used for everything just because it's easy
> > doesn't make it better.
> 
> I am probably being a bit too harsh here, but in principle the
> thing is we had the problem "what should we do with containers
> that have out-of-band data".
> We "solved" that by introducing an out-of-band mechanism inside
> FFmpeg.
> I say "solved" because all it really did was move the problem
> one step further to "I have out-of-band data, what should I do
> with it when the container/filter/application/whatever doesn't
> support it".
> From my point of view, the current answer to that is "make it
> in-band data".
> This patch instead makes the answer to that question "just drop it".
> But are crappy solutions.
> The second somewhat more so though IMHO.
> But I do understand your other points, so I don't think the
> patch makes things exactly worse.
> Still, I think in the end it is just moving the dirt from
> under carpet A to under carpet B.

I agree that it would be better if there was no side data. But if you
look at the way things were done before, side data is a clear
improvement. (Before, decoders just updated the fields in the
AVStream's AVCodecContext struct, and it was hard to tell after which
packet the fields had to be updated, and a general pain to any software
that used a new AVCodecContext instead of AVStream.codec.)

If anything, side data somewhat rectifies the (IMHO) initial design
mistake to make demuxer and decoder directly share the same data
structure.

Keeping side data in-band or out-of-band is orthogonal to this issue.
I'll just say that I think merging out-of-band data into the payload is
inelegant, because if it weren't, the designers of the codec would have
put it in-band as well.