[FFmpeg-devel] [PATCH] mov.c log qt ref extenal essence metadata

[Jan Ekström]

On Sat, Mar 6, 2021 at 10:38 AM emcodem <emcodem at ffastrans.com> wrote:
>
> ---
>  libavformat/mov.c | 8 ++++++++
>  1 file changed, 8 insertions(+)
>
> In quicktime reference files, exposing the parsed info for external essences location can be very handy for users
>

Unfortunately, as per the discussion we had yesterday on #ffmpeg, Data
References are not as simple as mov.c might make it feel like.

So, if we think that a single MOV/MP4 Track is:
1. A set of decode'able packets of a certain media type (as far as I
can tell that has been the limitation, while other things can change
during a Track the media type is one that doesn't).
2. Which is then presented according to a virtual time line (edit
lists, which we will in this case ignore since they get applied on top
of the decoded result on the presentation layer, and data references
are on the packet set level).

Thus if we go through the layers:
1. We have samples (packets in FFmpeg parliance more or less, stsz
defines the sizes of them and so forth).
2. Samples get put into chunks, which are basically tuple of
(sample_description_index, data offset) - see stsc, stco, co64.
3. Sample Description can thought of as a tuple of (AVCodec, the
extradata (if any) required, data reference index), there is a list of
them in the Track's stsd box.
4. Then finally we get to the data reference list in the dref box of the track.

Currently as far as I can tell from reading mov_read_stsd /
ff_mov_read_stsd_entries, it does generate extradata buffer for each
sample description, but effectively only keeps a single data reference
around in the MOVStreamContext, skipping the whole chunk matching etc
part of things :) (if I am reading the code correctly, which I might
not be).

So yea, there's two questions:
1. Should this be exposed?
2. If it should be exposed, how? A set of metadata this should not be,
as this at the very least would end up being a weird set/list of byte
offsets/sizes and references :)

So yea, sorry for things not actually being as simple as they look by
the code in mov.c.

Jan