[FFmpeg-devel] headers for gapless playback

[Marton Balint]

On Sun, 24 Nov 2019, Tomas Härdin wrote:

> tor 2019-11-21 klockan 23:35 +0100 skrev Marton Balint:
>> 
>> On Thu, 21 Nov 2019, Tomas Härdin wrote:
>> 
>> > tis 2019-11-19 klockan 19:16 +0000 skrev John Kaplan:
>> > > Hi Guys,
>> > > 
>> > > We just did some research on gapless playback across multiple client
>> > > devices, and could not find support for adding gapless metadata
>> > > headers in ffmpeg. Our interest is specifically in AAC/MP4, and we
>> > > ended up shifting to the fdk-aac package, which exposes options for
>> > > adding moov/edts/elst headers (which are the ones we used) as well as
>> > > iTunSMPB.
>> > > 
>> > > So, I wanted to open a discussion with the team on ffmpeg support for
>> > > gapless metadata.
>> > > I’m not sure if you guys have discussed before, so here are a few
>> > > opening questions to get me up to speed…
>> > > 
>> > > 
>> > >   1.  Does ffmpeg support addition of gapless metadata in any form
>> > > currently, and we missed the doc as to how to specify it?
>> > >   2.  If not, have you discussed previously? If so, what were the
>> > > results? (I.e. rejected as evil, put on roadmap but in future,
>> > > currently in development, etc.)
>> > 
>> > Something very similar to this is being discussed in 
>> > https://trac.ffmpeg.org/ticket/8366
>> > 
>> > >   3.  If support is still in the future, would you be open to me
>> > > contributing a patch that implements it?
>> > 
>> > Yes please
>> > 
>> > I think opening a generic ticket for this would be a good move. Having
>> > the ability to trim essence would be very useful. Then we could also
>> > remove that godawful hack in mov.c as well, hopefully. There's more
>> > than a few formats that need some "warm-up" data or that leave a few
>> > frames of garbage at the end of the file, like all block transform
>> > audio codecs
>> 
>> For audio we have AV_PKT_DATA_SKIP_SAMPLES packet side data
>
> This is not enough for sample-level cutting as needed for gapless
> playback

Why not? The side data contains the number of samples to be skipped from 
the start and at the end of the packet.

>
>> for video AV_PKT_FLAG_DISCARD packet flag. What else is needed?
>
> Why is this on the packets? It's not the demuxer's job to decide what
> should and should not be cut.

The demuxer only signals this, frames are dropped after they are decoded 
in generic libavcodec decode.c.

Regards,
Marton