[FFmpeg-devel] qt-faststart update

[Frank Barchard]

On Wed, Jun 24, 2009 at 2:24 AM, Baptiste Coudurier <
baptiste.coudurier at gmail.com> wrote:

> Hi,
>
> Frank Barchard wrote:
> >> [...]
> >>
> >>> Why?
> >>> When decoding mp4 files with ffmpeg, a seek to end of file slows down
> >>> startup latency.
> >> How much is the latency ?
> >
> >
> > The worst case is a browser or protocol that doesnt handle seeks and
> > will have to cache the entire file before the movie can start.
> > Depends on the connection, but on the order of 30 seconds.
> > The next case is a protocol that can handle seeks, but needs to open a
> > new connection.  Roughly 2 seconds.
> > The best case is harddrive or really fast protocol, where the savings
> > is about 150 milliseconds.
> > This change doesnt improve on savings - it just makes it possible for
> > ffmpeg to work efficiently with content created by mp4box, quicktime
> > and other tools.
>
> I understand, there are 2 cases:
> Protocol not handling seeking:
>            err = parse(c, pb, a);
>            if (url_is_streamed(pb) && c->found_moov && c->found_mdat)
>                break;
> Parsing will stop.


Exactly - Thats the interesting one.  Could that change:?

From:
if (url_is_streamed(pb) && c->found_moov && c->found_mdat)

To
if (c->found_moov && c->found_mdat)

Theres another case, but its less interesting because its near seeks, which
simple buffering can handle.
The downside to url_is_streamed on mov/mp4 is that the duration of the video
is unknown?  Haven't looked into why.

The current behavior of url_is_streamed is not overly strict.  It coaxed the
demux to avoid seeks, but in the case where a seek really is necessary (ie
seek to middle of video), it allows the seek.


>
> Protocol having slow seeking:
> This case is a bit obscure IMHO. In this case it might be nice to
> instruct the demuxer that it should not attempt to seek.
>
> Now we could add a way for the user to specify that even if the protocol
> can seek, demuxer must not try to.
>
> >>> qt-faststart moves the 'moov' header atom to the beginning of the file.
> >>>  Which works great if you use ffmpeg to mux, but if you use quicktime
> or
> >>> mp4box to mux, ffmpeg demux will still seek to end of file?
> >> Well, I believe the demuxer could be changed to avoid this case, which
> >> would
> >> avoid modifying a ton of files if only the seek is causing problems,
> >> on the other end, seek can be wanted, see below.
> >
> >
> > Yes, I think we should look at that too.   But the library change
> > is riskier, at least for a newbie like me, and doesn't eliminate the need
> > for qt-faststart.
> > It would only help with mp4box files, not ffmpeg or quicktime muxed
> files,
> > where the moov header can still be at the end.
> >
> >
> >>
> >>> Unless you use url_is_streamed, the open will scan thru all the atoms,
> >> and
> >>> if anything follows mdat, a far seek will occur.
> >>> In practice, 'free' atoms are being added by to the end of file.   The
> >> atoms
> >>> are not required and removing them solves the seek.
> >> Do you have statistics on the presence of only 'free' atoms at the end
> of
> >> files ?
> >
> >
> > This took awhile to gather.  I've only got 41 interesting mov/mp4 files
> > handy... they tend to come from vimeo or apple.
> >
> > 29 of the files had a free atom, but not necessarily at the end.
> >
> > 1 had a free atom at end, and mdat just before that.  So it was simply
> > truncated.
> >
> > 20 had moov after the mdat
> > 8 of those had a free atom before the moov atom.
> > 1 of those 20 with moov at the end, had a free atom after the moov.
> > 1 of those 20 also had a uuid near the start of file, and was correctly
> > fixed up.
> >
> > 1 file had a uuid at end of file, and was not fixable.  The moov header
> is
> > at the start of file, but ffmpeg will still seek to end of file to read
> the
> > uuid atom.
> >
> > That leaves 20 of the 41 files that were already okay.
>
> Ok, that's 50%. Thanks for the statistics.
>
> >>
> >> 'mdat' can be followed by fragments and it will be better for the
> >> demuxer to parse them at the beginning if file is seekable.
> >> You can also have useful userdata which the user will definitely want,
> >> no matter the structure of the file, so you might want to seek
> >> at end of file to verify, I don't know much it happens in the wild
> though.
> >
> >
> > qt-faststart maintains those atoms.  It'll move the 'moov' and keep
> > everything else.
> > But those other atoms will cause runtime seeks.
> > I haven't seen this in practice.
> > If anyone runs into a case where qt-faststart could help, I'd be happy to
> > get sample data and try to fix it.
>
> Ok.
>
> >>
> >>> How?
> >>> The old code would move the 'moov' header after 'ftyp', adjust stco
> >> offsets,
> >>> and output the rest of the file.
> >>> It would fail if the last atom was not 'moov'.
> >> That's a fix to the code, and would be very welcome as a separate
> >> patch which only fix this.
> >
> >
> > There are 4 variations of this, and it would be hard to backtrack and
> > isolate the changes now.
> > One case is the moov is near the end, but followed by a 'free'.
> > Another is the moov is near the end, but preceeded by a 'free'.  This
> would
> > work, but it was necessary to call the tool twice to get rid of the
> trailing
> > free.
> > And the last is the last atom is a 'free' atom and the tool needs to
> > continue with just the copy rest of file.
> > Another is just error handling, if its not a valid case and it was
> crashing.
>
> Hummm let's see the diff :)
>
> >>> [...]
> >>>
> >>> Future work
> >>> Instead of moving the 'moov' to the beginning, it would be better to
> move
> >>> the 'mdat' to the end.  This would stop other atoms, such as 'uuid'
> from
> >>> causing seeks to the end.
> >> Yes, indeed, beware that you can have multiple 'mdat' and fragments
> >> which should not be moved.
> >
> >
> > I assumed that could happen, but in practice, large files use 64 bit
> atoms.
> > The behavior hasn't changed in this respect.
> > The over file content is moved by the size of the 'moov' header and
> > relocation only needs to know how much the overall file shifted.
>
> I have a bunch of files having 2 'mdat' at least. One at beginning, one
> at the end. Mainly because Final Cut Pro puts timecode data packet at
> the end in a seperate 'mdat'


I'd love to have a copy of final cut, but cant afford $1299 price tag.  I
have iMovieHD 08.
One video I exported is 11 GB as PNG, but its got the headers at start of
file.

qt-faststart SpinPNG.mov Spin_Fast.mov
ftyp          0 32
moov         32 128319
free     128351 16
mdat     128367 11864313083
Last atom in file was not a moov or free atom.  Skipping.

Do you have a sample with multiple mdats on an ftp or whatnot that I
could access?
Reguardless, if the tool worked before on these, it'll still work.


>
>
> I'll review the patch tomorrow if nobody beats me.
> Need some sleep :)
>
> --
> Baptiste COUDURIER                              GnuPG Key Id: 0x5C1ABAAA
> Key fingerprint                 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA
> FFmpeg maintainer                                  http://www.ffmpeg.org
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at mplayerhq.hu
> https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-devel
>