[FFmpeg-devel] qt-faststart update

[Baptiste Coudurier]

Hi,

Frank Barchard wrote:
>> [...]
>>
>>> Why?
>>> When decoding mp4 files with ffmpeg, a seek to end of file slows down
>>> startup latency.
>> How much is the latency ?
> 
> 
> The worst case is a browser or protocol that doesnt handle seeks and
> will have to cache the entire file before the movie can start.
> Depends on the connection, but on the order of 30 seconds.
> The next case is a protocol that can handle seeks, but needs to open a
> new connection.  Roughly 2 seconds.
> The best case is harddrive or really fast protocol, where the savings
> is about 150 milliseconds.
> This change doesnt improve on savings - it just makes it possible for
> ffmpeg to work efficiently with content created by mp4box, quicktime
> and other tools.

I understand, there are 2 cases:
Protocol not handling seeking:
            err = parse(c, pb, a);
            if (url_is_streamed(pb) && c->found_moov && c->found_mdat)
                break;
Parsing will stop.

Protocol having slow seeking:
This case is a bit obscure IMHO. In this case it might be nice to
instruct the demuxer that it should not attempt to seek.

Now we could add a way for the user to specify that even if the protocol
can seek, demuxer must not try to.

>>> qt-faststart moves the 'moov' header atom to the beginning of the file.
>>>  Which works great if you use ffmpeg to mux, but if you use quicktime or
>>> mp4box to mux, ffmpeg demux will still seek to end of file?
>> Well, I believe the demuxer could be changed to avoid this case, which
>> would
>> avoid modifying a ton of files if only the seek is causing problems,
>> on the other end, seek can be wanted, see below.
> 
> 
> Yes, I think we should look at that too.   But the library change
> is riskier, at least for a newbie like me, and doesn't eliminate the need
> for qt-faststart.
> It would only help with mp4box files, not ffmpeg or quicktime muxed files,
> where the moov header can still be at the end.
> 
> 
>>
>>> Unless you use url_is_streamed, the open will scan thru all the atoms,
>> and
>>> if anything follows mdat, a far seek will occur.
>>> In practice, 'free' atoms are being added by to the end of file.   The
>> atoms
>>> are not required and removing them solves the seek.
>> Do you have statistics on the presence of only 'free' atoms at the end of
>> files ?
> 
> 
> This took awhile to gather.  I've only got 41 interesting mov/mp4 files
> handy... they tend to come from vimeo or apple.
> 
> 29 of the files had a free atom, but not necessarily at the end.
> 
> 1 had a free atom at end, and mdat just before that.  So it was simply
> truncated.
> 
> 20 had moov after the mdat
> 8 of those had a free atom before the moov atom.
> 1 of those 20 with moov at the end, had a free atom after the moov.
> 1 of those 20 also had a uuid near the start of file, and was correctly
> fixed up.
> 
> 1 file had a uuid at end of file, and was not fixable.  The moov header is
> at the start of file, but ffmpeg will still seek to end of file to read the
> uuid atom.
> 
> That leaves 20 of the 41 files that were already okay.

Ok, that's 50%. Thanks for the statistics.

>>
>> 'mdat' can be followed by fragments and it will be better for the
>> demuxer to parse them at the beginning if file is seekable.
>> You can also have useful userdata which the user will definitely want,
>> no matter the structure of the file, so you might want to seek
>> at end of file to verify, I don't know much it happens in the wild though.
> 
> 
> qt-faststart maintains those atoms.  It'll move the 'moov' and keep
> everything else.
> But those other atoms will cause runtime seeks.
> I haven't seen this in practice.
> If anyone runs into a case where qt-faststart could help, I'd be happy to
> get sample data and try to fix it.

Ok.

>>
>>> How?
>>> The old code would move the 'moov' header after 'ftyp', adjust stco
>> offsets,
>>> and output the rest of the file.
>>> It would fail if the last atom was not 'moov'.
>> That's a fix to the code, and would be very welcome as a separate
>> patch which only fix this.
> 
> 
> There are 4 variations of this, and it would be hard to backtrack and
> isolate the changes now.
> One case is the moov is near the end, but followed by a 'free'.
> Another is the moov is near the end, but preceeded by a 'free'.  This would
> work, but it was necessary to call the tool twice to get rid of the trailing
> free.
> And the last is the last atom is a 'free' atom and the tool needs to
> continue with just the copy rest of file.
> Another is just error handling, if its not a valid case and it was crashing.

Hummm let's see the diff :)

>>> [...]
>>>
>>> Future work
>>> Instead of moving the 'moov' to the beginning, it would be better to move
>>> the 'mdat' to the end.  This would stop other atoms, such as 'uuid' from
>>> causing seeks to the end.
>> Yes, indeed, beware that you can have multiple 'mdat' and fragments
>> which should not be moved.
> 
> 
> I assumed that could happen, but in practice, large files use 64 bit atoms.
> The behavior hasn't changed in this respect.
> The over file content is moved by the size of the 'moov' header and
> relocation only needs to know how much the overall file shifted.

I have a bunch of files having 2 'mdat' at least. One at beginning, one
at the end. Mainly because Final Cut Pro puts timecode data packet at
the end in a seperate 'mdat'

I'll review the patch tomorrow if nobody beats me.
Need some sleep :)

-- 
Baptiste COUDURIER                              GnuPG Key Id: 0x5C1ABAAA
Key fingerprint                 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA
FFmpeg maintainer                                  http://www.ffmpeg.org