[FFmpeg-devel] Seeking in Apple HTTP Live Streaming

[Takis Issaris]

Hi,

----- Original Message -----

> From: Michael Niedermayer <michaelni at gmx.at>
> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> Cc: 
> Sent: Tuesday, November 22, 2011 1:20 AM
> Subject: Re: [FFmpeg-devel] Seeking in Apple HTTP Live Streaming
> 
> On Mon, Nov 21, 2011 at 04:14:03PM -0800, Takis Issaris wrote:
>> 
>> 
>>  ----- Original Message -----
>>  > From: Michael Niedermayer <michaelni at gmx.at>
>>  > To: FFmpeg development discussions and patches 
> <ffmpeg-devel at ffmpeg.org>
>>  > Cc: 
>>  > Sent: Tuesday, November 22, 2011 12:40 AM
>>  > Subject: Re: [FFmpeg-devel] Seeking in Apple HTTP Live Streaming
>>  > 
>>  > On Mon, Nov 21, 2011 at 03:01:43PM -0800, Takis Issaris wrote:
>>  >>  Hi Michael,
>>  >> 
>>  >> 
>>  >>  ----- Original Message -----
>>  >>  > From: Michael Niedermayer <michaelni at gmx.at>
>>  >>  > To: FFmpeg development discussions and patches 
>>  > <ffmpeg-devel at ffmpeg.org>
>>  >>  > Cc: 
>>  >>  > Sent: Monday, November 21, 2011 8:34 PM
>>  >>  > Subject: Re: [FFmpeg-devel] Seeking in Apple HTTP Live 
> Streaming
>>  >>  > 
>>  >>  > On Mon, Nov 21, 2011 at 08:21:28AM -0800, Takis Issaris 
> wrote:
>>  >>  >>  Hi all,
>>  >>  >> 
>>  >>  >> 
>>  >>  >>  As a follow-up to my previous patch, I'd like to 
> implement 
>>  > more exact 
>>  >>  > seeking on HTTP Live Streams. Currently, when seeking to a 
> specific 
>>  > timestamp in 
>>  >>  > such a stream, the seeking results in arriving at the start 
> of the 
>>  > correct 
>>  >>  > segment, but not at the requested timestamp within that 
> segment. 
>>  > Worse, when 
>>  >>  > seeking to a specific timestamp which happens to be a part 
> of the 
>>  > segment which 
>>  >>  > is currently being played, the playback resets to the start 
> of the 
>>  > current 
>>  >>  > timestamp (that could most likely be fixed rather easily). 
> So, when 
>>  > pressing 
>>  >>  > forward arrow key when using ffplay, you'll actually 
> move 
>>  > backwards to the 
>>  >>  > start of the segment (if current timestamp+10s lies within 
> the current 
>>  > segment).
>>  >>  >> 
>>  >>  >> 
>>  >>  >>  I'd like to change that so that the applehttp.c 
> seeking 
>>  > implementation 
>>  >>  > uses mpegts.c to actually seek to the specified timestamp, 
> but I'm 
>>  > not sure 
>>  >>  > on how to implement this.
>>  >>  >>  Currently, when seeking using applehttp_read_seek, I 
> think 
>>  > there's no 
>>  >>  > valid MPEG-TS demuxer context in that function. Should I 
> make sure I 
>>  > get a valid 
>>  >>  > demuxer there, or should I defer the actual seeking using 
> the MPEG-TS 
>>  > demuxer 
>>  >>  > until a valid demuxer is constructed elsewhere in the 
> applehttp.c 
>>  > file?
>>  >>  > 
>>  >>  > I assume theres no way to tell the server to start a segment 
> from a
>>  >>  > specific time? (i havnt found one)
>>  >> 
>>  >>  No, I don't think so (haven't found one either).
>>  >> 
>>  >> 
>>  >>  > Do the servers support seeking to a specific byte position 
> within a
>>  >>  > segment ?
>>  >> 
>>  >>  Yes, it's just an Apache HTTP server, so it supports GET with 
> a byte 
>>  > range (If that
>>  >>  was what you were hinting at).
>>  >> 
>>  >> 
>>  >>  > If yes, seeking per CBR assumption is an option which also 
> doesnt need
>>  >>  > the demuxer
>>  >> 
>>  >>  Yes, initially I'd never even considered CBR. But as I start 
> to think 
>>  > about it,
>>  >>  the seeking speedup could counterweight the quality/bitrate 
> disadvantage.
>>  >> 
>>  >> 
>>  >>  > The reason why i suggect to assume CBR is that the 
> alternative is
>>  >>  > the default seek per timestamp which does a binary search 
> and this
>>  >>  > could be quite slow with http needing to send a few packets 
> back and
>>  >>  > forth for each iteration of the search
>>  >> 
>>  >> 
>>  >>  Yes, but the draft recommends using 10 second segments, so 
> I'd hoped 
>>  > that
>>  >>  the actual number of HTTP GETs needed would be rather limited.
>>  >> 
>>  >>   
>>  >>  > depending on network speed, bitrate and segment size a 
> linear search
>>  >>  > could also be considered, that is demux and throw away 
> packets until
>>  >>  > the right time, this may or may not be faster than a binary 
> search
>>  >> 
>>  >>  Yes, I'd thought about doing something like that as the 
> applehttp.c 
>>  > seek 
>>  >>  already skips to the correct 10s segment, so a linear search 
> within that 
>>  > should
>>  >>  not take that long.
>>  >> 
>>  >>  Would something like that be useful in applehttp.c or should it 
> be kept in 
>>  > application
>>  >>  specific code?
>>  > 
>>  > requireing applehttp specific code in every app does not seem a good
>>  > idea to me thus IMHO its better in applehttp.c
>> 
>>  Agreed. Should the seeking be completed after returning from 
> applehttp.c's
>>  seeking call or would it be acceptable that the actual search happens 
> elsewhere?
>>  (f.e. somewhere near the read frame call or near the open url call).
> 
> do whatever makes most sense technically
> 
> [...]


I had a look at this last Tuesday, and only did some experiments, but,
nevertheless, I'd rather post the code and get your opinion on the approach, then let it
rot for another week :-/

It is in no way clean nor did I take into account possible variants, MPEG-TS
files with many streams etc. But, if the general approach is okay (that is, storing
seeking state in applehttp_read_seek() and afterwards skipping packets in
applehttp_read_packet(), I'll start cleaning and correcting the code for all cases.

So, I'm obviously not suggesting committing this :-)

In the patch ending in v0p2.diff, I tried to allow seeking to keyframes, but didn't know how to do this cleanly, thus the horrible hack. Should the MPEG TS demuxer be modified to set AV_PKT_FLAG_KEY in pkt.flags?


With friendly regards,
Takis
-------------- next part --------------
A non-text attachment was scrubbed...
Name: phmi-20111128T1324-ffmpeg-hls_seek_timestamp_v0.diff
Type: application/octet-stream
Size: 2980 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20111128/c93d32ca/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: phmi-20111128T1324-ffmpeg-hls_seek_timestamp_v0p2.diff
Type: application/octet-stream
Size: 3002 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20111128/c93d32ca/attachment-0001.obj>