[FFmpeg-devel] [PATCH 12/18] avformat/hls: parse ID3 timestamps for elementary audio streams

[Anssi Hannula]

30.12.2013 16:12, Michael Niedermayer kirjoitti:
> On Mon, Dec 30, 2013 at 01:14:26PM +0200, Anssi Hannula wrote:
>> HLS provides MPEG TS timestamps via ID3 tags in the beginning of each
>> segment of elementary audio streams.
>>
>> Signed-off-by: Anssi Hannula <anssi.hannula at iki.fi>
>> ---
>>
>> This is a bit hacky, but I could not see any better way.
> 
> this seems to break some hls streams
> for example:
> http://qthttp.apple.com.edgesuite.net/c123pibhargjknawdconwecown/0064/prog_index.m3u8
> 
> didnt investigate why

The ID3 tag in the beginning of each segment seems to contain an
attached picture (APIC) in addition to the timestamp, so the ID3 tag is
too large for the id3 hack to fully intercept (since the hack currently
depends on the ID3 tag fitting in the caller-provided buffer), so the
caller gets 0 bytes.

The APIC is currently parsed to a static mjpeg stream by
ff_id3v2_parse_apic() in avformat_open_input(). I guess it could
theoretically change mid-stream as well, though it might not currently
happen in practice (no idea, though [1](13) suggests it does not).

Before the patch it works because avformat_open_input() reads metadata
in the beginning of input. It of course will not detect the mid-stream
metadata changes (in the beginning of segments), if there are such streams.

I guess there are several alternative ways to fix this:
a) Make the hack properly request additional bytes into its own buffer
   so that the entire ID3 tag can be read, and get the picture shipped
   over to the attached picture stream (and maybe any changed text
   metadata as well?). Not sure how that should work with changing
   pictures, though, as AV_DISPOSITION_ATTACHED_PIC says "single
   packet". Or alternatively just not create it is an attachment in the
   first place, but as a regular video stream. Well, I guess for now
   I could just ignore all data after the first ID3 tag, and deal
   with the changing stuff if it appears...
or
b) Assume the timestamp we want is in the beginning of the packet (so
   we do not need to request any additional bytes) and then pass the
   full ID3 tag along after extracting the timestamp we wanted. However,
   I guess that is a pretty big assumption, and of course this way we
   miss any metadata in mid-stream ID3 tags (though that is a minor
   problem as such streams may not exist).
or
c) Some entirely different solution to this ID3 timestamp thing
   altogether, preferably something that would also allow us to drop
   the manual recalculation of timestamps in this patch (by leveraging
   the lavf timestamp regeneration code).
  I)
   I already dabbled a bit with a private HLS ID3 tagged audio demuxer
   but the issue was that from the perspective of such a demuxer, the
   position of the ID3 tags is only known by the AVIOContext, and I
   could see no easy way to communicate the position of ID3 tags to
   the demuxer externally (well, I guess I could rather easily just
   access the private demuxer priv_data.. but that would again be
   quite hacky and probably would make no sense to have the HLS ID3
   audio demuxer even registered publicly in allformats.c..)
  II)
   Hmm, continuing on the subdemuxer idea, maybe a HLS ID3 audio demuxer
   could even handle the segment downloading itself so it would then
   know where to expect ID3 tags, sharing code with the main HLS demuxer
   but without any extra communication between them.
   I don't see how the probing would work, exactly, though... i.e.
   how would the HLS ID3 subdemuxer be selected? Just probing for the
   ID3 timestamp tag in probe() would not be enough, since at that point
   the main HLS demuxer is already handling segments via custom
   AVIOContext, so it would have to handover somehow, which is again
   in the hack category... damn, I thought I had it this time.


I guess I'm going to look at implementing (a), unless you can devise a
workable idea along the lines of (c)...


[1]
https://developer.apple.com/library/ios/documentation/networkinginternet/conceptual/streamingmediaguide/FrequentlyAskedQuestions/FrequentlyAskedQuestions.html

-- 
Anssi Hannula