[FFmpeg-devel] [RFC/PATCH] Pass PRIVATE_STREAM_2 MPEG-PS packets to caller

[Richard]

On 24/02/13 22:37, Michael Niedermayer wrote:
> On Sun, Feb 24, 2013 at 09:56:05PM +0100, Richard wrote:
>> On 24/02/13 21:50, Richard wrote:
>>> On 24/02/13 21:06, Michael Niedermayer wrote:
>>>> On Sun, Feb 24, 2013 at 08:31:35PM +0100, Richard wrote:
>>>>> On 18/02/13 07:24, Richard wrote:
>>>>>> Hi,
>>>>>>
>>>>>> In order to improve DVD playback, I need the DVD NAV packets in-sync
>>>>>> with the other audio/video/subtitle packets.  The information is stored
>>>>>> in 'PRIVATE_STREAM_2' MPEG-PS packets (startcode 0x1bf), which are
>>>>>> currently filtered out.
>>>>>>
>>>>>> The attached patch adds a new codec ID
>>>>>> 'AV_CODEC_ID_PRIVATE_STREAM_2' to
>>>>>> identify these packets.
>>>>>>
>>>>>> The contents of these packets is clear for DVDs but looking at the
>>>>>> existing code, it appears that Dreamcast videos (Sofdec) also use this
>>>>>> startcode but for other purposes, so I'm not sure it's feasible to
>>>>>> create any sort of decoder for these packets.  As it is, the patch
>>>>>> simply returns the contents of the packet unchanged.  It is up to the
>>>>>> calling application to process the contents.
>>>>>>
>>>>>> All comments welcome.
>>>>>
>>>>> Any comments on this?
>>>>
>>>> do you have an actual application that uses this ? that is has the
>>>> output been tested somehow ?
>>>
>>> Yes, I'm working on improving DVD playback in MythTV.  There is a wealth
>>> of information stored in these packets that lets the player code
>>> determine, for example, when there's a timecode discontinuity, or when
>>> the next block with video packets will come.  Particularly because of
>>> the timecodes discontinuities, it's important to get these packets
>>> *before* the discontinuities occur to be able to anticipate them.
>>>
>>> The contents of these packets (in a DVD context) is describe on these
>>> two pages (just to give an idea of the sort of data available):
>>>
>>> http://dvdnav.mplayerhq.hu/dvdinfo/pci_pkt.html
>>> http://dvdnav.mplayerhq.hu/dvdinfo/dsi_pkt.html
>>>
>>> Ideally, these packets would be delivered completely in-sync with the
>>> audio/video/subpicture packets, but that doesn't seem to be possible due
>>> to, as I mentioned, the buffering required for the 'normal' packets.  If
>>> there is a way, I'm all ears.
>>
>> Sorry, in response to the question 'Has the output been tested
>> somehow?', yes, I've tested this in Myth.  I don't have the code
>> written yet to handle the changes but I can confirm that a new data
>> stream is detected and that the packets are returned via
>> av_read_frame.
>
> what i wanted to know is if use of the packets has been tested, above
> sounds like a "no"
> and also how they would be used ?

The contents of the packets are already used (directly or indirectly) by 
any program using libdvdnav.  libdvdnav parses the contents into 
structures but the problem I'm having is getting this information to the 
part of the code that calls 'av_read_frame'.  That call eventually lands 
in code that queries libdvdnav.  This, in turn, will usually just return 
the next 2048 byte block from the DVD, which will contain video, audio 
or subpicture data.  It will, however, return a 'NAV' packet 
approximately twice a second.  This is a sector containing the two 
packets marked with the 'private stream 2' start code (also pack header 
- 0x1ba, and system stream - 0x1bb, but they don't appear to contain 
anything too useful at the moment).

> ATM this patch smells wrong but i dont know what its used for so i
> could be wrong and sure cant recommand how to improve it when i dont
> even know what this is used for exactly

Specifically, I started with the problem of playing audio DVDs (in the 
sense of a normal video DVD with primarily only audio).  These have a 
single video frame followed by an arbitrary number of audio frames.  At 
the moment, the demultiplexing code in Myth ends up reading and 
buffering too much audio data because it is normally slowed down by the 
availability of video buffers.  Now it could be argued that it should be 
making more use of the timecodes in the audio and video packets but that 
is also difficult when discontinuities are to be expected and there's no 
way to ensure the demultiplexer/decoder is informed about the 
discontinuity ahead of time (ideally 'now' but certainly not afterwards).

For my in-depth ideas with respect to single images with audio, I can 
point you here:
http://irc.mythtv.org/ircLog/channel/4/2013-01-24:15:41

> for example random buffering + no timestamps just feels wrong also
> this feels a bit like a hack, to output a private stream raw like
> that. we dont do that for audio or video either, there you get clean
> packets one for each frame with timestamps, a codec id and various
> other things

The problem is that the contents of packets with startcode '0x1bf' is 
not uniquely defined.  In the context of DVDs, it is, but given any MPEG 
program stream, it isn't.  That makes it difficult to implement a 
parser.  The PCI structure indicates the start and end PTS values for 
the following VOBU, so the start PTS could be used, but the DSI 
structure only references the system clock reference.  It might be 
possible to write a parser to decode the contents based on 'if this byte 
is X and that byte is Y and those bytes are all zero, then we've almost 
certainly got a PCI structure' but I'm not convinced that is that much 
cleaner.  The contents are simply undefined so there'll always be a 
chance of false-positives and false-negatives.

The code in mpeg.c also indicates that these packets are used in 
'Sofdec' videos, but obviously not for the same purpose.

The 'random buffering' is more a function of how lavf seems to work. 
The problem is that 'chunks' of video or audio frames must be buffered 
until a complete frame is available.  Only then can they be returned via 
av_read_frame.  Packets with 'private stream 2' startcodes are small 
enough that they don't need buffering and so are passed back immediately 
(also, since the contents are undefined, there's no way to determine in 
lavf whether two or more chunks should be combined).  This means that 
they end up being returned somewhat early.  I'm not that clear on *why* 
it is, but I've been seeing these new packets being returned, closely 
followed by five or six AC3 frames and up to two video frames, even 
though all those frames had been completed before the data packets were 
even read.

In the end, it's this buffering that makes it near impossible to get the 
information in these data packets back to the main application via any 
other route under all circumstances.  It ends up being an unreliable 
hack in the application code to get round the fact that the data packets 
are filtered out in lavf.

In the end, I may well just use the receipt of one of these packets as a 
trigger to get pre-parsed data from the DVD code.  The problem, as I 
said, is ensuring you get them at all.

I see this subject has been discussed before:

http://ffmpeg.org/pipermail/ffmpeg-devel/2009-April/060342.html

but it appears that nothing came of it (at least, the thread peters out 
and I didn't find any other references).

I'm open to any suggestions for a better solution.

Richard.