[FFmpeg-devel] [RFC] rtpdec: Reordering RTP packets

[Michael Niedermayer]

On Tue, May 25, 2010 at 10:12:11AM +0300, Martin Storsj? wrote:
> On Mon, 24 May 2010, Michael Niedermayer wrote:
> 
> > On Mon, May 24, 2010 at 05:35:50AM +0200, Luca Barbato wrote:
> > > On 05/24/2010 01:46 AM, Michael Niedermayer wrote:
> > > > this might work with playing a video but for lowdelay communication this
> > > > design fails
> > > 
> > > Let me make a summary:
> > > 
> > > In rtp you have timestamp and sequence number. rtp mandates that every
> > > packets is within the mtu since lower protocols might not be adequate to
> > > handle fragmentation. That makes you have a frame spanning multiple rtp
> > > packets.
> > > 
> > > Sometimes for some reasons you might get misordered packets, the simple
> > > solution in those case is just discard them, the decoder gets garbage at
> > > best and does the concealment. Another solution applicable if you can
> > > tolerate some delay is keep fetching some more packets and once you get
> > > all you need you feed them to the demuxer so the codec gets sane data
> > > with a bit of delay (say 5-10 packets at most)
> > > 
> > > > There normally is some buffer, to compensate variable network delay
> > > > and there is one (possible the same) buffer to compensate bitrate
> > > > variation between frames (and all video since mpeg1 has variable sized
> > > > frames, keyframes are much larger) thats the vbv buffer, its mandatory.
> > > > currently these buffers are in the application and thus cannot be used
> > > > by rtpdec.
> > > > if rtpdec would return a packet if it has one when the application
> > > > requests one then these buffers could be inside rtpdec and rtpdec
> > > > could use them for free for reordering packets until the packets are
> > > > needed by the application.
> > > 
> > > The behavior for the application that is accounting for network glitches
> > > won't change at all. If you don't have misordering you get the frame as
> > > soon you get the last packet containing it; if you have misordering the
> > > frame might be ready a bit later or never.
> > > 
> > > That said _probably_ the current network protocol implementation might
> > > be improved, right now we have a separate select loop in each _read
> > > function and the application has no way to know which is the network
> > > status (e.g. accessing rtcp statistics).
> > > 
> > > Right now ffplay works between fairly well and really well, even in
> > > situations in which mplayer or vlc (with either live555 or libnemesi)
> > > have problems.
> > 
> > please tell me if you
> > A. Do understand that your design requires additional delay compared
> >    to my suggestion but consider this irrelevant / too difficult to fix
> > B. Do not understand that your design requires additional delay compared
> >    to reordering and vbv using the same buffer
> 
> I do understand that our design requires additional delay, in the cases 
> where we actually receive packets out of order - as long as they arrive in 
> order, there's no extra delay at all. I also do think it's possible to do 
> some kind of reordering even without delay, e.g. by doing something like 
> this:
> 
> In udp_read_packet, do:
> - with a nonblocking read, read all outstanding packets, enqueue and 
>   reorder them
> - after the nonblocking read doesn't return any more packets, return the 
>   first packet in the queue
> - if there's no more enqueued packets, do a blocking read
> - if the blocking read returns a packet, return it immediately without 
>   reordering
> 
> In that setup, one would get reordering if the packets are already in the 
> kernel buffer, but if we're forced to wait for the packet, we'd just 
> return it immediately. Then there's no extra delay.
> 
> That said, I'm not in favour of this approach - if we really want 
> reordering, we should also be ok with some extra delay, since the out of 
> order packets very well can arrive with some delay. And if we want a very 
> low delay, set the max_delay parameter to a sufficiently low value to 
> avoid all reordering/queuing.

very low delay is absolutely mandatory for some applications and refusing
to return the correct packet if you have 2 packets is just insane.

i dont object to a max_delay (in milli second or such)
i object to the code returning the wrong packet of several
that it has immedeately available if max_delay=0 or for example
if there are 5 packets in the kernel buffer and max_delay=1 to then
limit reordering to 1 packet (and drop 3 as they kernel buffer overflows
till the next call of av_read_frame())

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Frequently ignored awnser#1 FFmpeg bugs should be sent to our bugtracker. User
questions about the command line tools should be sent to the ffmpeg-user ML.
And questions about how to use libav* should be sent to the libav-user ML.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100525/bb10e31b/attachment.pgp>