[FFmpeg-devel] [RFC] rtpdec: Reordering RTP packets

[Michael Niedermayer]

On Sun, May 23, 2010 at 01:43:04PM +0300, Martin Storsj? wrote:
> On Wed, 19 May 2010, Michael Niedermayer wrote:
> 
> > The obvious way to handle streaming (overall not just libav) is that
> > 1. packets are received and put in a buffer
> > 2. each packet is decoded and displayed once its time has come
> > 3. the initial delay is set so that display likely wont get stuck and
> >    that the delay doesnt annoy the user too much
> > 4. if the next packet is unavailable by the time it is needed then
> >    either one has to decode without it or one has to wait which implicates
> >    that the delay between receive and display from now on is larger
> > 5. Clocks used to decode/display have to be synchronized between server
> >    and client (using SCR/PCR in mpeg and NTP in rtp)
> > 
> > I might be missing some things and there may be better and different
> > approuches and i dont mind at all if its done differently.
> > But i do mind if its done in a way that is worse in terms of delay vs.
> > reordering resistance because i dont think the world needs another rtp
> > implementation that doesnt work properly.
> > 
> > now if we assume the packet que is inside lavf, with any que between
> > lavf and lavc in the user app being as small as possible then the
> > application will call av_read_frame() when it _needs_ the next packet
> > and the rtp code simply has to return the next packet in this case.
> > i see no complexity here.
> > This might need minor adjustments in ffplay related to que sizes and
> > some 2 line initial delay buildup loop between av_open* and av_read_frame()
> > and we may need a way for the user app to indicate which stream it needs
> > a packet from first for av_read_frame().
> > But it really does not look all that complex.
> > As said dont mind at all to do this differently if the end result is
> > equally good from the users point of view in performance and features.
> > I also dont mind if just a subset of this is implemented to keep work
> > and complexity small.
> > But i do mind if work is going of toward some dead end that doesnt get
> > us closer to a proper implementation
> 
> While I agree with you in general, there are a few complications in doing 
> it this way in the particular case with RTP/UDP.
> 
> Since UDP packets are dropped silently if not received in due time, the 
> receiving thread should spend most its time blocking in select()/recv() or 
> something similar, so that packets are received by the application and not 
> dropped by some kernel queue. But that's no problem with the current 
> design, by adjusting the queue sizes, I think.

keep in mind some applications using lav* are single thread applications


> 
> But for reordering, one frame can be fragmented over numerous packets, 
> that all have the same timestamp. When these packets are received out of 
> order, they are fed into a parser within lavf, that merges them together 
> to reconstruct the full original frame. The only clue on how to reorder 
> them properly is the RTP sequence number, which is only available within 
> rtpdec and not exposed outside of that - unless you want to expose such 
> RTP-specific details in the general code of course.
> 

> So given this, I still think the reordering should be done at the rtpdec 
> layer.

thats what i suggested
what is unclear to me is how you intend to decide what and when to return
things from the que.  I suggested that the user application decide. how do
you want to handle this, an application might need audio a bit before video

And how do you intent to keep the clocks synchronized? currently an application
coulde decode too fast and then get stuck when its out of data and all buffers
are empty

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have often repented speaking, but never of holding my tongue.
-- Xenocrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100523/64f71663/attachment.pgp>