[FFmpeg-devel] [RFC] Synchronize the start times of multiple RTP streams

[Martin Storsjö]

On Sun, 28 Mar 2010, Ronald S. Bultje wrote:

> On Fri, Mar 26, 2010 at 5:52 PM, Martin Storsj? <martin at martin.st> wrote:
> > This patch implements my suggestion on how to solve the synchronization of
> > multiple RTP streams, the problem that Ronald had that originally lead to
> > SVN rev 21857, but without making packets start at a random timestamp.
> [..]
> > -    if (ret < 0)
> > +    if (ret < 0) {
> > +        /* Either bad packet, or a RTCP packet. Check whether the
> > +         * first_rtcp_ntp_time field was initialized. */
> > +        RTPDemuxContext *rtpctx = rtsp_st->transport_priv;
> > +        if (rtpctx->first_rtcp_ntp_time != AV_NOPTS_VALUE) {
> > +            /* first_rtcp_ntp_time has been initialized for this stream,
> > +             * copy the same value to all other uninitialized streams,
> > +             * in order to map their timestamp origin to the same ntp time
> > +             * as this one. */
> > +            int i;
> > +            for (i = 0; i < rt->nb_rtsp_streams; i++) {
> > +                RTPDemuxContext *rtpctx2 = rtsp_st->transport_priv;
> > +                if (rtpctx2 &&
> > +                    rtpctx2->first_rtcp_ntp_time == AV_NOPTS_VALUE)
> > +                    rtpctx2->first_rtcp_ntp_time = rtpctx->first_rtcp_ntp_time;
> > +            }
> > +        }
> >          goto redo;

A detail on the actual patch - this clause should be moved up a few lines, 
so that it is only run for RTP, not for RDT... Changed locally.

> One problem here is that it doesn't necessarily take "lowest start
> time", but "first start time". Imagine that we have h264+aac, h264
> coming in first at a time of 100 (99 start + 1 time), and then aac at
> 50 (49 + 1). This code will set first_rtcp to 99 for each, and thus
> set the timestamp of the first aac data to -50.

True, that's a problem.

> What's more worrying is that I think this shouldn't be needed. There's
> several functions in utils.c that do this automatically (no action
> needed), but they don't work because of overflows because of the crazy
> randomness in starttime here. We should fix (imo) that, rather than
> trying to workaround it in the RTSP demuxer...

Ok, if the rest of lavf is supposed to take care of this, then this indeed 
is redundant.

// Martin