[FFmpeg-devel] [PATCH] Sync the NTP timestamps for RTP streams

[Martin Storsjö]

On Thu, 11 Mar 2010, Michael Niedermayer wrote:

> On Thu, Mar 11, 2010 at 11:51:08AM +0200, Martin Storsj? wrote:
> > 
> > If adding another field to the context is ok with you, the attached 
> > series would be another way of solving it.
> > 
> > Instead of storing NTP time (which isn't all that common outside of RTP), 
> > the field stores unix time (in microseconds). Also, I chose to store this 
> > in AVStream instead of in AVFormatContext, since it could be used to store 
> > extra info for the RTP streams when using the RTSP demuxer.
> > 
> > Is this acceptable, and does it seem generally usable for other things 
> > than this one in particular, so that it could be accepted in the general 
> > context?
> > 
> > // Martin
> >  avformat.h |    8 ++++++++
> >  1 file changed, 8 insertions(+)
> > d43f659c8bfdbc7b0fbfad8256d7d3856e2e77d7  0001-Add-a-field-to-AVStream-for-specifying-a-real-world-.patch
> > From 61fd799efc279662d0e7718db7366fc642d5c472 Mon Sep 17 00:00:00 2001
> > From: Martin Storsjo <martin at martin.st>
> > Date: Thu, 11 Mar 2010 11:11:43 +0200
> > Subject: [PATCH 1/4] Add a field to AVStream for specifying a real world start time for the stream
> > 
> > ---
> >  libavformat/avformat.h |    8 ++++++++
> >  1 files changed, 8 insertions(+), 0 deletions(-)
> > 
> > diff --git a/libavformat/avformat.h b/libavformat/avformat.h
> > index 10371a4..7c74951 100644
> > --- a/libavformat/avformat.h
> > +++ b/libavformat/avformat.h
> > @@ -526,6 +526,14 @@ typedef struct AVStream {
> >       * Number of frames that have been demuxed during av_find_stream_info()
> >       */
> >      int codec_info_nb_frames;
> > +
> > +    /**
> > +     * Start time of the stream in real world time, in microseconds
> > +     * since the unix epoch.
> > +     * - encoding: Set by user.
> > +     * - decoding: Unused.
> > +     */
> > +    int64_t start_time_realtime;
> 
> this is not completely clear, we have
> time of transmission of first packet
> time of decoding of first packet
> time of presentation of first packet
> time corresponding to timestamp=0 instead of first packet
> 
> and i think you should write out what the unix epoch is

Ok, I'll clarify it. In this case, it is the real-world time corresponding 
to timestamp = 0 in the stream, that is, when the data was captured.

> that said, i didnt follow the thread very much, could you explain in a few
> lines why this is needed?

Syncing of multiple RTP streams is done through sender reports, where the 
sender says that timestamp X in the RTP stream corresponds to real-world 
time Y.

When sending multiple RTP streams, currently the timestamp=0 <-> realworld 
time mapping is made when the RTP muxer is opened, which makes the 
timestamp=0 map to different realworld times in the muxers, since the RTP 
muxers can't be opened at the exact same time. (In practice, one can get a 
5-30 ms difference between them, when doing RTSP commands inbetween.)

So when writing multiple RTP streams, we need to make sure timestamp=0 
maps to the exact same realworld time, so that the receiver can sync the 
streams back together.

In the RTP scenarios, the actual value of the NTP timestamps doesn't 
matter all that much, only the difference between them is used for syncing 
the streams together.

// Martin