[Ffmpeg-devel] Audio/Video Syncing...

[Michael Niedermayer]

Hi

On Fri, Oct 13, 2006 at 09:15:20PM -0500, Ryan Martell wrote:
> 
> On Oct 13, 2006, at 7:06 PM, Michael Niedermayer wrote:
> 
> >Hi
> >
> >On Fri, Oct 13, 2006 at 05:58:57PM -0500, Ryan Martell wrote:
> >>Hi--
> >>
> >>I have implented the rtsp streaming over h264.  Everything works
> >>against Darwin Streaming Server (packetization mode 1 & 2).  The only
> >>problem is that I can't figure out what I'm doing wrong on getting
> >>the audio/video to sync.
> >>
> >>I have called this on the stream:
> >>    av_set_pts_info(stream, 33, 1, 90000)
> >>
> >>And 90kHz is the standard for h264 streaming.
> >>
> >>When I let ffmpeg figure out the frame rate, sometimes it shows up as
> >>10 fps(r), and other times it shows up at 29.97 fps(r).
> >>
> >>If I hardcode the fps (num=30000, and den= 1001), it will still
> >>sometimes playback at 10.
> >
> >well the question here is what is the real framerate of the stream
> >or in other words how many frames do you pass to libavformat per
> >second? and what is the difference between the pts values of the
> >frames you pass to lavf
> 
> Well, I'm passing them as fast as I get them, which I suspect is  
> close to 30fps.  The source movie is 30fps, and occasionally  
> (probably due to ffplay's a/v syncing stuff) it plays faster than 30  
> fps.
> 
> I'm wondering if the frame rate randomness is some sort of artifact  
> of when the rtcp packet shows up?  Again, the stream info (with the  
> fps) doesn't show up before some delay (which I haven't fully  
> investigated yet.  Maybe my frame rate is affected by the arrival of  
> the rtcp packet?  (I'm going to try to discard all packets that  
> arrive before the rtcp packet shows up, and see what happens).
> 
> 
> Here's an output with DEBUG_SYNC defined, at the point that it  

well, my question was more, what dts/pts do you pass to libavformat
not what ffplay contains internally, ffplay buffers packets and stuff
so its internal timestamps might not be exactly what you passed to it

also does ffmpeg show the same AVsync issue when transcoding to some
random fileformat? what if you specify some -async ?


[...]
> 
> >>When I enable DEBUG_SYNC in ffplay, the pts value I'm giving it
> >>starts off around -9000 (this is because there is no rtcp packet yet
> >>to sync the timebase to, I think), then jumps to 0, and proceeds to
> >>go up from there.  The A-V delay will end up around 11 seconds at
> >>times, other times it will be closer to zero.
> >
> >does the delay stay constant or does it increase?
> >
> >and can you elaborate on the rtcp packet sync thing ...
> 
> The delay tends to decrease, albeit slowly.
> 
> The rtcp packet sync thing is this code:
> 
>     if (s->last_rtcp_ntp_time != AV_NOPTS_VALUE) {
>         int64_t addend;
>         int32_t delta_timestamp;
> /*	
> 	ntp format represents seconds and fraction as a 64-bit unsigned  
> fixed-point integer with decimal point
> 	to the left of bit 32 numbered from the left. The 32-bit seconds  
> field spans about 136 years, while the
> 	32-bit fraction field precision is about 232 picoseconds.
> */
>         //      fprintf(stderr, " (has timestamp) ");
>         /* XXX: is it really necessary to unify the timestamp base ? */
>         /* compute pts from timestamp with received ntp_time */
>         delta_timestamp = timestamp - s->last_rtcp_timestamp;
> //      if(delta_timestamp<0) fprintf(stderr, "Backwards in time: %d 
> \n", delta_timestamp);
>         /* convert to 90 kHz without overflow */
>         addend = (s->last_rtcp_ntp_time - s->first_rtcp_ntp_time) >>  
> 14;
>         addend = (addend * 5625) >> 14;
>         adjusted_timestamp = addend + delta_timestamp;
>     }
> 
> This code comes from rtp.c.  

note, patches with code duplication will not be accepted unless removing
the duplication would be very difficult, but thats not the case here, it
needs a mere addition of a function which does this timestamp adjust


> The aac stuff doesn't adjust pts  
> anywhere in the rtp code (that I can see), so I'm not sure how it's  
> generating it's pts.

maybe thats why it doesnt work ...


> 
> Essentially, because of above, before the first rtcp packet, i have  
> pts values that aren't necessarily scaled correctly.  The Stream Info  
> won't show up (ffplay -stats) until the first rtcp packet shows up,  
> so there is no audio or video playing, but the packets are being  
> cached.  I guess I could pitch them until we get the first packet,  
> and see what that does....

sounds like a good idea, even if it doesnt fix the issue, packets with
random timestamps arent good ...

[...]
> 
> PS: Did you get a chance to look over my code on the other thread to  
> see if it was getting close to acceptable?

its 80k looking through that takes time, be patient

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

In the past you could go to a library and read, borrow or copy any book
Today you'd get arrested for mere telling someone where the library is