[Libav-user] Video and audio timing / syncing

[Kalileo]

On Apr 1, 2013, at 06:31 , Brad O'Hearne wrote:

> On Mar 31, 2013, at 1:25 AM, Alex Cohn <alexcohn at netvision.net.il> wrote:
>> I am not sure when "duration" is taken into account, but you could
>> simply set current->pts = prev->pts+2. Note that this was my original
>> proposal.
> 
> Alex -- I considered that approach, and essentially ran that same test -- manually manipulating the pts values that is. But the problem with this is twofold: 
> 
> - A "+2" pts increase likely will only work when you have an actual frame rate that is half that of the expected frame rate which is used to initially set the time_base. That's not my use case (the closest approximation is 24 fps expected, and on this particular computer / camera I'm testing with, 15fps. And I think in theory, the only video source that can feed an encoder with fixed-fps video is one that is fully known and controllable prior to encoding (like an existing video file, or just generating data like the examples do). More about this in a moment. 


There's a lot of half-theory in your questions, and i get confused about your intentions. Do you want to solve a problem (of video/audio not being in sync) or do you want to redesign the dts/pts concept of video/audio syncing?

> 
> - If I'm going to muck with the video pts in this fashion, the audio pts also has to be mucked with to keep it in sync, and of course, audio samples are received at a different rate than the video. So that's a complexity there. 

Didn't you say that it's _not_ in sync now? So obviously you've to correct one side, not do the same modification on both sides.


I do not understand why you need to make this so complicated. It is so easy, same PTS = to be played at the same time. You cannot expect your player to play Audio with PTS 456789 at the same time as video with PTS 123456, but if audio and video both have a PTS around 456789 then most players will make an attempt to play them at the same time.

Your stream, the one which is not in sync, have you checked the DTS/PTS values of audio and video at the beginning and later? If you do so, you see where they drift apart, and you get a hint what and where to correct. FFPROBE is your friend for such probing.

As the gazillions of perfectly in sync videos out there show, the system is working, no matter if it's the best or not. But you must set the values to allow the system to work, that means if it is to be played at the same time, then PTS must be (about) the same.

The beauty of this is that it works even if half of the other data is messed up, fps and whatever can be wrong, the player can use the audio as master and correct the video timing using PTS alone, and it plays perfectly in sync.

Over and out.