[Libav-user] Video and audio timing / syncing

[Brad O'Hearne]

On Mar 29, 2013, at 1:43 PM, Alex Cohn <alexcohn at netvision.net.il> wrote:

> Note that the time base may be set separately for the container and for the video stream (depends on the format and on the codec).

Greetings Alex! Geesh...so many folks taking the time to answer, posting to the Libav-user mailing list appears to be a great way to meet some well-meaning folks!

Anyway, can you expound a little further on your comment? I'm interested to know when you would do such a thing, and what the net effect is for doing so. This dovetails with another question I've had rattling around in my head, and that is the curious place where pts and dts are being set in different code examples I've come across. 

In some examples I've found, pts and dts are being set prior to encoding. In others, pts and dts are being set after encoding, but prior to writing the packet to an output source (file or stream). I think I've even come across an example or two which do both. Now I understand that the packet returned (if one is returned at all) from the av_encode_video2 function call might not contain the same frame data that was passed in the frame data buffer, so it leads me to believe that there might be significance both before and after encoding to a packets pts and dts values. 

So...when are pts / dts supposed to be set?

a) before encoding

b) after encoding

c) both before and after encoding -- and if so, why is this? 

Another way to ask this same question -- is the AVPacket.pts value relevant: 

a) only during encoding

b) only during writing to output file or stream

c) both during encoding and during writing to output file or stream -- and again, I'm curious if this is the case, why the AVPacket returned by the av_encode_video2 function hasn't itself set the pts value properly -- if I'm receiving frames arbitrarily from a capture source, and the av_encode_video2 function is returning out a frame that isn't necessarily the same one I sent it, how would I necessarily know what the pts value should be, other than a calculation based on the frame rate (which puts me back in the boat of having to reconcile the frame rate expected vs. what is actually received)? 

Thx....:-)

Brad