[Libav-user] Video and audio timing / syncing

[Brad O'Hearne]

On Mar 31, 2013, at 12:03 AM, Alex Cohn <alexcohn at netvision.net.il> wrote:
> The trick is that not all containers and codecs support variable frame
> rate. 

Thanks for the reply, Alex. I'm somewhat torn with labeling this a case of a "variable frame rate" -- while being able to handle a variable frame rate would certainly solve the problem, the truth is that the video frames I'm receiving don't appear to actually be variable. The frame rate actually appears to be consistent the entire video (albeit not the frame rate I was initially expecting). To be completely specific about the nature of the problem, it appears the problem is that the frame rate isn't actually known until frames are received, but at the same time the time_base.den which is based on the frame rate needs to be configured prior to receiving captured frames. 

This has me wondering about the nature of the AVPacket duration property. This is another value whose use is rather vague and inconsistent across example code and documentation. On one hand, the documentation says: 

"Duration of this packet in AVStream->time_base units, 0 if unknown. Equals next_pts - this_pts in presentation order."

Virtually all examples I've read don't set the duration at all, in fact it isn't even referenced. So I'm not exactly sure how duration is used if it can be set and left at zero without a problem, and to what degree there is a relationship between it and pts. Given that QT capture delivers duration time with each sample buffer, then perhaps this can be used to manage the frame rate difference. For example, suppose that the codec context's time_base.den value is set to 30 (for a 30fps). However let's say capture produces and delivers only 15fps. That's essentially the same scenario I'm dealing with (though I'm looking for 24fps, not 30, but 30 makes a good round number for this example). Anyway, suppose that I receive only 15fps from capture, or to restate, half the number of frames as expected by time_base.den, what happens if the duration is set to 2 * 1/frame rate -- meaning that each frame had two frames worth of duration at the time_base frame rate. 

Will setting a longer packet duration compensate for receiving fewer frames? 

Thanks,

Brad