[Libav-user] How to perform drift compensation in MPEG-2 recorder

[Mike Versteeg]

>> On Mon, Dec 17, 2012 at 10:59 AM, Dolevo Jay <cmst at live.com> wrote:
>>> I capture the video frame by frame from the desktop screen and send them
>>> over network after encoding. This happens on the PC processor and I have no
>>> problem with this. It works perfect for now. However, I capture the audio
>>> separately with the help of an HID device which has its own processor. I can
>>> send the audio even without encoding but still the problem is the video and
>>> audio get out of sync and drift occurs although I sync the processors at the
>>> very beginning. Could anyone shed any light on this issue?
>>
>> You can set the video PTS in sync with the current audio "clock". Most
>> likely, your audio packets arrive at fixed nominal frequency which is
>> not precise (from the point of view of your PC clock). In this case
>> you should calculate PTS by multiplying the expected audio packet
>> duration by number of actual audio packets received.
>
>
> Even if a nominal fixed frequency were precise enough with respect to the video frequency it seems you're probably looking at transport delays that are neither constant nor identical (enough) for the two streams. It's not clear if you send video and audio over the same network stream if you maybe even capture the audio on a separate host (possibly the remote)? How big is the drift, I suppose we're not talking about a 1 or 2 frames jitter?

I have a minor jitter, there is no network involved.

I aforementioned example all PTS's are calculated automatically. Where
do I insert my own video PTS? I assume it is in the write_video_frame
function, but is it before the call to avcodec_encode_video() or
after? Do I set the codec PTS or the packet, or both?

Here's (I think) the relevant piece of code:

// encode the image
int out_size = avcodec_encode_video(c, pVideoOut, iVideoOutSize, stPicture);
// if zero size, it means the image was buffered
if (out_size > 0) {
  AVPacket pkt;
  av_init_packet(&pkt);
  if (c->coded_frame->pts != AV_NOPTS_VALUE)
    pkt.pts = av_rescale_q(c->coded_frame->pts, c->time_base, st->time_base);
}

Thanks,

Mike