[Libav-user] Setting PTS/time_base to sync audio/video

[Michael Vernick]

I need to figure out how to get proper lip sync when creating an MP4 
file with separate audio and video inputs.

I have written a program that has 2 inputs:

1. raw audio samples (from mic) with clock time stamp
2. H.264 video frames (from camera) with clock time stamp

The two inputs are in two stored files (it is not live).  To create the 
inputs, one process captures samples from the mic, timestamps them and 
saves them to a file.  Another process captures H264 frames from a 
camera, timestamps them and saves them to a file.  The timestamps are 
based on the same global clock.

My program compresses the audio samples into AAC, combines them with the 
H264 video frames and outputs an MP4 file.  This file can be played back 
by VLC, but the lip sync is off. 

How do I set the audio PTS/time_base and the video PTS/time_base so one 
stream can sync to the other stream to get the correct lip sync?  The 
timestamps in the two input files reference a global clock so I know 
which video frames should be synced to which audio samples and can write 
them at the appropriate time in the output file.  I just can't figure 
out how to set the PTS of the audio/video so one references the other. 

For example, when the resulting mp4 file is played, I want the player to 
play back the audio samples at a rate of 44100hz.  Then I want each 
video frame to be played at the same time as their associated audio 
samples, not based on any kind of video playback rate.

Any help would be appreciated.

Mike