[FFmpeg-user] Issue with A/V sync due to dropped frames

[Mark Huang]

Hi all,

I am running into some issues when trying to do audio/video recording from
live source. I am running it on OS X 10.11 using avfoundation.

The issue is that the audio and video will get out of sync as the recording
duration increases. I suspect it is due to frames being dropped at the
device.

To debug, I added the -debug_ts flag and recorded a 20 minute video. The
result video has audio/video out of sync by like 1s at the end. The
timestamp logging indicated that the packet from the output of the filter
has incorrect timestamp (off by 1s), see bolded numbers.

My question is that is there away to tell ffmpeg filter that a frame is
dropped and it shouldn't modify the PTS to keep the constant fps, or I
guess disable the sanitizing pts all together.

I tried using the -vf fps=fps=30 and it didn't help.

Command line:

*ffmpeg -rtbufsize 100M -probesize 32 -debug_ts -y -threads 4 -vsync vfr
-async 1 -f avfoundation -s 640x480 -r 30.000000 -video_device_index 0
-audio_device_index 0 -i  -strict -2 -vcodec libx264 -x264opts
keyint=45:min-keyint=30 -preset fast -crf 28 -tune zerolatency -pix_fmt
yuv420p -f mp4 -movflags +faststart video.mp4*

Partial console output showing issue:

demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:2224907433
pkt_pts_time:2224.91 pkt_dts:2224907433 pkt_dts_time:2224.91
off:-185132159300 off_time:-185132
decoder -> ist_index:0 type:video frame_pts:2224907433 frame_pts_time:
*2224.91* best_effort_ts:2224907433 best_effort_ts_time:2224.91 keyframe:1
frame_type:1 time_base:1/1000000
filter -> pts:66706 pts_time:*2223.53* exact:66706.000008 time_base:1/30
encoder <- type:video frame_pts:66706 frame_pts_time:2223.53 time_base:1/30
encoder -> type:video pkt_pts:66706 pkt_pts_time:2223.53 pkt_dts:66706
pkt_dts_time:2223.53
encoder -> type:video pkt_pts:34153472 pkt_pts_time:2223.53
pkt_dts:34153472 pkt_dts_time:2223.53
muxer <- type:video pkt_pts:34153472 pkt_pts_time:2223.53 pkt_dts:34153472
pkt_dts_time:2223.53 size:743
filter -> pts:22235 pts_time:2223.5 exact:22235.333336 time_base:1/10
encoder <- type:video frame_pts:22236 frame_pts_time:2223.6 time_base:1/10
encoder -> type:video pkt_pts:22236 pkt_pts_time:2223.6 pkt_dts:22236
pkt_dts_time:2223.6
    Last message repeated 1 times
muxer <- type:video pkt_pts:22236 pkt_pts_time:2223.6 pkt_dts:22236
pkt_dts_time:2223.6 size:230400
demuxer -> ist_index:1 type:audio next_dts:2224943080 next_dts_time:2224.94
next_pts:2224943080 next_pts_time:2224.94 pkt_pts:187357102381
pkt_pts_time:187357 pkt_dts:187357102381 pkt_dts_time:187357
off:-185132159300 off_time:-185132


Thank
-Mark