[FFmpeg-user] Issue with A/V sync due to dropped frames

[Mark Huang]

Hi Carl,

Thank you for responding to my question, for some reason, gmail blocked
your reply. I just saw your reply in the archive today.

For any one that had the same issue, the fix is actually very easy. Don't
use the -r 30 to specify the framerate as that will result in a constant
framerate filter being inserted. The avfoundation device has the -framerate
option which is the one you want to use instead. I guess the behaviour that
-r will also set the -framerate is what confused me. As for the rest of the
pipeline, ffmpeg seems to be fine with variable framerate and the vsync
setting and async 1 as well.

The async 1 could be extra, but I found the avfoundation device sometimes
have different starting pts for video vs audio.
The vsync vfr to my understanding just passes the pts timestamp without
trying to modify it. I have no idea if it does anything negative so I just
left it there.

-Mark

On Thu, Apr 14, 2016 at 1:38 PM, Mark Huang <mmhmmh at gmail.com> wrote:

> Hi all,
>
> I am running into some issues when trying to do audio/video recording from
> live source. I am running it on OS X 10.11 using avfoundation.
>
> The issue is that the audio and video will get out of sync as the
> recording duration increases. I suspect it is due to frames being dropped
> at the device.
>
> To debug, I added the -debug_ts flag and recorded a 20 minute video. The
> result video has audio/video out of sync by like 1s at the end. The
> timestamp logging indicated that the packet from the output of the filter
> has incorrect timestamp (off by 1s), see bolded numbers.
>
> My question is that is there away to tell ffmpeg filter that a frame is
> dropped and it shouldn't modify the PTS to keep the constant fps, or I
> guess disable the sanitizing pts all together.
>
> I tried using the -vf fps=fps=30 and it didn't help.
>
> Command line:
>
> *ffmpeg -rtbufsize 100M -probesize 32 -debug_ts -y -threads 4 -vsync vfr
> -async 1 -f avfoundation -s 640x480 -r 30.000000 -video_device_index 0
> -audio_device_index 0 -i  -strict -2 -vcodec libx264 -x264opts
> keyint=45:min-keyint=30 -preset fast -crf 28 -tune zerolatency -pix_fmt
> yuv420p -f mp4 -movflags +faststart video.mp4*
>
> Partial console output showing issue:
>
> demuxer+ffmpeg -> ist_index:0 type:video pkt_pts:2224907433
> pkt_pts_time:2224.91 pkt_dts:2224907433 pkt_dts_time:2224.91
> off:-185132159300 off_time:-185132
> decoder -> ist_index:0 type:video frame_pts:2224907433 frame_pts_time:
> *2224.91* best_effort_ts:2224907433 best_effort_ts_time:2224.91
> keyframe:1 frame_type:1 time_base:1/1000000
> filter -> pts:66706 pts_time:*2223.53* exact:66706.000008 time_base:1/30
> encoder <- type:video frame_pts:66706 frame_pts_time:2223.53 time_base:1/30
> encoder -> type:video pkt_pts:66706 pkt_pts_time:2223.53 pkt_dts:66706
> pkt_dts_time:2223.53
> encoder -> type:video pkt_pts:34153472 pkt_pts_time:2223.53
> pkt_dts:34153472 pkt_dts_time:2223.53
> muxer <- type:video pkt_pts:34153472 pkt_pts_time:2223.53 pkt_dts:34153472
> pkt_dts_time:2223.53 size:743
> filter -> pts:22235 pts_time:2223.5 exact:22235.333336 time_base:1/10
> encoder <- type:video frame_pts:22236 frame_pts_time:2223.6 time_base:1/10
> encoder -> type:video pkt_pts:22236 pkt_pts_time:2223.6 pkt_dts:22236
> pkt_dts_time:2223.6
>     Last message repeated 1 times
> muxer <- type:video pkt_pts:22236 pkt_pts_time:2223.6 pkt_dts:22236
> pkt_dts_time:2223.6 size:230400
> demuxer -> ist_index:1 type:audio next_dts:2224943080
> next_dts_time:2224.94 next_pts:2224943080 next_pts_time:2224.94
> pkt_pts:187357102381 pkt_pts_time:187357 pkt_dts:187357102381
> pkt_dts_time:187357 off:-185132159300 off_time:-185132
>
>
> Thank
> -Mark
>