[FFmpeg-user] How to calculate difference between wall clock start time of audio input and wall clock start time of video input from inside the asetpts filter?

[David Bayles]

Here is my code:

Lapaki:~ Lapaki$ /Users/Lapaki/Desktop/ffmpeg -f avfoundation -video_size
960x540 -pixel_format uyvy422 -framerate ntsc -thread_queue_size 8B -i
"XI:none" -f avfoundation -thread_queue_size 8B -i "none:XI" -vf
'crop=iw-240:ih:120:0' -af 'asetpts=PTS+.58735/TB' -pix_fmt yuv420p -aspect
4:3 -s 720x480 -q:v 3 -maxrate 5000k -bufsize 2000k -acodec ac3 -ac 2 -ab
256k -ar 48000 -f dvd /Users/Lapaki/Desktop/FF\ Test/`date +%F`\ `date
+%H_%M_%S`.mpg

ffmpeg version 3.2.3-tessus Copyright (c) 2000-2017 the FFmpeg developers

  built with Apple LLVM version 8.0.0 (clang-800.0.42.1)

  configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg
--extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl
--enable-libass --enable-libbluray --enable-libfreetype --enable-libgsm
--enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb
--enable-libopencore-amrwb --enable-libopus --enable-libschroedinger
--enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora
--enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis
--enable-libvpx --enable-libwavpack --enable-libx264 --enable-libx265
--enable-libxavs --enable-libxvid --enable-libzmq --enable-version3
--disable-ffplay --disable-indev=qtkit --disable-indev=x11grab_xcb

  libavutil      55. 34.101 / 55. 34.101

  libavcodec     57. 64.101 / 57. 64.101

  libavformat    57. 56.101 / 57. 56.101

  libavdevice    57.  1.100 / 57.  1.100

  libavfilter     6. 65.100 /  6. 65.100

  libswscale      4.  2.100 /  4.  2.100

  libswresample   2.  3.100 /  2.  3.100

  libpostproc    54.  1.100 / 54.  1.100

Input #0, avfoundation, from 'XI:none':

  Duration: N/A, start: 648413.295900, bitrate: N/A

    Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 960x540,
29.97 fps, 29.97 tbr, 1000k tbn, 1000k tbc

Input #1, avfoundation, from 'none:XI':

  Duration: N/A, start: 648413.884042, bitrate: 3072 kb/s

    Stream #1:0: Audio: pcm_f32le, 48000 Hz, stereo, flt, 3072 kb/s

Output #0, dvd, to '/Users/Lapaki/Desktop/FF Test/2017-02-16 04_16_33.mpg':

  Metadata:

    encoder         : Lavf57.56.101

    Stream #0:0: Video: mpeg2video (Main), yuv420p, 720x480 [SAR 8:9 DAR
4:3], q=2-31, 200 kb/s, 29.97 fps, 90k tbn, 29.97 tbc

    Metadata:

      encoder         : Lavc57.64.101 mpeg2video

    Side data:

      cpb: bitrate max/min/avg: 5000000/0/200000 buffer size: 2000000
vbv_delay: -1

    Stream #0:1: Audio: ac3, 48000 Hz, stereo, fltp, 256 kb/s

    Metadata:

      encoder         : Lavc57.64.101 ac3

Stream mapping:

  Stream #0:0 -> #0:0 (rawvideo (native) -> mpeg2video (native))

  Stream #1:0 -> #0:1 (pcm_f32le (native) -> ac3 (native))

Press [q] to stop, [?] for help

[swscaler @ 0x7f8e0c8ab400] Warning: data is not aligned! This can lead to
a speedloss

frame=   33 fps=0.0 q=3.0 size=     266kB time=00:00:01.06
bitrate=2051.9kbits/sframe=   49 fps= 48 q=3.0 size=     444kB
time=00:00:01.54 bitrate=2358.8kbits/sframe=   64 fps= 42 q=3.0 size=
652kB time=00:00:02.08 bitrate=2560.5kbits/sframe=   79 fps= 39 q=3.0 size=
    838kB time=00:00:02.59 bitrate=2642.4kbits/sframe=   94 fps= 37 q=3.0
size=    1022kB time=00:00:03.07 bitrate=2720.0kbits/sframe=  109 fps= 36
q=3.0 size=    1208kB time=00:00:03.59 bitrate=2756.5kbits/sframe=  124
fps= 35 q=3.0 size=    1406kB time=00:00:04.07 bitrate=2830.0kbits/sframe=
127 fps= 35 q=3.0 Lsize=    1474kB time=00:00:04.19 bitrate=2876.4kbits/s
dup=12 drop=0 speed=1.15x

video:1310kB audio:113kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: 3.604597%


I'm currently doing a quick capture, just to manually calculate the
difference between the start times of the two streams, then using that
value inside the asetpts filter to offset the audio stream by that amount.
This roughly works, but the difference between the input start times
changes slightly every time, so I'd like to be able to calculate this
difference inside the asetpts filter, using I something like -af
'asetpts=PTS+([stream 1 start time]-[stream 2 start time])/TB'.


Is there a way to do this? What is the proper syntax?


Thanks!!