[FFmpeg-user] help with live capture and audio sync issues

Fri Sep 4 05:12:04 CEST 2015

Hi,

I have very inconsistent results with doing live capture. Sometimes the resulting file has perfect audio sync, but sometimes it doesn’t. I’ve tried different combinations but the results are never consistent.

I am capturing video from a black magic design UltraStudio Mini Recorder and the audio from an external Mic.

This is done on a MacBook Pro.

Here’s the capture command:

ffmpeg -y \
	-thread_queue_size 2048 -f avfoundation -async 1 -i none:'C-Media USB Audio Device' \
	-thread_queue_size 2048 -f decklink -vsync 0 -i 'UltraStudio Mini Recorder at 12':none\
	-c:v libx264 -preset veryfast -profile:v baseline -vf scale=640:360 \
	-pix_fmt yuv420p \
	-c:a libfdk_aac -b:a 80k \
	-map 1:1 -map 0:0 \
	video_tests/test-avsync.mp4

Here’s the ffmpeg output:

ffmpeg version N-73104-g7604358 Copyright (c) 2000-2015 the FFmpeg developers
  built with Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
  configuration: --prefix=/usr/local --enable-gpl --enable-nonfree --enable-ffplay --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-decklink --extra-cflags=-I/Users/ptl/blackmagicsdk/Mac/include --extra-ldflags=-L/Users/ptl/blackmagicsdk/Mac/include
  libavutil      54. 27.100 / 54. 27.100
  libavcodec     56. 44.100 / 56. 44.100
  libavformat    56. 38.100 / 56. 38.100
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 17.100 /  5. 17.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.100 /  1.  2.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, avfoundation, from 'none:C-Media USB Audio Device':
  Duration: N/A, start: 1057.070635, bitrate: 1411 kb/s
    Stream #0:0: Audio: pcm_f32le, 44100 Hz, mono, flt, 1411 kb/s
[decklink @ 0x7fdc23843600] Found Decklink mode 1920 x 1080 with rate 29.97(i)
Guessed Channel Layout for  Input Stream #1.0 : stereo
Input #1, decklink, from 'UltraStudio Mini Recorder at 12:none':
  Duration: N/A, start: 0.000000, bitrate: 1536 kb/s
    Stream #1:0: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
    Stream #1:1: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1920x1080, -5 kb/s, 29.97 tbr, 1000k tbn, 29.97 tbc
-async is forwarded to lavfi similarly to -af aresample=async=1:min_hard_comp=0.100000:first_pts=0.
[libx264 @ 0x7fdc23845c00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 @ 0x7fdc23845c00] profile Constrained Baseline, level 3.0
[libx264 @ 0x7fdc23845c00] 264 - core 144 r2533 c8a773e - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'video_tests/test-avsync.mp4':
  Metadata:
    encoder         : Lavf56.38.100
    Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 640x360, q=-1--1, 29.97 fps, 30k tbn, 29.97 tbc
    Metadata:
      encoder         : Lavc56.44.100 libx264
    Stream #0:1: Audio: aac (libfdk_aac) ([64][0][0][0] / 0x0040), 44100 Hz, mono, s16, 80 kb/s
    Metadata:
      encoder         : Lavc56.44.100 libfdk_aac
Stream mapping:
  Stream #1:1 -> #0:0 (rawvideo (native) -> h264 (libx264))
  Stream #0:0 -> #0:1 (pcm_f32le (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
frame=   18 fps=0.0 q=29.0 size=       0kB time=00:00:00.03 bitrate=  11.5kbits/frame=   33 fps= 33 q=29.0 size=      79kB time=00:00:00.53 bitrate=1216.8kbits/frame=   48 fps= 32 q=29.0 size=     148kB time=00:00:01.03 bitrate=1173.1kbits/frame=   63 fps= 31 q=29.0 size=     245kB time=00:00:01.53 bitrate=1305.0kbits/frame=   79 fps= 31 q=29.0 size=     369kB time=00:00:02.06 bitrate=1460.7kbits/frame=   94 fps= 31 q=29.0 size=     491kB time=00:00:02.56 bitrate=1565.9kbits/frame=  109 fps= 31 q=29.0 size=     593kB time=00:00:03.06 bitrate=1583.1kbits/frame=  124 fps= 31 q=29.0 size=     691kB time=00:00:03.57 bitrate=1585.9kbits/frame=  139 fps= 31 q=29.0 size=     785kB 
...
time=00:00:26.79 bitrate=1108.6kbits/frame=  822 fps= 30 q=-1.0 Lsize=    3732kB time=00:00:27.42 bitrate=1114.6kbits/s    
video:3445kB audio:263kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.636530%
[libx264 @ 0x7fdc23845c00] frame I:9     Avg QP:25.83  size: 18818
[libx264 @ 0x7fdc23845c00] frame P:813   Avg QP:27.67  size:  4130
[libx264 @ 0x7fdc23845c00] mb I  I16..4: 32.9%  0.0% 67.1%
[libx264 @ 0x7fdc23845c00] mb P  I16..4:  5.6%  0.0%  1.3%  P16..4: 36.7% 12.2%  4.7%  0.0%  0.0%    skip:39.5%
[libx264 @ 0x7fdc23845c00] coded y,uvDC,uvAC intra: 39.7% 51.8% 17.8% inter: 21.0% 15.6% 0.4%
[libx264 @ 0x7fdc23845c00] i16 v,h,dc,p: 29% 44% 18%  9%
[libx264 @ 0x7fdc23845c00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 30% 18%  6%  6%  6%  8%  5%  6%
[libx264 @ 0x7fdc23845c00] i8c dc,h,v,p: 47% 32% 15%  6%
[libx264 @ 0x7fdc23845c00] kb/s:1028.74

Here’s the ffprobe output for the resulting video_tests/test-avsync.mp4 file:

ffprobe -i video_tests/test-avsync.mp4 
ffprobe version N-73104-g7604358 Copyright (c) 2007-2015 the FFmpeg developers
  built with Apple LLVM version 6.1.0 (clang-602.0.53) (based on LLVM 3.6.0svn)
  configuration: --prefix=/usr/local --enable-gpl --enable-nonfree --enable-ffplay --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-decklink --extra-cflags=-I/Users/ptl/blackmagicsdk/Mac/include --extra-ldflags=-L/Users/ptl/blackmagicsdk/Mac/include
  libavutil      54. 27.100 / 54. 27.100
  libavcodec     56. 44.100 / 56. 44.100
  libavformat    56. 38.100 / 56. 38.100
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 17.100 /  5. 17.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.100 /  1.  2.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video_tests/test-avsync.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf56.38.100
  Duration: 00:00:16.08, start: 0.046440, bitrate: 1406 kb/s
    Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 640x360, 1321 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 80 kb/s (default)
    Metadata:
      handler_name    : SoundHandler

I can see that there’s an offset of 0.046440, but I’m not sure if that’s a consistent result. I’m trying to get the capture to produce a result without sync delay but in any case, why would the results be inconsistent?

The MacBook is always pretty idle with no memory issues when doing the capture, and ffmpeg isn’t consuming too much cpu.

Any suggestions on what else I can try to get consistent sync?

thanks
Ricardo