[FFmpeg-user] FFmpeg capture, mkvtimestamp_v2, vsync 0 and timecode don't play nice

[Bouke]

Trying to capture and modify the TC in-file afterwards.
I've found a nice way to store the timestamps from the capture.
(https://stackoverflow.com/questions/53391150/ffmpeg-obtain-the-system-time-corresponding-to-each-frame-present-in-a-video)

This works fine using this line:

ffmpeg -hide_banner -f "decklink" -queue_size "1073741824" -raw_format "auto" -format_code "Hi50" -video_input "sdi" -i "bm mini One" -filter_complex "settb=1/1000,setpts=RTCTIME/1000-1500000000000,mpdecimate,split[out][ts];[out]setpts=N/25/TB[out]" -map "[out]" -c:a "copy" -c:v "prores" -profile:v "1" -vendor "ap10" -pix_fmt "yuv422p10le" "/Volumes/Data/tst1.mov" -map "[ts]" -f mkvtimestamp_v2 "/Volumes/Data/time.txt" -vsync 0

But, when I add -timecode "00:00:00:00" (to force a TC atom in the output), horrible things happen.

ffmpeg -f "decklink" -queue_size "1073741824" -raw_format "auto" -format_code "Hi50" -video_input "sdi" -i "bm mini One" -filter_complex "settb=1/1000,setpts=RTCTIME/1000-1500000000000,mpdecimate,split[out][ts];[out]setpts=N/25/TB[out]" -map "[out]" -timecode "00:01:00:00" -c:a "copy" -c:v "prores" -profile:v "1" -vendor "ap10" -pix_fmt "yuv422p10le" "/Volumes/Data/tst1.mov" -map "[ts]" -f mkvtimestamp_v2 "/Volumes/Data/time.txt" -vsync 0

The timecode does not run at the video speed, skips a frame or two here and there, and the image freezes after a random amount of time (between 10 seconds and a minute or so).

How come the timecode can mess up stuff that much? From what I understand it's just a couple of atoms in the moov atom, and a reference where the actual TC value (as frames) is stored in the mdat.

I highly suspect the -vsync 0 to also work on the video. If I omit that, the video is fine, the TC is fine, but there is no metadata output, just the # timecode format v2

The alternative to this dance is to use -timestamp_align, but I also want to use non-decklink audio, and I can’t get that in sync. (Using -ss for sound is not accurate.)
 
Bouke

Output from the terminal: (non-working file)

bouke at Boukes-iMac ~ %  ffmpeg -f "decklink" -queue_size "1073741824" -raw_format "auto" -format_code "Hi50" -video_input "sdi" -i "bm mini One" -filter_complex "settb=1/1000,setpts=RTCTIME/1000-1500000000000,mpdecimate,split[out][ts];[out]setpts=N/25/TB[out]" -map "[out]" -timecode "00:01:00:00" -c:a "copy" -c:v "prores" -profile:v "1" -vendor "ap10" -pix_fmt "yuv422p10le" "/Volumes/Data/tst1.mov" -map "[ts]"  -f mkvtimestamp_v2 "/Volumes/Data/time.txt" -vsync 0
ffmpeg version git-2021-03-31-b3e9a84 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.29)
  configuration: --enable-videotoolbox --enable-nonfree --enable-gpl --enable-lv2 --enable-openssl --enable-libsvtav1 --enable-libx264 --enable-libx265 --enable-libvpx --enable-libxvid --enable-libvidstab --enable-libaom --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libtheora --enable-libfdk-aac --enable-libwebp --enable-libsrt --disable-debug --disable-doc --disable-shared --enable-pthreads --enable-static --enable-small --enable-version3 --enable-libfreetype --enable-libfontconfig --enable-decklink --extra-cflags='-I/Users/bouke/ffmpeg-build-script/workspace/include -I/Users/bouke/ffmpeg-build-script/include' --extra-ldexeflags= --extra-ldflags='-L/Users/bouke/ffmpeg-build-script/workspace/lib -L/Users/bouke/ffmpeg-build-script/include' --extra-libs='-ldl -lpthread -lm -lz' --pkgconfigdir=/Users/bouke/ffmpeg-build-script/workspace/lib/pkgconfig --pkg-config-flags=--static --prefix=/Users/bouke/ffmpeg-build-script/workspace
  libavutil      56. 63.101 / 56. 63.101
  libavcodec     58.117.101 / 58.117.101
  libavformat    58. 65.101 / 58. 65.101
  libavdevice    58. 11.103 / 58. 11.103
  libavfilter     7. 96.100 /  7. 96.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
[decklink @ 0x7fc41a808200] Found Decklink mode 1920 x 1080 with rate 25.00(i)
[decklink @ 0x7fc41a808200] Frame received (#1) - No input signal detected - Frames dropped 1
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, decklink, from 'bm mini One':
  Duration: N/A, start: 0.000000, bitrate: 830976 kb/s
    Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
    Stream #0:1: Video: rawvideo (UYVY / 0x59565955), uyvy422(top first), 1920x1080, 829440 kb/s, 25 tbr, 1000k tbn, 1000k tbc
Stream mapping:
  Stream #0:1 (rawvideo) -> settb
  setpts -> Stream #0:0 (prores)
  split:output1 -> Stream #1:0 (rawvideo)
Press [q] to stop, [?] for help
Output #0, mov, to '/Volumes/Data/tst1.mov':
  Metadata:
    timecode        : 00:01:00:00
    encoder         : Lavf58.65.101
    Stream #0:0: Video: prores (apcs / 0x73637061), yuv422p10le(tv, top coded first (swapped)), 1920x1080, q=2-31, 200 kb/s, 0.04 fps, 12800 tbn
    Metadata:
      encoder         : Lavc58.117.101 prores
Output #1, mkvtimestamp_v2, to '/Volumes/Data/time.txt':
  Metadata:
    encoder         : Lavf58.65.101
    Stream #1:0: Video: rawvideo (Y42B / 0x42323459), yuv422p(tv, top coded first (swapped)), 1920x1080, q=2-31, 829440 kb/s, 25 fps, 1k tbn
    Metadata:
      encoder         : Lavc58.117.101 rawvideo
frame=    1 fps=0.0 q=0.0 q=-0.0 size=       0kB time=33848:30:24.72 bitrate=   [decklink @ 0x7fc41a808200] Frame received (#8) - Input returned - Frames dropped 2
frame=    9 fps=0.0 q=-0.0 q=-0.0 size=    2304kB time=33848:30:25.24 bitrate=  frame=   22 fps= 21 q=-0.0 q=-0.0 size=    7424kB time=33848:30:25.76 bitrate=  frame=   35 fps= 22 q=-0.0 q=-0.0 size=   12800kB time=33848:30:26.28 bitrate=  frame=   48 fps= 23 q=-0.0 q=-0.0 size=   17920kB time=33848:30:26.80 bitrate=  frame=   61 fps= 23 q=-0.0 q=-0.0 size=   23296kB time=33848:30:27.32 bitrate=  frame=   71 fps= 23 q=-0.0 Lq=-0.0 size=   28128kB time=33848:30:27.72 bitrate=   0.0kbits/s speed=3.99e+07x    
video:315677kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown