[FFmpeg-user] Recording and synchronizing from two video input streams and an audio input stream

[Kim Thomassen]

I have two IP cameras and a USB microphone, and basically, I need to
record/sync up these three devices to an accuracy of about a millisecond,
though perhaps I can get away with them being a couple of milliseconds out
of sync, and write them to a single output file. I need them to be so close
to each other because they will later be connected into a single panorama
output.

The cameras themselves are some cheap no-brand ones, recording in 4k, and
they can't keep track of time, so I have to do all the managing on my
machine. The timestamps I get from them are all wrong.

I early on experimented with just getting my streams through one ffmpeg
command;
    >ffmpeg -i rtsp://192.168.0.202/av0_0 -map 0 -y -t 13 "file-202.ts" -i
rtsp://192.168.0.203/av0_0 -map 1 -y -t 13 "file-203.ts"

However I wasn't able to get them to synchronize anywhere near close
enough, so I abandoned this setup pretty quickly.

What I'm attempting at the moment, is to use the "at" utility, to schedule
the two video streams to start recording at the same time, and I then
filter them through vstack to test the synchonizity, and while I do get
pretty close, it's sometimes in the wind wether the recording is actually
in sync.

The command I use to record is, located in a record.sh script;
    >ffmpeg -thread_queue_size 512 -rtsp_transport tcp -i
rtsp://192.168.0.$THISIP/av0_0
-y -t $TIME "$THISIP.ts"

And pertinent parts of the connector script is;
    >for ((n=0;n<$NOCAMERAS;n++))
    >  do
    >    at now +1 minutes <<< "$PWD/record.sh $THISIP $TIME" &
    >  done

And the part of the connector script which stacks the videos atop each
other;
    >ffmpeg -i "202.ts" -i "203.ts" -y -filter_complex
"[1][0]scale2ref[2nd][ref];[ref][2nd]vstack" -c:v libx264 -crf 23 -preset
veryfast "mixer-$NOW.mp4"

I'm not excactly completely satisfied with the result but as it's somewhat
close, I've started on also doing the audio recording;
    >ffmpeg -f alsa -i hw:2 -rtsp_transport tcp -i rtsp://
192.168.0.202/av0_0 -y -c:a aac -vcodec copy -t 12 "VidAud.mp4"

But this gives horrible results, the audio/video streams are sometimes out
of sync by up over a second.

I wanted to incoorporate the audio recording into the connector script, but
I'm not sure that I have the right direction on solving this problem
anymore, and I was hoping maybe someone had some input.

Please let me know if you need me to expand on anything.