[FFmpeg-user] Can't synchronize two webcam streams side by side using ffmpeg

[bob Darkblue]

Hello,

I'm trying to put side by side two webcam streams to create a stereo vision
setup.
I'm on ubuntu 16.04 using v4l2.


It works but there is a delay of ~1 second between the two screens.

Here is what I do :


ffmpeg -f v4l2  -framerate 15 -video_size 1280x720 -input_format mjpeg -i
/dev/video1 -f v4l2 -framerate 15  -input_format mjpeg -video_size 1280x720
-i /dev/video0 -filter_complex "
nullsrc=size=2560x720 [background];
[0:v] setpts=PTS-STARTPTS, scale=1280x720 [left];
[1:v] setpts=PTS-STARTPTS, scale=1280x720 [right];
[background][left]       overlay=shortest=1       [background+left];
[background+left][right] overlay=shortest=1:x=1280 [left+right]
" -map [left+right] -r 15  -f rtp_mpegts rtp://192.168.0.5:8888

ffplay -fflags nobuffer rtp://192.168.0.5:8888

Various options I've tried include various combinations of :
 -re flag
 -ts mono2abs -ts abs
 -fflags nobuffer
 different video size and input_format and frame rate


It's not an hardware problem because I'm able using two ffmpeg process and
two ffplay process to stream without synchronisation issue between the two
stream :

ffmpeg -re -fflags nobuffer -f v4l2  -framerate 15 -video_size 1280x720
-input_format mjpeg -i /dev/video1 -f rtp_mpegts rtp://192.168.0.5:8888

ffmpeg -re -fflags nobuffer -f v4l2  -framerate 15 -video_size 1280x720
-input_format mjpeg -i /dev/video0 -f rtp_mpegts rtp://192.168.0.5:9888

ffplay -fflags nobuffer rtp://192.168.0.5:8888
ffplay -fflags nobuffer rtp://192.168.0.5:9888

(When I tried to recombine the rtp stream using a third ffmpeg program it
reintroduces the lag and it also work way worse than the first solution)


Please advise.

Bob

PS:

Output of ffmpeg look normal (the deprecated pixel format warning seems
irrelevant):


ffmpeg version 3.2 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg
--mandir=/usr/share/man --enable-avresample --disable-debug
--enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb
--enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb
--enable-libpulse --enable-libfreetype --enable-gnutls --enable-libx264
--enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame
--enable-libopus --enable-libvpx --enable-libspeex --enable-libass
--enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvidstab
--enable-libwavpack --enable-nvenc
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.100 / 57. 64.100
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Input #0, video4linux2,v4l2, from '/dev/video1':
  Duration: N/A, start: 858.256014, bitrate: N/A
    Stream #0:0: Video: mjpeg, yuvj422p(pc, bt470bg/unknown/unknown),
1280x720, -2 kb/s, 15 fps, 15 tbr, 1000k tbn, 1000k tbc
Input #1, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 858.878636, bitrate: N/A
    Stream #1:0: Video: mjpeg, yuvj422p(pc, bt470bg/unknown/unknown),
1280x720, -2 kb/s, 15 fps, 15 tbr, 1000k tbn, 1000k tbc
[swscaler @ 0x3282800] deprecated pixel format used, make sure you did set
range correctly
[swscaler @ 0x32b9840] deprecated pixel format used, make sure you did set
range correctly
Output #0, rtp_mpegts, to 'rtp://192.168.0.5:8888':
  Metadata:
    encoder         : Lavf57.56.100
    Stream #0:0: Video: mpeg4, yuv420p, 2560x720 [SAR 1:1 DAR 32:9],
q=2-31, 200 kb/s, 15 fps, 90k tbn, 15 tbc (default)
    Metadata:
      encoder         : Lavc57.64.100 mpeg4
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
Stream mapping:
  Stream #0:0 (mjpeg) -> setpts
  Stream #1:0 (mjpeg) -> setpts
  overlay -> Stream #0:0 (mpeg4)
Press [q] to stop, [?] for help
frame=    8 fps=0.0 q=12.5 size=     339kB time=00:00:00.46
bitrate=5942.2kbits/frame=   16 fps= 16 q=31.0 size=     384kB
time=00:00:01.00 bitrate=3144.9kbits/