[FFmpeg-user] sync 2 live input streams

[Madovsky]

----- Original Message ----- 
From: "Carl Eugen Hoyos" <cehoyos at ag.or.at>
To: <ffmpeg-user at ffmpeg.org>
Sent: Saturday, February 02, 2013 4:10 AM
Subject: Re: [FFmpeg-user] sync 2 live input streams


> Madovsky <infos <at> madovsky.org> writes:
>
>> I'm trying to resync 2 live input live streams asynced due
>> to filter complex overlay
>
> Is there a bug that we should know about?
>
>> with -itsoffset but unfortunately
>> the audio from the second input doesn't follow the offset.
>
> Failing command line together with complete, uncut
> console output missing / works fine here.
>
> Carl Eugen

I thought it was a normal behaviour as filter_complex needed
a headroom to overlay... so theorically are 2 same input live overlayed 
streams
must run without delay ?
for latency test I use the same live source stream:

==================

ffmpeg -re -i "rtmp://server/app/stream" -re -i 
"rtmp://server/app/stream" -i back.png -map 0:1 -map 1:0 -filter_complex 
"[0:0]pad=682:450:21:105[left];[left][1:0]overlay=341:105[base];[base][2:0]overlay=0:main_h-20; 
[0:1][1:1]amerge=inputs=2" -r 24 -tune zerolatency -ar 44100 -b:a 192k -b:v 
768k -shortest -y output.mkv
ffmpeg version N-49519-g99eedfc Copyright (c) 2000-2013 the FFmpeg 
developers
  built on Feb  1 2013 16:33:20 with gcc 4.3.2 (GCC) 20081105 (Red Hat 
4.3.2-7)
  configuration: --enable-static --enable-gpl --enable-nonfree --enable-libfaac 
 --enable-libx264 --enable-pthreads --enable-postproc --enable-libspeex --enable-librtmp 
 --enable-runtime-cpudetect --enable-libfreetype
  libavutil      52. 17.100 / 52. 17.100
  libavcodec     54. 91.100 / 54. 91.100
  libavformat    54. 61.104 / 54. 61.104
  libavdevice    54.  3.103 / 54.  3.103
  libavfilter     3. 35.100 /  3. 35.100
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
rtmp server sent error
[flv @ 0x2d12680] Estimating duration from bitrate, this may be inaccurate
Input #0, flv, from rtmp://server/app/stream':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: flv1, yuv420p, 320x240, 1k tbr, 1k tbn, 1k tbc
    Stream #0:1: Audio: speex, 16000 Hz, mono, s16
rtmp server sent error
[flv @ 0x2d35dc0] Estimating duration from bitrate, this may be inaccurate
Input #1, flv, from rtmp://server/app/stream':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #1:0: Video: flv1, yuv420p, 320x240, 16.58 tbr, 1k tbn, 1k tbc
    Stream #1:1: Audio: speex, 16000 Hz, mono, s16
Input #2, image2, from 'neterviewBack.png':
  Duration: 00:00:00.04, start: 0.000000, bitrate: N/A
    Stream #2:0: Video: png, rgb24, 80x20, 25 tbr, 25 tbn, 25 tbc
[Parsed_amerge_3 @ 0x2dea1e0] Input channel layouts overlap: output layout 
will be determined by the number of distinct input channels
[libx264 @ 0x2d48020] using cpu capabilities: MMX2 SSE2 SSE3 Cache64
[libx264 @ 0x2d48020] profile High, level 3.0
[libx264 @ 0x2d48020] 264 - core 129 r2245 bc13772 - H.264/MPEG-4 AVC 
codec - Copyleft 2003-2013 - http://www.videolan.org/x264.html - options: 
cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 
psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 
cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=4 
lookahead_threads=4 sliced_threads=1 slices=4 nr=0 decimate=1 interlaced=0 
bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=250 
keyint_min=24 scenecut=40 intra_refresh=0 rc=abr mbtree=0 bitrate=768 
ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'output.mkv':
  Metadata:
    encoder         : Lavf54.61.104
    Stream #0:0: Video: h264 (H264 / 0x34363248), yuv420p, 682x450, q=-1--1, 
768 kb/s, 1k tbn, 24 tbc
    Stream #0:1: Audio: ac3 ([0] [0][0] / 0x2000), 44100 Hz, stereo, fltp, 
192 kb/s
Stream mapping:
  Stream #0:0 (flv) -> pad
  Stream #0:1 (libspeex) -> amerge:in0
  Stream #1:0 (flv) -> overlay:overlay
  Stream #1:1 (libspeex) -> amerge:in1
  Stream #2:0 (png) -> overlay:overlay
  overlay -> Stream #0:0 (libx264)
  amerge -> Stream #0:1 (ac3)
Press [q] to stop, [?] for help
Buffer queue overflow=18.0 size=     645kB time=00:00:07.66 bitrate= 
689.9kbits/s dup=0 drop=12
[Parsed_amerge_3 @ 0x2dea1e0] Buffer queue overflow
    Last message repeated 6 times
Buffer queue overflow=20.0 size=     869kB time=00:00:10.13 bitrate= 
702.3kbits/s dup=0 drop=12
Buffer queue overflow=20.0 size=    1093kB time=00:00:12.50 bitrate= 
716.3kbits/s dup=0 drop=12
[Parsed_amerge_3 @ 0x2dea1e0] Buffer queue overflow
    Last message repeated 14 times
Buffer queue overflow=21.0 size=    1093kB time=00:00:12.58 bitrate= 
711.5kbits/s dup=0 drop=12
[Parsed_amerge_3 @ 0x2dea1e0] Buffer queue overflow
    Last message repeated 23 times
Buffer queue overflow=21.0 size=    1093kB time=00:00:12.58 bitrate= 
711.5kbits/s dup=0 drop=12
[Parsed_amerge_3 @ 0x2dea1e0] Buffer queue overflow
    Last message repeated 23 times
Buffer queue overflow=21.0 size=    1093kB time=00:00:12.58 bitrate= 
711.5kbits/s dup=0 drop=12
[Parsed_amerge_3 @ 0x2dea1e0] Buffer queue overflow
    Last message repeated 11 times
Buffer queue overflow=21.0 size=    1157kB time=00:00:13.05 bitrate= 
726.2kbits/s dup=0 drop=12
Buffer queue overflow=19.0 size=    1157kB time=00:00:13.66 bitrate= 
693.5kbits/s dup=0 drop=12
[Parsed_amerge_3 @ 0x2dea1e0] Buffer queue overflow
    Last message repeated 54 times
Buffer queue overflow=19.0 size=    1157kB time=00:00:14.99 bitrate= 
631.9kbits/s dup=0 drop=12
[Parsed_amerge_3 @ 0x2dea1e0] Buffer queue overflow
    Last message repeated 82 times
Buffer queue overflow=19.0 size=    1157kB time=00:00:15.62 bitrate= 
606.6kbits/s dup=0 drop=12
[Parsed_amerge_3 @ 0x2dea1e0] Buffer queue overflow
    Last message repeated 23 times
Buffer queue overflow=19.0 size=    1157kB time=00:00:15.62 bitrate= 
606.6kbits/s dup=0 drop=12
[Parsed_amerge_3 @ 0x2dea1e0] Buffer queue overflow
    Last message repeated 2 times
frame=  243 fps= 14 q=19.0 Lsize=    1277kB time=00:00:15.62 bitrate= 
669.4kbits/s dup=0 drop=12
video:967kB audio:304kB subtitle:0 global headers:0kB muxing overhead 
0.438866%
[libx264 @ 0x2d48020] frame I:1     Avg QP:20.57  size:  7868
[libx264 @ 0x2d48020] frame P:242   Avg QP:11.39  size:  4055
[libx264 @ 0x2d48020] mb I  I16..4: 67.8% 22.8%  9.5%
[libx264 @ 0x2d48020] mb P  I16..4:  0.8%  1.7%  0.6%  P16..4: 16.2%  6.8% 
5.7%  0.0%  0.0%    skip:68.2%
[libx264 @ 0x2d48020] final ratefactor: 21.68
[libx264 @ 0x2d48020] 8x8 transform intra:51.6% inter:43.2%
[libx264 @ 0x2d48020] coded y,uvDC,uvAC intra: 49.9% 80.0% 74.5% inter: 
11.2% 21.8% 14.2%
[libx264 @ 0x2d48020] i16 v,h,dc,p: 56% 26% 11%  8%
[libx264 @ 0x2d48020] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 17% 28%  2%  7%  6% 
6%  4%  6%
[libx264 @ 0x2d48020] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 39% 21% 19%  3%  6%  5% 
4%  3%  2%
[libx264 @ 0x2d48020] i8c dc,h,v,p: 53% 12% 29%  5%
[libx264 @ 0x2d48020] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x2d48020] ref P L0: 60.9%  4.7% 23.6% 10.7%
[libx264 @ 0x2d48020] kb/s:781.66

================

the "Buffer queue overflow" is due when I stopped the live stream so it's 
normal I think.
so oddly the latency comes from the left stream , not the right overlayed at 
the end of the filter_complex.
there are about 2.4 seconds delay

Thanks

Franck