[FFmpeg-user] Stitching two overlapping fisheye videos with maskedmerge
Michael Koch
astroelectronic at t-online.de
Tue Mar 3 11:33:10 EET 2020
I'd like to stitch two fisheye videos together to an equirectangular
video. The input videos have larger than 180° field of view, so that
there is an overlap region and I want to use the maskedmerge filter for
merging the videos.
My first (successful) test is with png images:
First create the mask file for maskedmerge:
ffmpeg -f lavfi -i nullsrc=size=640x640 -vf
"format=gray8,geq='clip(128-128/10*(180-191.5/(640/2)*hypot(X-640/2,Y-640/2)),0,255)',v360=fisheye:e:ih_fov=191.5:iv_fov=191.5,format=rgb24"
-frames 1 -y mergemap.png
The pixel values in this image are 0 for one fisheye region, 255 for the
other fisheye region and a 0-255 ramp in the overlap region.
Now create two fisheye inputs (one is red and the other is green) and
transform them to equirectangular:
ffmpeg -f lavfi -i color=red:size=640x640 -lavfi
"drawbox=w=1:h=1:color=black,v360=input=fisheye:output=e:ih_fov=191.5:iv_fov=191.5"
-frames 1 -y eq1.png
ffmpeg -f lavfi -i color=green:size=640x640 -lavfi
"drawbox=w=1:h=1:color=black,v360=input=fisheye:output=e:ih_fov=191.5:iv_fov=191.5:yaw=180"
-frames 1 -y eq2.png
Now merge the two images with maskedmerge:
ffmpeg -i eq1.png -i eq2.png -i mergemap.png -lavfi "maskedmerge"
-frames 1 -y out.png
Up to this point, everything did work as expected. The output image
looks correct. As the next step, I want to do the same thing with
videos. I replace the file extensions by *.mp4, I replace "-frames 1" by
"-t 5", and I add "-loop 1" for the third input file (which is the same
as before):
ffmpeg -f lavfi -i color=red:size=640x640 -lavfi
"drawbox=w=1:h=1:color=black,v360=input=fisheye:output=e:ih_fov=191.5:iv_fov=191.5"
-t 5 -y eq1.mp4
ffmpeg -f lavfi -i color=green:size=640x640 -lavfi
"drawbox=w=1:h=1:color=black,v360=input=fisheye:output=e:ih_fov=191.5:iv_fov=191.5:yaw=180"
-t 5 -y eq2.mp4
ffmpeg -i eq1.mp4 -i eq2.mp4 -loop 1 -i mergemap.png -lavfi
"maskedmerge" -t 5 -y out.mp4
The intermediate videos eq1.mp4 and eq2.mp4 look correct, but the output
video is wrong.
I did already try many format conversions, but I haven't yet figured out
what's going wrong here.
Below is the console output for the last command line.
Michael
C:\Users\mKoch\Desktop>c:\ffmpeg\ffmpeg -i eq1.mp4 -i eq2.mp4 -loop 1 -i
mergema
p.png -lavfi "maskedmerge" -t 5 -y out.mp4
ffmpeg version git-2020-03-01-1281399 Copyright (c) 2000-2020 the FFmpeg
develop
ers
built with gcc 9.2.1 (GCC) 20200122
configuration: --enable-gpl --enable-version3 --enable-sdl2
--enable-fontconfi
g --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d
--enable-libb
luray --enable-libfreetype --enable-libmp3lame
--enable-libopencore-amrnb --enab
le-libopencore-amrwb --enable-libopenjpeg --enable-libopus
--enable-libshine --e
nable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame
--enable
-libvpx --enable-libwavpack --enable-libwebp --enable-libx264
--enable-libx265 -
-enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib
--enable-gmp --enab
le-libvidstab --enable-libvorbis --enable-libvo-amrwbenc
--enable-libmysofa --en
able-libspeex --enable-libxvid --enable-libaom --enable-libmfx
--enable-ffnvcode
c --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc
--enable-nvd
ec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
libavutil 56. 42.100 / 56. 42.100
libavcodec 58. 73.102 / 58. 73.102
libavformat 58. 39.101 / 58. 39.101
libavdevice 58. 9.103 / 58. 9.103
libavfilter 7. 77.100 / 7. 77.100
libswscale 5. 6.100 / 5. 6.100
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'eq1.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.39.101
Duration: 00:00:05.00, start: 0.000000, bitrate: 22 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p,
1280x640
[SAR 1:1 DAR 2:1], 19 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
handler_name : VideoHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'eq2.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.39.101
Duration: 00:00:05.00, start: 0.000000, bitrate: 21 kb/s
Stream #1:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p,
1280x640
[SAR 1:1 DAR 2:1], 17 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
Metadata:
handler_name : VideoHandler
Input #2, png_pipe, from 'mergemap.png':
Duration: N/A, bitrate: N/A
Stream #2:0: Video: png, rgb24(pc), 1280x640 [SAR 1:1 DAR 2:1], 25
fps, 25 t
br, 25 tbn, 25 tbc
Stream mapping:
Stream #0:0 (h264) -> maskedmerge:base
Stream #1:0 (h264) -> maskedmerge:overlay
Stream #2:0 (png) -> maskedmerge:mask
maskedmerge -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[libx264 @ 00000000029ac840] using SAR=1/1
[libx264 @ 00000000029ac840] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2
AVX FMA3 BMI2 AVX2
[libx264 @ 00000000029ac840] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 00000000029ac840] 264 - core 159 - H.264/MPEG-4 AVC codec -
Copyleft
2003-2019 - http://www.videolan.org/x264.html - options: cabac=1 ref=3
deblock=1
:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1
me_rang
e=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1
chroma_qp_
offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1
interla
ced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2
b_adapt=1 b_bias
=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25
scenecut=40
intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60
qpmin=0 qpma
x=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'out.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.39.101
Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p,
1280x640 [S
AR 1:1 DAR 2:1], q=-1--1, 25 fps, 12800 tbn, 25 tbc (default)
Metadata:
encoder : Lavc58.73.102 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame= 47 fps=0.0 q=0.0 size= 0kB time=00:00:00.00 bitrate=N/A
speed=
frame= 96 fps= 93 q=28.0 size= 0kB time=00:00:01.72 bitrate=
0.2kbits/
frame= 125 fps= 88 q=-1.0 Lsize= 15kB time=00:00:04.88 bitrate=
24.7kbits
/s speed=3.44x
video:12kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing ov
erhead: 18.545912%
[libx264 @ 00000000029ac840] frame I:1 Avg QP:17.92 size: 6000
[libx264 @ 00000000029ac840] frame P:31 Avg QP:16.01 size: 71
[libx264 @ 00000000029ac840] frame B:93 Avg QP:12.67 size: 41
[libx264 @ 00000000029ac840] consecutive B-frames: 0.8% 0.0% 0.0% 99.2%
[libx264 @ 00000000029ac840] mb I I16..4: 9.3% 89.0% 1.7%
[libx264 @ 00000000029ac840] mb P I16..4: 0.0% 0.0% 0.0% P16..4:
0.1% 0.3
% 0.0% 0.0% 0.0% skip:99.5%
[libx264 @ 00000000029ac840] mb B I16..4: 0.0% 0.0% 0.0% B16..8:
0.1% 0.0
% 0.0% direct: 0.0% skip:99.9% L0:84.8% L1:15.2% BI: 0.0%
[libx264 @ 00000000029ac840] 8x8 transform intra:88.9% inter:100.0%
[libx264 @ 00000000029ac840] coded y,uvDC,uvAC intra: 1.8% 7.4% 6.6%
inter: 0.0%
0.0% 0.0%
[libx264 @ 00000000029ac840] i16 v,h,dc,p: 56% 17% 28% 0%
[libx264 @ 00000000029ac840] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 68% 3% 28%
0% 0%
0% 0% 0% 0%
[libx264 @ 00000000029ac840] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 42% 26% 19%
3% 1%
3% 5% 2% 2%
[libx264 @ 00000000029ac840] i8c dc,h,v,p: 88% 6% 6% 1%
[libx264 @ 00000000029ac840] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 00000000029ac840] ref P L0: 99.2% 0.2% 0.4% 0.2%
[libx264 @ 00000000029ac840] ref B L0: 19.8% 80.2%
[libx264 @ 00000000029ac840] kb/s:19.25
More information about the ffmpeg-user
mailing list