[FFmpeg-user] Stitching two overlapping fisheye videos with maskedmerge

[Paul B Mahol]

Hi,

On 3/3/20, Michael Koch <astroelectronic at t-online.de> wrote:
> I'd like to stitch two fisheye videos together to an equirectangular
> video. The input videos have larger than 180° field of view, so that
> there is an overlap region and I want to use the maskedmerge filter for
> merging the videos.
> My first (successful) test is with png images:
>
> First create the mask file for maskedmerge:
> ffmpeg -f lavfi -i nullsrc=size=640x640 -vf
> "format=gray8,geq='clip(128-128/10*(180-191.5/(640/2)*hypot(X-640/2,Y-640/2)),0,255)',v360=fisheye:e:ih_fov=191.5:iv_fov=191.5,format=rgb24"
> -frames 1 -y mergemap.png
> The pixel values in this image are 0 for one fisheye region, 255 for the
> other fisheye region and a 0-255 ramp in the overlap region.
>
> Now create two fisheye inputs (one is red and the other is green) and
> transform them to equirectangular:
> ffmpeg -f lavfi -i color=red:size=640x640 -lavfi
> "drawbox=w=1:h=1:color=black,v360=input=fisheye:output=e:ih_fov=191.5:iv_fov=191.5"
> -frames 1 -y eq1.png
> ffmpeg -f lavfi -i color=green:size=640x640 -lavfi
> "drawbox=w=1:h=1:color=black,v360=input=fisheye:output=e:ih_fov=191.5:iv_fov=191.5:yaw=180"
> -frames 1 -y eq2.png
>
> Now merge the two images with maskedmerge:
> ffmpeg -i eq1.png -i eq2.png -i mergemap.png -lavfi "maskedmerge"
> -frames 1 -y out.png
>
> Up to this point, everything did work as expected. The output image
> looks correct. As the next step, I want to do the same thing with
> videos. I replace the file extensions by *.mp4, I replace "-frames 1" by
> "-t 5", and I add "-loop 1" for the third input file (which is the same
> as before):
> ffmpeg -f lavfi -i color=red:size=640x640 -lavfi
> "drawbox=w=1:h=1:color=black,v360=input=fisheye:output=e:ih_fov=191.5:iv_fov=191.5"
> -t 5 -y eq1.mp4
> ffmpeg -f lavfi -i color=green:size=640x640 -lavfi
> "drawbox=w=1:h=1:color=black,v360=input=fisheye:output=e:ih_fov=191.5:iv_fov=191.5:yaw=180"
> -t 5 -y eq2.mp4
> ffmpeg -i eq1.mp4 -i eq2.mp4 -loop 1 -i mergemap.png -lavfi
> "maskedmerge" -t 5 -y out.mp4
>
> The intermediate videos eq1.mp4 and eq2.mp4 look correct, but the output
> video is wrong.
> I did already try many format conversions, but I haven't yet figured out
> what's going wrong here.
> Below is the console output for the last command line.

ffmpeg -i eq1.mp4 -i eq2.mp4 -loop 1 -i mergemap.png -lavfi
"[0:v]format=gbrp[a],[1:v]format=gbrp[b],[2:v]format=gbrp[c],[a][b][c]maskedmerge,format=gbrp"
-t 5 -y out.mp4

If input videos are in YUV colorspace, better make map also in YUV
colorspace and not in RGB colorspace.