[FFmpeg-user] streamselect in a realtime application

[Michael Koch]

Am 13.06.2022 um 22:20 schrieb Michael Koch:
> Hello,
>
> I'm using FFmpeg for a realtime wormhole simulation in a planetarium. 
> There are two inputs:
> a) An equirectangular video for the environment.
> b) A realtime stream from a 360° camera, this is mapped inside the 
> wormhole.
>
> Both inputs are stitched together with xstack, and then a suitable 
> remap function is applied. The output is in fisheye format and 
> streamed to the planetarium projector. So far, that's working fine.
>
> But I also want to switch the wormhole on and off at certain times (t1 
> and t2). To do this, I did try two approaches:
>
> Approach 1:
> Two streams are generated, one without and the other with wormhole. 
> Then one of them is selected by a streamselect filter. The drawback of 
> this approach is that double CPU power is required. Both streams must 
> be generated by remap filters, although only one of them is used.
>
> Approach 2:
> I use two sets of xmap and ymap files for the remap filter. One set 
> without and the other set with wormhole:
> [xmap1][xmap2]streamselect at 1=map=0[xmap];[ymap1][ymap2]streamselect at 2=map=0[ymap];[a][xmap][ymap]remap... 
>
> The drawback of this approach is that I must use -loop 1 before each 
> of the four mapping file inputs. Which means the files are reloaded 
> for each frame. This is a huge waste of time. If I don't use -loop 1, 
> then it seems the streamselect filters don't work.

I would like to understand why in some cases -loop 1 is required before 
the input files, and in some cases it can be omitted.

This works without -loop 1:
ffmpeg -i in.mp4 -i xmap.pgm -i ymap.pgm -lavfi [0][1][2]remap out.mp4

When I use streamselect to select one of two mapping files, it doesn't 
work. There is no error message, but it's always the first input used, 
regardless which commands I send to the streamselect filers:
ffmpeg -i in.mp4 -i x1.pgm -i x2.pgm -i y1.pgm -i y2.pgm -lavfi 
[1][2]streamselect at 1=map=0[xmap];[3][4]streamselect at 2=map=0[ymap];[0][xmap][ymap]remap 
out.mp4

However the same example does work when I add -loop 1 before each 
mapping file:
ffmpeg -i in.mp4 -loop 1 -i x1.pgm -loop 1 -i x2.pgm -loop 1 -i y1.pgm 
-loop 1 -i y2.pgm -lavfi 
[1][2]streamselect at 1=map=0[xmap];[3][4]streamselect at 2=map=0[ymap];[0][xmap][ymap]remap 
out.mp4

Another question:
Does -loop 1 mean the file is reloaded from disk for each frame? Or is 
it copied from an internal buffer?

Michael