[FFmpeg-user] Concatenation Much Slower with Process Substitution for File List

[Scott Colby]

Hello,

I am a bit perplexed about the difference in speed between these two commands:

ffmpeg \
    -f concat -safe 0 -i <(
        find . -type f -iname '*.mp3' -maxdepth 1 -print0 |
            xargs -0 -- realpath |
            sort -n |
            sed -e "s/'/'\\\''/g" -e "s/^/file '/" -e "s/$/'/" |
            tee flist
    ) \
    -i cover.jpg \
    -c:a copy -c:v copy \
    out.mp3

ffmpeg -f concat -safe 0 -i flist -i cover.jpg -c:a copy -c:v copy out.mp3

The former runs at about speed=30x while the latter runs at speed=1.29e+04x.

(Note that I'm using the same file list for the second command from the output
of tee in the first command.)

If I remove the `-i cover.jpg` part from either command, they both run at about
speed=5.39e+03x. (It's strange that it's slower when doing less work, but that's
not the main question I'm asking here.)

Changing the order of the inputs to put the cover image first does not affect
the speed.

I am running with bash 3.2.57(1)-release (arm64-apple-darwin23) on macOS 14.2.1.

When I run the fish-equivalent command with fish 3.7.0, I get the high speed
output:

ffmpeg -f concat -safe 0 -i (
    find . -type f -iname '*.mp3' -maxdepth 1 -print0 |
    xargs -0 -- realpath |
    sort -n |
    sed -e "s/'/'\\\''/g" -e "s/^/file '/" -e "s/\%/'/" |
    psub
) -i cover.jpg -c:a copy -c:v copy out.mp3

I believe this is because fish uses a temporary file for process substitution
instead of /dev/fd/63 like bash does.

I have run all these commands with -loglevel debug and do not see any
significant differences in the log output; the only differences are in the exact
values in the log tags at the beginning of the lines (e.g. [mp3 @ 0x130005a90]),
but those are different for every run anyway.

Why is there this vast difference in performance between the normal file input
and the process substitution input? Is there another flag that I can pass to get
the high-speed processing when using a process substitution?

Thanks,
Scott