[FFmpeg-user] Synchronized xstack or overlay content

[Troy]

Hi there,

I'm struggling to build a command that will synchronise xstack content, as a "stretch goal" I'd like to also keep audio in sync, but that is less important.

It appears that because each input is started in sequence, and then buffered until all inputs are active, the first input can be minutes old by the time the stack is ready to display.

This is for live content, not generating a mosaic of pre-recorded content.

I feel like what I need to be doing is dropping frames from any active input until all the inputs are active - but I just can't work out how to do that.

Really appreciate any advice on things I may be doing wrong. The goal is a live 3x3 mosaic of inputs. The inputs are currently sourced from UDP broadcast traffic inside a docker network.

The command currently:
ffmpeg -nostdin \-hide_banner \
-loglevel \
${FFMPEG_LOG_LEVEL:-error} \
-err_detect \
ignore_err \
-y \
-hwaccel \
cuda \
-fflags \
nobuffer \
-flags \
low_delay \
-fflags \
+genpts+discardcorrupt \
-thread_queue_size \
${FFMPEG_THREAD_QUEUE_SIZE:-4096} \
-i \
${SOURCE_ONE} \
-hwaccel \
cuda \
-fflags \
nobuffer \
-flags \
low_delay \
-fflags \
+genpts+discardcorrupt \
-thread_queue_size \
${FFMPEG_THREAD_QUEUE_SIZE:-4096} \
-i \
${SOURCE_TWO} \
-hwaccel \
cuda \
-fflags \
nobuffer \
-flags \
low_delay \
-fflags \
+genpts+discardcorrupt \
-thread_queue_size \
${FFMPEG_THREAD_QUEUE_SIZE:-4096} \
-i \
${SOURCE_THREE} \
-hwaccel \
cuda \
-fflags \
nobuffer \
-flags \
low_delay \
-fflags \
+genpts+discardcorrupt \
-thread_queue_size \
${FFMPEG_THREAD_QUEUE_SIZE:-4096} \
-i \
${SOURCE_FOUR} \
-hwaccel \
cuda \
-fflags \
nobuffer \
-flags \
low_delay \
-fflags \
+genpts+discardcorrupt \
-thread_queue_size \
${FFMPEG_THREAD_QUEUE_SIZE:-4096} \
-i \
${SOURCE_FIVE} \
-hwaccel \
cuda \
-fflags \
nobuffer \
-flags \
low_delay \
-fflags \
+genpts+discardcorrupt \
-thread_queue_size \
${FFMPEG_THREAD_QUEUE_SIZE:-4096} \
-i \
${SOURCE_SIX} \
-hwaccel \
cuda \
-fflags \
nobuffer \
-flags \
low_delay \
-fflags \
+genpts+discardcorrupt \
-thread_queue_size \
${FFMPEG_THREAD_QUEUE_SIZE:-4096} \
-i \
${SOURCE_SEVEN} \
-hwaccel \
cuda \
-fflags \
nobuffer \
-flags \
low_delay \
-fflags \
+genpts+discardcorrupt \
-thread_queue_size \
${FFMPEG_THREAD_QUEUE_SIZE:-4096} \
-i \
${SOURCE_EIGHT} \
-hwaccel \
cuda \
-fflags \
nobuffer \
-flags \
low_delay \
-fflags \
+genpts+discardcorrupt \
-thread_queue_size \
${FFMPEG_THREAD_QUEUE_SIZE:-4096} \
-i \
${SOURCE_NINE} \
-filter_complex \
[0:v]setpts=PTS-STARTPTS,scale=nhd[a0];[1:v]setpts=PTS-STARTPTS,scale=nhd[a1];[2:v]setpts=PTS-STARTPTS,scale=nhd[a2];[3:v]setpts=PTS-STARTPTS,scale=nhd[a3];[4:v]setpts=PTS-STARTPTS,scale=nhd[a4];[5:v]setpts=PTS-STARTPTS,scale=nhd[a5];[6:v]setpts=PTS-STARTPTS,scale=nhd[a6];[7:v]setpts=PTS-STARTPTS,scale=nhd[a7];[8:v]setpts=PTS-STARTPTS,scale=nhd[a8];[a0][a1][a2][a3][a4][a5][a6][a7][a8]xstack=inputs=9:layout=0_0|w0_0|w0+w1_0|0_h0|w0_h0|w0+w1_h0|0_h0+h1|w0_h0+h1|w0+w1_h0+h1:shortest=1,fps=24[out] \
-map \
[out] \
-codec:v \
h264_nvenc \
-preset:v \
medium \
-b:v \
16384k \
-maxrate \
16384k \
-bufsize \
16384k \
-pix_fmt \
yuv420p \
-g \
48 \
-map \
0:a? \
-c:a \
aac \
-map \
1:a? \
-c:a \
aac \
-map \
2:a? \
-c:a \
aac \
-map \
3:a? \
-c:a \
aac \
-map \
4:a? \
-c:a \
aac \
-map \
5:a? \
-c:a \
aac \
-map \
6:a? \
-c:a \
aac \
-map \
7:a? \
-c:a \
aac \
-map \
8:a? \
-c:a \
aac \
-metadata \
title=Mosaic \
-metadata \
service_provider=ffmpeg \
-max_muxing_queue_size \
9000000 \
-threads \
0 \
-f \
matroska \
tcp://0.0.0.0:${STREAMING_TCP_PORT:-7000}?listen=1&tcp_mss=1360

Regards, Troy
Brevity is the elixir of life.
Father Hector McGrath, Pixie 2020
-------------- next part --------------
A non-text attachment was scrubbed...
Name: publickey - troy at troykelly.com - 0x3DB9B3FB.asc
Type: application/pgp-keys
Size: 3109 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-user/attachments/20230127/388b18b6/attachment.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 855 bytes
Desc: OpenPGP digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-user/attachments/20230127/388b18b6/attachment.sig>