[FFmpeg-user] Expected latency with RTSP/TCP H264 stream

[Clément Péron]

Hello,

Replying to myself, I hope this can help other people.

Easier for debugging use -debug_ts and look at the line:
muxer <- pts:18 pts_time:2 dts:18 dts_time:2 duration:1
duration_time:0.111111 size:24883254 latency(total:547.743ms,
demux-decode: 133.354ms/24%, decode 403.334ms/73%)
This will give out the ffmpeg latency (avoid starting and flushing packets).
Which is what I measure myself but easier.

And after more research in the libavcodec/cuviddec.c I found there is this tweak
https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/cuviddec.c#L1049

Adding the "-flags +low_delay" will give you
muxer <- pts:47 pts_time:5.222222 dts:47 dts_time:5.222222 duration:1
duration_time:0.111111 size:24883254 latency(total:104.805ms, decode
96.373ms/91%)

So 105ms, which is quite nice :)
If anyone as another trick, please let me know
Thanks



On Fri, 1 Nov 2024 at 18:06, Clément Péron <peron.clem at gmail.com> wrote:
>
> Hello,
>
> I'm trying to have a low latency stream if possible, less than 500ms.
> By "latency" I mean the delta between a frame is sent by the camera
> and when it's received and decoded by FFMpeg until being available to
> the stdout.
>
> To measure it, I add an overlay from the camera side with the source timestamp
> And I then add a timestamp on the ffmpeg side using the drawtext
> plugin and output it as a BMP.
>
> My measurement show:
> 1080p: 616ms
> 4k: 699ms
>
> I use the Cuvid decoder with the following command:
> ./ffmpeg -allowed_media_types video -c:v h264_cuvid -err_detect
> careful -fflags nobuffer -init_hw_device cuda:0 -max_port 65000
> -min_port 5000 -nostats -re -rtsp_flags prefer_tcp -rtsp_transport tcp
> -surfaces 8 -timeout 1000000 -i
> rtsp://root:XXX@10.98.5.107:554/axis-media/media.amp -vf
> "drawtext=text='%{localtime\:%Y/%m/%d %H\\\\\:%M\\\\\:%S.%N}': x=0:
> y=50: fontcolor=white: fontsize=48: box=1: boxcolor=black at 0.5"
> -pix_fmt rgb24 -vframes 1 output_image_X.bmp
>
> Do you think reducing the latency to less than 500ms is doable?
> Or is there some kind of limitation in the pipeline somewhere?
>
> Thanks for your help