[FFmpeg-user] Using FFMpeg to produce LL-HLS

[Ted Park]

Hi Simon,

> No, I'm not going to re-encode - I cannot afford the CPU power.
But can you be sure that the latency is due to the network and not because the original stream is a little funky at the beginning and it takes a while to set up the decoder for playback? If the latter is the case re-encoding may be the only choice, besides, I don’t think LL-HLS extensions are going to help if you don’t have a stream that is encoded for it (with GOP of ~2 seconds, etc) and you have a server with a pretty advanced HTTP/2 config.

> Yes, that is the only error.  I have done this:
> C:\xampp\htdocs>\ffmpeg-20200216-8578433-win64-static\bin\ffmpeg.exe
> -fflags igndts+nofillin -i udp://127.0.0.1:9034 <udp://127.0.0.1:9034> -map 0 -c:v copy -an -f
> h264 udp://127.0.0.1:9036 <udp://127.0.0.1:9036>
> in one DOS prompt, and this:
> C:\xampp\htdocs>\ffmpeg-20200216-8578433-win64-static\bin\ffmpeg.exe -i
> udp://127.0.0.1:9036 <udp://127.0.0.1:9036> -codec copy -an -window_size 5 -extra_window_size 5
> -use_timeline 1 -seg_duration 1 -frag_duration 0.2 -streaming 1
> -adaptation_sets "id=0,streams=v id=1,streams=a" -dash_segment_type mp4
> -ldash 1 -strict experimental -f dash manifest.mpd
> in the second DOS prompt.  This gives me a stream that I can view in a
> browser, but latency is 8s which is not acceptable.  We need it <1s if at
> all possible.

Is there a reason for igndts and nofillin? Or a reason you need to stream raw udp instead of, for example, saving to a scratch disk and removing later by script?

In the first place you have not been able to achieve lower “latency” using any other method right? I’m just wondering if it’s the original encoded stream that is at the root of the problem, and not how it is served over the network.

Just to make sure, by latency do you mean the delay when you play the results of the second command, compared to if you were to play the output of the first command?

> So clearly if I send an h264 stream at the dash muxer it is happy, yet if I
> just use the ts with -map 0 -c:v copy -an it doesn't work and gives the
> earlier error.
> There are other side effects in that it starts complaining about
> non-monotonous DTS but that goes away after a while:
> ffmpeg version git-2020-02-16-8578433 Copyright (c) 2000-2020 the FFmpeg
> developers
>  built with gcc 9.2.1 (GCC) 20200122
>  configuration: --enable-gpl --enable-version3 --enable-sdl2
> --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass
> --enable-libdav1d --enable-libbluray --enable-libfreetype
> --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb
> --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy
> --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx
> --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265
> --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp
> --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc
> --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom
> --enable-libmfx --enable-ffnvcodec --enable-cuvid --enable-d3d11va
> --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth
> --enable-libopenmpt --enable-amf
>  libavutil      56. 41.100 / 56. 41.100
>  libavcodec     58. 70.100 / 58. 70.100
>  libavformat    58. 38.101 / 58. 38.101
>  libavdevice    58.  9.103 / 58.  9.103
>  libavfilter     7. 76.100 /  7. 76.100
>  libswscale      5.  6.100 /  5.  6.100
>  libswresample   3.  6.100 /  3.  6.100
>  libpostproc    55.  6.100 / 55.  6.100
> [mov,mp4,m4a,3gp,3g2,mj2 @ 000001b04824bec0] DTS discontinuity in stream 0:
> packet 3 with DTS 400001, packet 4 with DTS 800000
> [mov,mp4,m4a,3gp,3g2,mj2 @ 000001b04824bec0] DTS discontinuity in stream 0:
> packet 5 with DTS 800001, packet 6 with DTS 1200000
> [mov,mp4,m4a,3gp,3g2,mj2 @ 000001b04824bec0] DTS discontinuity in stream 0:
> packet 7 with DTS 1200001, packet 8 with DTS 1600000
> [mov,mp4,m4a,3gp,3g2,mj2 @ 000001b04824bec0] DTS discontinuity in stream 0:
> packet 9 with DTS 1600001, packet 10 with DTS 2000000
> [mov,mp4,m4a,3gp,3g2,mj2 @ 000001b04824bec0] DTS discontinuity in stream 0:
> packet 11 with DTS 2000001, packet 12 with DTS 2400000
> [mov,mp4,m4a,3gp,3g2,mj2 @ 000001b04824bec0] DTS discontinuity in stream 0:
> packet 13 with DTS 2400001, packet 14 with DTS 9800000
> [mov,mp4,m4a,3gp,3g2,mj2 @ 000001b04824bec0] DTS discontinuity in stream 0:
> packet 15 with DTS 9800001, packet 16 with DTS 10200000
> [mov,mp4,m4a,3gp,3g2,mj2 @ 000001b04824bec0] DTS discontinuity in stream 0:
> packet 17 with DTS 10200001, packet 18 with DTS 10600000
> [mov,mp4,m4a,3gp,3g2,mj2 @ 000001b04824bec0] DTS discontinuity in stream 0:
> packet 19 with DTS 10600001, packet 20 with DTS 11000000
> Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'udp://127.0.0.1:9036?listen': <udp://127.0.0.1:9036?listen':>
>  Metadata:
>    major_brand     : isml
>    minor_version   : 512
>    compatible_brands: ismlpiff
>    encoder         : Lavf58.38.101
>  Duration: 00:00:02.70, start: 0.000000, bitrate: N/A
>    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv),

Okay why is it using the mov demuxer here and not mpegts I’m confused. This is when you stream mpegts over udp?

> Is the latency because of all the chaff at the start, or is it because of
> FFMpeg's start up time, or something else?

Try comparing the results of playing the original file, the h264 stream output by the first command, and playing the final output using the first and second command you included first (the one that worked, albeit with unacceptable latency). That should make it clear.

> I'm trying to reduce the latency of streaming video to a web page.
> I have tried a few options:
> 1) WebRTC - great, but doesn't support interlaced video
> 2) fMP4 - couldn't get a working solution
> 3) HLS - great, plays interlaced video as well, but latency is appalling
> 4) LL-HLS - can't find anything that produces it.

Apple has a media stream segmenter that I assume is supposed to serve as the reference implementation of LL-HLS if you have a Mac to try it out.

Regards,
Ted