[FFmpeg-devel] [PATCH v2 0/6] WebRTC sub-second live streaming support

[Michael Riedl]

This patch series adds support for WHIP and WHEP (WebRTC-HTTP ingestion protocol
and WebRTC-HTTP egress protocol). It supersedes the previous patch series. Thank
you all for your feedback and suggestions!

The WHIP and WHEP are defined in the following draft RFCs:
- WHIP: https://datatracker.ietf.org/doc/draft-ietf-wish-whip
- WHEP: https://datatracker.ietf.org/doc/draft-murillo-whep/

The implementation builds on FFmpeg's existing RTP, HTTP and SDP support as well
as libdatachannel for the rest. This library is much more lightweight than other
libraries (e.g. libwebrtc). At the same time, using this library avoids
reimplementing parts of WebRTC in FFmpeg.

This patch series was tested with WebRTC servers Dolby.io (formerly Millicast)
and SRS (https://github.com/ossrs/srs). Using a local server, an end-to-end
latency of 3 frames (50 ms at 60 fps) was measured with video-only output, and
12 frames (200 ms at 60 fps) with audio and video output. Using a DeckLink input
device, an end-to-end latency of about 4 frames (70 ms at 60 fps) was measured
with both, video-only and audio and video output.

Using a remote server, only the RTT is added to the end-to-end latency. For
example, using a server in Amsterdam (The Netherlands) from a location in
Salzburg (Austria), with RTT=18ms, an end-to-end latency of 50+18=68 ms was
measured for video-only output.


Michael Riedl (6):
  libavformat/http: expose actual Location header value
  libavformat/sdp: remove whitespaces in fmtp
  configure: add libdatachannel as external library
  libavformat/webrtc: add common code for WebRTC streaming
  libavformat/webrtc_demux: add WebRTC-HTTP egress protocol (WHEP)
    demuxer
  libavformat/webrtc_mux: add WebRTC-HTTP ingestion protocol (WHIP)
    muxer

 Changelog                        |   5 +
 MAINTAINERS                      |   1 +
 configure                        |   7 +
 doc/demuxers.texi                |  22 ++
 doc/muxers.texi                  |  21 ++
 libavformat/Makefile             |   2 +
 libavformat/allformats.c         |   2 +
 libavformat/http.c               |   1 +
 libavformat/sdp.c                |   4 +-
 libavformat/webrtc.c (new)       | 410 +++++++++++++++++++++++++++++++
 libavformat/webrtc.h (new)       |  70 ++++++
 libavformat/webrtc_demux.c (new) | 246 +++++++++++++++++++
 libavformat/webrtc_mux.c (new)   | 273 ++++++++++++++++++++
 13 files changed, 1062 insertions(+), 2 deletions(-)
 create mode 100644 libavformat/webrtc.c
 create mode 100644 libavformat/webrtc.h
 create mode 100644 libavformat/webrtc_demux.c
 create mode 100644 libavformat/webrtc_mux.c

--
2.39.2