[FFmpeg-user] SCTE-35 implementation already (bounty)

[Devin Heitmueller]

Hi there,

On Tue, Sep 22, 2020 at 7:23 AM Dennis Mungai <dmngaie at gmail.com> wrote:
> To clarify: Devin's ffmpeg branch (linked above) delivers the first two
> objectives. However, these patches need to be forward-ported to git master,
> as they also require significant fix-ups to be applied to the mpegts muxer.

To expand on this a bit, the patches are off a branch I did a couple
of years ago which I am running in production.  The patches can be
forward ported to master, but they are dependent on a few other
commits earlier in that branch.  This includes some changes to the
MPEG-TS demux, a change to stash the source PTS value as AVPacket side
data, and some changes to libavformat/mux.c to treat SCTE-35 packets
as sparse so that the mux framework doesn't stall waiting for packets.
There might be a couple of other misc patches in there as well which
would need to be ported, but I haven't looked in a while so I might
not be thinking of them.

> That way, a downstream packager can use the SCTE-35 payloads to insert the
> relevant splices to the HLS and DASH manifests.

It's worthwhile to note that the patch series today doesn't require
any actual parsing of the SCTE-35 payload beyond modifying the
pts_adjust field (which is at a fixed offset of every packet).  In
order to construct HLS manifests containing SCTE-35 you would have to
actually decode the SCTE-35 payload to extract certain fields
(depending on which form of SCTE-35 over HLS is being implemented --
IIRC there are at least three different methods).  In-house we do this
with libklscte35 and my long term thought would be to get that library
merged upstream as a dependency (as we did for libklvanc), but there
is cleanup required on the library API itself before any patches using
it in ffmpeg would be accepted upstream.

It's probably also worth noting that the patch series Dennis
referenced has a minor bug in the math that results in the splice
point being off by 1-2 frames, even if not transcoding the actual
video.  It's probably fine for most applications but it's on my TODO
list to go through the math and chase that down.

Regarding the ability to force a keyframe at the splice point, this is
harder than one might expect.  We've implemented the code to adjust
the PTS of the SCTE-35 packet as a BSF on the output side, but in
order to influence the behavior of the encoder we would have to
implement a feedback mechanism to notify the encoder (which is further
upstream in the pipeline) on which frame to force the keyframe.  The
ffmpeg frameworks don't really provide an easy way to propagate
information from a BSF back to an upstream video filter (in general
metadata only flows downstream).   My thought was to implement a video
filter which listens on a UDP socket, and then have the BSF do the
math to calculate the actual PTS of the splice point, and then send an
out-of-band message via UDP back to the video filter to set the
appropriate AVFrame flags to force a key frame when appropriate. **

** Note, I haven't actually prototyped this part yet, so don't know if
it will actually work or if there are some unexpected gotcha related
to what the PTS values are at various points in the pipeline.

Devin

-- 
Devin Heitmueller, Senior Software Engineer
LTN Global Communications
o: +1 (301) 363-1001
w: https://ltnglobal.com  e: devin.heitmueller at ltnglobal.com