[FFmpeg-user] Generating aligned HLS segments for ABR delivery

[Stefan Moro]

On 30 March 2016 at 11:39, Stefan Moro <stefan.moro at gmail.com> wrote:

> Hi,
>
> I am using the segment muxer as part of a system to create an adaptive HLS
> stream. The segment muxer creates MPEG-TS segments from key-frame-aligned,
> closed-GOP and audio aligned renditions of the same content. The renditions
> are transcoded such that is should be possible to segment them into PTS
> aligned, equal length segments.
>
> One of the renditions is encoded with H264 high profile, which utilizes
> B-frames. When the high profile rendition is segmented, I see that it has a
> different start PTS than the segments of the other renditions. It seems
> that this causes the number of audio frames in the high profile segments to
> differ from the number of audio frames in the other segments. I am aware
> that it might make sense to change the start PTS of an MPEG TS in order to
> avoid a negative DTS, but since all renditions start with a key-frame, we
> should not encounter a negative DTS.
>
> What I want is to not change any timing information at all, which would
> ensure that each segment has the same number of video- and audio frames
> with the same timing information. Is there any way for me to do this? I
> have tried several combinations of vsync, copyts, avoid_negative_ts etc,
> but with no success. The segments of the high profile rendition always has
> a different number of audio frames compared the segments of the other
> renditions.
> If it would help, adding the same offset to all A/V timestamps would also
> be OK, e.g. adding 10 seconds to every timestamp. The important thing is
> that all timestamps stay the same, relative to each other.
>
> Below is the version of ffmpeg I last tried with and one version of the
> ffmpeg command line I tried with. As I mentioned above, I also tried many
> other variations using -copyts, -vsync, avoid_negative_ts, muxdelay,
> mpegts_copyts etc but I'm posting the "basic" version of the command here.
>
> Please see the following link for the script and the video renditions I am
> using to test:
> https://www.dropbox.com/sh/j4295icxcw4dqso/AACrH0-QCskr_oIXhSKvmnuJa?dl=0
>
> ffmpeg version:
> ----------------------
> ffmpeg version N-79160-gb73c271 Copyright (c) 2000-2016 the FFmpeg
> developers
> built with gcc 5.2.1 (Ubuntu 5.2.1-22ubuntu2) 20151010
> configuration: --prefix=/tmp/tmp_build_dir --pkg-config-flags=--static
> --extra-cflags=-I/tmp/tmp_build_dir/include
> --extra-ldflags=-L/tmp/tmp_build_dir/lib --bindir=/root/bin --enable-gpl
> --enable-nonfree --enable-libx264 --enable-libsoxr --enable-libfdk-aac
> libavutil      55. 19.100 / 55. 19.100
> libavcodec     57. 32.100 / 57. 32.100
> libavformat    57. 29.101 / 57. 29.101
> libavdevice    57.  0.101 / 57.  0.101
> libavfilter     6. 40.102 /  6. 40.102
> libswscale      4.  0.100 /  4.  0.100
> libswresample   2.  0.101 /  2.  0.101
> libpostproc    54.  0.100 / 54.  0.100
>
>
> Script (Basic version):
> -------------------------------
> #!/bin/bash
>
> rm dst/*
>
> for src in low mid high
> do
>     ffmpeg -i src/${src}.mp4 \
>         -map 0:v:0 \
>         -map 0:a:0 \
>         -codec copy \
>         -bsf:v h264_mp4toannexb \
>         -f segment \
>         -segment_format mpegts \
>         -reference_stream 0:v:0 \
>         -segment_list_type m3u8 \
>         -segment_list_size 0 \
>         -segment_time 9 \
>         -segment_time_delta 0.1 \
>         -segment_list dst/${src}.m3u8 \
>         dst/${src}_%d.ts
> done
>
>
> Thanks!
> Stefan
>


Hi guys,

After some more digging I believe I've found a couple of issues with this
approach:

1. We do have negative DTS on packets in the HiP video stream. It seems I
fell in the same trap as this guy (https://trac.ffmpeg.org/ticket/2375). I
was checking the DTS on the output from ffprobe -show_frames and not
-show_packets and assumed that the DTS in show_frames was copied from the
corresponding packet (as it is for PTS).
2. I am not 100% sure about this one, but after looking at the code for the
segment muxer, it seems it finishes a segment when it encounters a key
frame in the reference stream. Since the A/V packets are interleaved
differently with the HiP H264 compared to the CBP H264, the segments will
not contain the exact same audio frames for the HiP video compared to the
CBP video files.

So, I guess the conclusion is that I cannot use the segmentation muxer to
achieve what I want, unless I apply it on A/V streams separately.

Thanks
Stefan