[FFmpeg-user] Generating HLS chunks on demand

[Vincent Deconinck]

Hi,

Here is my progress report.

I wanted to understand why files (even the ones created with the hls muxer)
cannot be played smoothly one after the other if they were generated by
different runs.
So I executed 13 HLS encodings (A-M) of different parts of the first 36
seconds of my sample file.
I then logged the sizes of the different segments, and here are the results:

Timespan  Run A  Run B  Run C  Run D  Run E  Run F  Run G  Run H  Run I
 Run J  Run K  Run L  Run M
========  ====== ====== ====== ====== ====== ====== ====== ====== ======
====== ====== ====== ======
00s-04s   567008 567196 567196 567196 567196 ------ ------ ------ ------
------ ------ ------ ------
00s-08s   ------ 631304 634688 634688 634688 630928 ------ ------ ------
------ ------ ------ ------
00s-12s   ------ ------ 512488 511172 511172 507224 603668 ------ ------
------ ------ ------ ------
00s-16s   ------ ------ ------ 637320 639012 651984 659504 664016 ------
------ ------ ------ ------
00s-20s   ------ ------ ------ ------ 461728 466240 461916 462292 634312
------ ------ ------ ------
04s-24s   ------ ------ ------ ------ ------ 599720 608932 626228 592012
592012 ------ ------ ------
08s-28s   ------ ------ ------ ------ ------ ------ 904092 892248 857280
817800 646344 ------ ------
12s-32s   ------ ------ ------ ------ ------ ------ ------ 572648 540688
508540 506660 733952 ------
16s-36s   ------ ------ ------ ------ ------ ------ ------ ------ 617016
600472 600472 564564 463608
(I hope fixed-width style passes through, otherwise please copy/paste to a
fixed-width editor)

1. Comparing the runs A to E:
One can see the first segment of runs B-E is the same, which makes sense as
it is a single pass encoding and files are generated in sequence.
Same for the second segment of runs C-E and the third segment of runs D-E.
But the last segment of each run (first of A, second of B, third of C,
etc.) is different from other runs for the same timeframe..

2. Comparing all runs:
When starting the encoding at different positions, the generated segments
are different. I guess an encoder "warms up" and learns about the video
from the previously encountered frames, and can optimize its parameters for
the subsequent ones, leading to different files.

So I came up with this idea:
If the last segment of each run is "special", let's make a playlist mixing
segments from different runs, but avoiding the last one.
So I created a playlist containing the first segment created by runs E to M
(so E0-F0-G0-H0-I0-J0-K0-L0-M0).
Here again, VLC aborts after the first segment. The playlist cannot be
played :-(

But here is a very strange thing :

When creating that playlist, I first made a mistake and pointed to segments
"E0-F1-G2-H3-I4-J5-K6-L7-M8" (forgetting that each runs names the first
segment "0" no matter its start time), and THAT PLAYLIST COULD BE READ !
Of course Iit skipped 4 seconds between segments and stopped after 20
seconds (files J5 to M8 do not exist), but VLC happily played them in a
seamless manner for the first time.

I first thought the reason could be an increasing "segment number" that
would be embedded in the TS files but then it struck me:

**TIMECODE** !

I'm pretty sure the HLS muxer embeds a continuous timecode inside the
generated segments, so the first encoded segment is from 00:00:00:00 to
00:00:03:24, the second one is from 00:00:04:00 to 00:00:07:24 and so on.
That would explain why independently generated segments cannot be played in
sequence: they are all declared with the same TC interval: 00:00:00:00 to
00:00:03:24 over and over again.

I quickly wrapped a test and patched the timecodes like so (blatant
copy/paste from a StackExhchange command):
ffmpeg -i hls_E0.ts -c copy -output_ts_offset 0  -muxdelay 0 -muxpreload 0
hls_TcE0.ts
ffmpeg -i hls_F0.ts -c copy -output_ts_offset 4  -muxdelay 0 -muxpreload 0
hls_TcF0.ts
ffmpeg -i hls_G0.ts -c copy -output_ts_offset 8  -muxdelay 0 -muxpreload 0
hls_TcG0.ts
ffmpeg -i hls_H0.ts -c copy -output_ts_offset 12 -muxdelay 0 -muxpreload 0
hls_TcH0.ts
ffmpeg -i hls_I0.ts -c copy -output_ts_offset 16 -muxdelay 0 -muxpreload 0
hls_TcI0.ts
ffmpeg -i hls_J0.ts -c copy -output_ts_offset 20 -muxdelay 0 -muxpreload 0
hls_TcJ0.ts
ffmpeg -i hls_K0.ts -c copy -output_ts_offset 24 -muxdelay 0 -muxpreload 0
hls_TcK0.ts
ffmpeg -i hls_L0.ts -c copy -output_ts_offset 28 -muxdelay 0 -muxpreload 0
hls_TcL0.ts
ffmpeg -i hls_M0.ts -c copy -output_ts_offset 32 -muxdelay 0 -muxpreload 0
hls_TcM0.ts

Then I edited my playlist to point to the patched files, and finally, I
have something that VLC accepts to play back from start to end. Yeah :-)

Now, let's not celebrate too fast. While the audio is almost perfect, the
video freezes at some 4-sec transitions.

I tried applying the PTS offset at HLS creation time using e.g.:
ffmpeg -ss 4 -i source.ts -vf "setpts=PTS+4/TB" -c:v libx264 -b:v 1000k
-c:a aac -b:a 128k -x264opts "keyint=25:min-keyint=25:no-scenecut"
-timecode 00:00:04:00 -t 20 -f hls -hls_time 4 -hls_playlist_type vod
hls_TcF.m3u8
but it seems the setpts and timecode options are ignored, probably because
the hls muxer overrides them.
I then tried using setpts in the patch phase, like:
ffmpeg -i hls_F0.ts -vf "setpts=PTS+4/TB" -c copy hls_TcF0.ts
But ffmpeg throws an error (Filtergraph 'setpts=PTS+4/TB' was defined for
video output stream 0:0 but codec copy was selected. Filtering and
streamcopy cannot be used together.)


=> Q: Any idea how to get better control of the PTS in the generated
segments ?


Thanks for having read this long message till the end.

Kind regards,

Vincent