[FFmpeg-user] EXT-X-PROGRAM-DATE-TIME value drifts behind the actual time on the start of the corresponding segment video

[Vignesh Ravichandran]

I am trying to create a video by stitching multiple HLS segments  (.ts
files)  together based on a start time and end time.  Basically, create a
video of a particular timeline in the HLS stream.

My setup has a camera pointing to a monitor that displays the system clock
(The camera control and the ffmpeg command run, is on the same system).
The frame rate of the camera is not constant but is capped at 25 FPS. The
frames captured from the camera are passed on to the following ffmpeg
command:
```
/usr/bin/ffmpeg -vaapi_device /dev/dri/renderD128 -y -v 9 -loglevel 99 -f
rawvideo -vcodec rawvideo -s 1920x1200 -pix_fmt uyvy422 -i - -an -r 25
-vcodec h264_vaapi -vf format=nv12,hwupload -force_key_frames
expr:gte(t,n_forced*1) -f hls -hls_list_size 10080 -hls_time 10 -hls_flags
delete_segments -hls_flags second_level_segment_index -hls_flags
+append_list+discont_start+program_date_time -strftime 1 -strftime_mkdir 1
-hls_segment_filename %Y/%m/%d/%%04d.ts playlist.m3u8
```

I parse the playlist file created and then pick up a set of .ts files that
lie between the given start and end time. This is done by comparing the
“#EXT-X-PROGRAM-DATE-TIME” value of each file in the playlist. Then, I
stitch these files together and trim to produce the final output video file.

Problem Encountered:
I noticed that the “#EXT-X-PROGRAM-DATE-TIME” value in the playlist file
for most segments and the system time shown in the start (00:00:00) of the
corresponding segment video is different. The “#EXT-X-PROGRAM-DATE-TIME” is
behind. For example, for below entry:
```
#EXTINF:10.000000,
#EXT-X-PROGRAM-DATE-TIME:2021-01-21T23:17:05.000+1100
2021/01/21/0085.ts
```
the value shown on the system clock in the video is
"2021-01-21T23:18:12.000+1100" which is more than a minute ahead.
This causes the final video created to contain incorrect timeline from the
stream.

My observation and conclusion:
Since the FPS of the camera is not constant and can vary between 0 to 25,
ffmpeg would have to duplicate frames when the input FPS drops below 25.
But I don’t see any mention of this happening in the logs. So I think
ffmpeg is filling in the current segment with frames outside the segment
duration i.e., with frames which are supposed to be in the next segment.
And the experiment mentioned above verifies it.

Looking at the code, the “do_video_out” function in “ffmpeg.c” file has
logic to duplicate frames if needed. But not sure why this is not
happening. Would greatly appreciate some help in debugging this issue. I
have also attached the log file from my test here.

Thanks,
Vignesh

 ffmpeg.log
<https://drive.google.com/file/d/1FDx2LzVNksl2w-6fz9ShC7vvbrhEXsQv/view?usp=drive_web>