[FFmpeg-user] Recorded Frame Timestamps are Inconsistent! How to Fix it?

[Marton Balint]

On Mon, 15 Mar 2021, Hassan wrote:

> Hello,
>
> I am using ffmpeg on a Windows 10 machine and I want to record the desktop
> at a high frame rate while appending accurate timestamps to each frame.
> I am recording my desktop using the following command:
>
> ffmpeg -f gdigrab -framerate 60 -i desktop -vf "settb=AVTB,
> setpts='trunc(PTS/1K)*1K+st(1,trunc(RTCTIME/1K))-1K*trunc(ld(1)/1K)',
> drawtext=fontfile=ArialBold.ttf:fontsize=40:fontcolor=white:text='%{localtime}.%{eif\:1M*t-1K*trunc(t*1K)\:d}:box=1:boxborderw=20:boxcolor=black at 1.0:x=10:y=10'"
> -c:v libx264rgb -crf 0 -preset ultrafast output.mkv
>
> The long text next to -vf flag is used to append timestamp (date and
> current time in milliseconds) on the top left corner of the frame with
> black background.
>
> The issue is that, ideally, when I am recording at 60 FPS, each subsequent
> frame should have a timestamp with an increment of 16.66 msec. However, the
> timestamp is not incremented as such. Instead, it stays the same on a lot
> of frames and then changes.
>
> For example, when I break the video into frames, the frame titled
> "img0428.png" has the timestamp 18:44:16.828 (hh:mm:ss.millisec)
> [image: image.png].
> Then until "next 40 frames, it says the same. On file "img0469.png", the
> timestamp changes and becomes 18:44:17.510.
> [image: image.png]
> So, the timestamp changed after 41 frames and the time difference is 682
> milliseconds. Ideally, each of the 40 frames between these two frames
> should carry an incremental timestamp by a step size of 16.66 msec but this
> is not happening.
>
> Therefore, my questions are as follows:
> 1. Am I using the right method to append timestamps to the recorded frames?

No. If you use a filter to set timestamps, that definitely will cause 
jitter in the output.

> 2. What is the reason that the timestamping on the frames is not correct?

Because you are setting time timestamps in a filter. If you want something 
like 10 millisecond accuracy, filter processing delays, or encoding 
latencies are not negligable at all.

> 3. How can I fix this issue?

Use the original timestamps which are returned by gdigrab. Also use
-copyts to pass the timestamps to the filters unmodified. Also use
-thread_queue_size parameter to enable threaded reading of input. And use 
'%{pts\:localtime}' variable expansion to get the date based on frame 
timestamp.

> 4. What are the alternate methods to append accurate epoch timestamps to
> each of the recorded frames?

As stated above. Note that it will still not be very accurate. In order to 
make it better I think a more accurate av_usleep() implementation is 
needed on Windows. With that, it should be decent enought.

Regards,
Marton