[Libav-user] Capturing Accurate Timestamp of Each Frame when Recording Desktop

[Hassan]

Hello,

Hope all is well. Please bear with me on this. I hope this is the right
place for these questions.

I want to record the video of a live stream on a desktop using FFmpeg on
Windows 10 PC (Specs: Intel i7-4790 CPU @ 3.60GHz, RAM: 16GB, HDD 1TB 7200
RPM).
I also want to capture the timestamp when each frame is recorded by the
FFmpeg with the highest accuracy possible.
One top of the head solution is to use *-vf* filter in FFmpeg command to
append the timestamp on each frame while recording the desktop. A sample
command is shown below:

ffmpeg -f gdigrab -framerate 60 -i desktop -vf "settb=AVTB,
setpts='trunc(PTS/1K)*1K+st(1,trunc(RTCTIME/1K))-1K*trunc(ld(1)/1K)',
drawtext=fontfile=ArialBold.ttf:fontsize=40:fontcolor=white:text='%{localtime}.%{eif\:1M*t-1K*trunc(t*1K)\:d}:box=1:boxborderw=20:boxcolor=black at 1.0:x=10:y=10'"
-c:v libx264rgb -crf 0 -preset ultrafast output.mkv

The -vf text inspiration is taken from the post here
<https://stackoverflow.com/questions/47543426/ffmpeg-embed-current-time-in-milliseconds-into-video>
.

However, there are few caveats with the above approach:
1. When I am recording at full HD resolution, the timestamps appended are
not continuous on each frame. For instance, at 60 FPS, each
subsequent timestamp on each frame should be 16.6 msec apart. But, what I
am getting is that multiple frames have one static timestamp and then it
increments by several milliseconds on the next.
*It shows that there are accuracy issues with this approach and some
processing overhead. *

2. An alternate method I tried is to reduce resolution and only capture the
area of interest of the screen. So, instead of 1920x1080, I capture the
640x240 area of the screen by applying offset. As a result, at least I
started getting an incremental timestamp on each frame that was 14 to 18
msec apart.
*This approach shows that by reducing the processing, things can become
better. It also shows that appending timestamps on frames may have its own
processing overhead. Thus, they are still not accurate enough for me.*

3. The third caveat is that in the worst case, I am wondering, if a frame
arrives at the monitor at t1 and FFmpeg captures it at t2, the delta
between the two times t2-t1 could be 16.6 msec (in worse case). Again, I am
not sure if this is the case but intuitively it makes sense and I would
appreciate any insights on it.

So given the above issues, I am convinced that overlaying timestamp on the
frame while recording will induce error in the timestamp and it will
compromise accuracy. However, if someone has a solution to capture
timestamps using the command only then I am eagerly looking forward to
using it.

*PROPOSED APPROACH:*
I am wondering how can I modify the code of FFmpeg to capture the timestamp
along with the frame number as soon as FFmpeg captures the frame rendered
on the LCD and then send that timestamp~frameNo data in bulk to a file.
This way:
i) I will be able to mitigate the overhead of overlaying timestamps on the
frame,
ii) I will be able to record at a higher resolution, and
iii) most important, I will be able to achieve higher accuracy.

The third caveat is still not addressed and I am looking forward to
reducing that error by half by recording at 120 FPS while capturing
timestamp from code using the proposed approach.

I am looking for help with the following:
a) How feasible is this approach?
b) How can I achieve this, as I am new when it comes to diving into FFmpeg
code? A detailed approach would be great.
c) Are there any better alternate solutions to get highly accurate time
stamps for each frame?
d) Any insights on the third caveat would be great to have.

I really appreciate for reading to the end. I am looking forward to the
solutions.
Thank you.




-- 
Regards
Hassan Iqbal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20210326/63ece9d7/attachment.html>