[FFmpeg-user] kmsgrab: Oh man, this could be so great. But it's just... not.

[Thomas Worth]

I've been experimenting with the kmsgrab device in ffmpeg. For recording
screens, and especially games, this could be huge for Linux nerds. And
ditching xcbgrab would be the icing on the cake. But, as it stands now,
it's meh. Here's why:

1.) There is a 2 year old bug that causes frames to be encoded out of
order: https://trac.ffmpeg.org/ticket/8377. I've tested it myself (yes,
with git head), on two separate AMD cards running on Debian with both the
radeon and amdgpu drivers, and this bug is definitely there when recording
alsa audio in the same process (the workaround is to capture streams
separately with two ffmpeg processes, but that's a sucky way to do it).
I've determined the issue has something to do with packet buffering, as
changing the thread_queue_size to 1 for video appears to eliminate the
problem, but then causes audio to be out of sync. So, not a solution,
really. The problem happens whether or not you use vaapi for encoding or
software encoding using libx264. So it happens well before the encoding
stage.

2.) As far as I can tell, kmsgrab (like x11grab or other screen devices)
wants a constant frame rate value (-r). That's not unusual, but the issue
is this won't work well with FreeSync rendering. More and more gamers are
using FreeSync (or Gsync) monitors that clock the refresh rate of the
monitor to the rendering calls. The correct way to capture this would be to
encode the output from kmsgrab using the system timer as the PTS or
whatever, so you end up with a variable frame rate mp4 that has each and
every frame discretely represented with the correct time value. Now,
whether or not you want to conform the entire thing later to 59.94p for
YouTube is another issue, but I think there needs to be an option (if there
isn't already one) to just write each frame as it is presented to the file
along with the system timestamp. This is exactly how camera input is
recorded on mobile devices, as they're all variable frame rate, and it
works fine. kmsgrab should do the same.

So those are my gripes for now. Hopefully this can get fixed and we can all
brag about our kill counts with dignity.