[FFmpeg-user] Hardware Accelerated Screen Recording (kmsgrab, VAAPI, hevc_vaapi) Washed Out Colors
Garrett Hopper
garrett at garretthopper.com
Tue Apr 28 04:10:17 EEST 2020
I'm trying to get hardware accelerated screen recording to work for
recording screencasts. I've gotten kmsgrab working to record my screen
and encode the result as an h265 MKV file, however the end result has
slightly different colors than actually appear on my screen, and I'm
unable to figure out the discrepancy.
I recently found this documentation:
https://trac.ffmpeg.org/wiki/Hardware/VAAPI#ScreenCapture which
mentioned the possibility of using kmsgrab and VAAPI to do the
recording and encoding all in the GPU while using very little CPU
power. Prior to that, I was using x11grab similar to the first two
commands from that link, and I had similar (but possibly slightly
different) color discrepency issues.
Here is my ffmpeg command:
ffmpeg \
-loglevel verbose \
-framerate 60 \
-f kmsgrab -i - \
-vf hwmap=derive_device=vaapi \
-codec:v hevc_vaapi \
-qp:v 10 \
out.mkv
And here is the output:
ffmpeg version 3.4.7 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 9.2.0 (GCC)
configuration: --disable-static
--prefix=/nix/store/z3hr6nlr5v78n2q6dh9jxj93lqwfvx4h-ffmpeg-3.4.7
--arch=x86_64 --target_os=linux --enable-gpl --enable-version3
--enable-shared --enable-pic --enable-runtime-cpudetect
--enable-hardcoded-tables --enable-pthreads --disable-w32threads
--disable-os2threads --enable-network --enable-pixelutils
--enable-ffmpeg --disable-ffplay --enable-ffprobe --disable-ffserver
--enable-avcodec --enable-avdevice --enable-avfilter --enable-avformat
--enable-avresample --enable-avutil --enable-postproc
--enable-swresample --enable-swscale --disable-doc --enable-bzlib
--enable-gnutls --enable-fontconfig --enable-libfreetype
--enable-libmp3lame --enable-iconv --enable-libtheora --enable-libssh
--enable-vaapi --enable-libdrm --enable-vdpau --enable-libvorbis
--enable-libvpx --enable-lzma --disable-opengl --enable-libpulse
--enable-sdl2 --enable-libsoxr --enable-libx264 --enable-libxvid
--enable-zlib --enable-libopus --enable-libspeex --enable-libx265
--disable-debug --enable-optimizations --disable-extra-warnings
--disable-stripping
libavutil 55. 78.100 / 55. 78.100
libavcodec 57.107.100 / 57.107.100
libavformat 57. 83.100 / 57. 83.100
libavdevice 57. 10.100 / 57. 10.100
libavfilter 6.107.100 / 6.107.100
libavresample 3. 7. 0 / 3. 7. 0
libswscale 4. 8.100 / 4. 8.100
libswresample 2. 9.100 / 2. 9.100
libpostproc 54. 7.100 / 54. 7.100
[AVHWDeviceContext @ 0xebe3c0] Opened DRM device /dev/dri/card0:
driver i915 version 1.6.0.
[kmsgrab @ 0xebd6a0] Using plane 31 to locate framebuffers.
[kmsgrab @ 0xebd6a0] Template framebuffer is 114: 2560x1440 32bpp 24b depth.
Input #0, kmsgrab, from 'pipe:':
Duration: N/A, start: 1588034012.025087, bitrate: N/A
Stream #0:0: Video: wrapped_avframe, 1 reference frame,
drm_prime, 2560x1440, 59.94 tbr, 1000k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (wrapped_avframe (native) -> hevc (hevc_vaapi))
[graph 0 input from stream 0:0 @ 0xed1b00] w:2560 h:1440
pixfmt:drm_prime tb:1/1000000 fr:60000/1001 sar:0/1 sws_param:flags=2
[AVHWDeviceContext @ 0xed45c0] libva: VA-API version 1.5.0
[AVHWDeviceContext @ 0xed45c0] libva: va_getDriverName() returns 0
[AVHWDeviceContext @ 0xed45c0] libva: Trying to open
/run/opengl-driver/lib/dri/i965_drv_video.so
[AVHWDeviceContext @ 0xed45c0] libva: Found init function __vaDriverInit_1_5
[AVHWDeviceContext @ 0xed45c0] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0xed45c0] Initialised VAAPI connection: version 1.5
[AVHWDeviceContext @ 0xed45c0] Matched "Intel i965 driver for
Intel(R) Kaby Lake - 2.4.0" as known driver "Intel i965 (Quick Sync)".
[hevc_vaapi @ 0xec3bc0] Input 2560x1440 -> Surface 2560x1440 -> CTU 80x45.
Output #0, matroska, to 'out.mkv':
Metadata:
encoder : Lavf57.83.100
Stream #0:0: Video: hevc (hevc_vaapi) (Main), 1 reference
frame, vaapi_vld, 2560x1440, q=2-31, 59.94 fps, 1k tbn, 59.94 tbc
Metadata:
encoder : Lavc57.107.100 hevc_vaapi
frame= 154 fps= 46 q=-0.0 Lsize= 1715kB time=00:00:03.60
bitrate=3898.2kbits/s speed=1.08x
video:1714kB audio:0kB subtitle:0kB other streams:0kB global
headers:0kB muxing overhead: 0.113014%
Input file #0 (pipe:):
Input stream #0:0 (video): 154 packets read (81312 bytes); 154
frames decoded;
Total: 154 packets (81312 bytes) demuxed
Output file #0 (out.mkv):
Output stream #0:0 (video): 154 frames encoded; 154 packets
muxed (1754650 bytes);
Total: 154 packets (1754650 bytes) muxed
Exiting normally, received signal 2.
The resulting video has these incorrect colors:
https://i.imgur.com/Jgzpjyt.png (attached as well)
The bottom is what was originally on my screen, and the top is what
was recorded.
These are what the colors were recorded as:
#F1FFE7 -> #FFFFF9
#59CD90 -> #4CCA92
#FAA613 -> #FFAE00
#3FA7D6 -> #2DA7E6
#A10702 -> #B40700
My understanding is that this is likely something to do with the pixel
format somewhere along the way, however I can't figure out where it
needs to be corrected.
I've attempted adding a `-format bgr0` (the default) before the
`-f kmsgrab` line and changing `bgr0` to a variety of things with very
little effect.
Using `bgra` resulted in no change, and using `rgb0`/`rgba` resulted
in completely different colors. I tried almost every other format from
`ffmpeg -pix_fmts`, and they all failed with some sort of error.
Using the same command with the addition of `-format bgr24` (as a
random example):
Routing option format to both codec and muxer layer
[AVHWDeviceContext @ 0x20725c0] Opened DRM device /dev/dri/card0:
driver i915 version 1.6.0.
[kmsgrab @ 0x2071820] Using plane 31 to locate framebuffers.
[kmsgrab @ 0x2071820] Template framebuffer is 114: 2560x1440 32bpp
24b depth.
Input #0, kmsgrab, from 'pipe:':
Duration: N/A, start: 1588034631.592462, bitrate: N/A
Stream #0:0: Video: wrapped_avframe, 1 reference frame,
drm_prime, 2560x1440, 59.94 tbr, 1000k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (wrapped_avframe (native) -> hevc (hevc_vaapi))
[graph 0 input from stream 0:0 @ 0x2085880] w:2560 h:1440
pixfmt:drm_prime tb:1/1000000 fr:60000/1001 sar:0/1 sws_param:flags=2
[AVHWDeviceContext @ 0x2088680] libva: VA-API version 1.5.0
[AVHWDeviceContext @ 0x2088680] libva: va_getDriverName() returns 0
[AVHWDeviceContext @ 0x2088680] libva: Trying to open
/run/opengl-driver/lib/dri/i965_drv_video.so
[AVHWDeviceContext @ 0x2088680] libva: Found init function
__vaDriverInit_1_5
[AVHWDeviceContext @ 0x2088680] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x2088680] Initialised VAAPI connection: version 1.5
[AVHWDeviceContext @ 0x2088680] Matched "Intel i965 driver for
Intel(R) Kaby Lake - 2.4.0" as known driver "Intel i965 (Quick Sync)".
[AVHWFramesContext @ 0x2095a80] DRM format not supported by VAAPI.
[Parsed_hwmap_0 @ 0x20855a0] Failed to map frame: -22.
Error while filtering: Invalid argument
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #0:0
Conversion failed!
Is this `kmsgrab` `-format` option likely to be what I'm looking for?
Is my framebuffer using some strange pixel format other than `bgr0`
for some reason?
The other thing I tried messing with was `scale_vaapi=format=$format`
in the filter chain. The main two being used were `nv12` and `p010`,
however I don't really know what they mean. Originally I was seeing
the format in the filter chain first and then using `hwupload` until I
found this which is faster (since everything stays in the GPU from
recording to encoding).
I also messed with options for `-color_range`, however all values for
it appeared to make things worse. (And I don't know exactly what it's
doing. :/)
Here's my `vainfo` output, if that's helpful:
libva info: VA-API version 1.5.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /run/opengl-driver/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_5
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.5 (libva 2.4.0)
vainfo: Driver version: Intel i965 driver for Intel(R) Kaby Lake - 2.4.0
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Simple : VAEntrypointEncSlice
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointEncSliceLP
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointEncSliceLP
VAProfileH264MultiviewHigh : VAEntrypointVLD
VAProfileH264MultiviewHigh : VAEntrypointEncSlice
VAProfileH264StereoHigh : VAEntrypointVLD
VAProfileH264StereoHigh : VAEntrypointEncSlice
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileNone : VAEntrypointVideoProc
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointEncPicture
VAProfileVP8Version0_3 : VAEntrypointVLD
VAProfileVP8Version0_3 : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice
VAProfileHEVCMain10 : VAEntrypointVLD
VAProfileHEVCMain10 : VAEntrypointEncSlice
VAProfileVP9Profile0 : VAEntrypointVLD
VAProfileVP9Profile0 : VAEntrypointEncSlice
VAProfileVP9Profile2 : VAEntrypointVLD
For what it's worth, I'm running on NixOS 20.03 with these settings
for OpenGL:
hardware.opengl = {
enable = true;
driSupport32Bit = true;
extraPackages = with pkgs; [
vaapiIntel # (2.4.0)
vaapiVdpau # (0.7.4)
libvdpau-va-gl # (0.4.2)
intel-media-driver # (19.3.0)
];
};
I'm not sure if these can be different, but here are the outputs of
`-h encoder=` commands for `hevc_vaapi` and `h264_vaapi`:
`ffmpeg -hide_banner -h encoder=hevc_vaapi`:
Encoder hevc_vaapi [H.265/HEVC (VAAPI)]:
General capabilities: delay
Threading capabilities: none
Supported pixel formats: vaapi_vld
h265_vaapi AVOptions:
-qp <int> E..V.... Constant QP (for
P-frames; scaled by qfactor/qoffset for I/B) (from 0 to 52) (default
25)
`ffmpeg -hide_banner -h encoder=h264_vaapi`:
Encoder h264_vaapi [H.264/AVC (VAAPI)]:
General capabilities: delay
Threading capabilities: none
Supported pixel formats: vaapi_vld
h264_vaapi AVOptions:
-qp <int> E..V.... Constant QP (for
P-frames; scaled by qfactor/qoffset for I/B) (from 0 to 52) (default
20)
-quality <int> E..V.... Set encode quality
(trades off against speed, higher is faster) (from 0 to 8) (default 0)
-low_power <int> E..V.... Use low-power encoding
mode (experimental: only supported on some platforms, does not support
all features) (from 0 to 1) (default 0)
-coder <int> E..V.... Entropy coder type
(from 0 to 1) (default cabac)
cavlc E..V....
cabac E..V....
vlc E..V....
ac E..V....
I briefly tried with QSV instead of VAAPI, however I was having
similar issues. Is there any benefit to using QSV over VAAPI?
More information about the ffmpeg-user
mailing list