[FFmpeg-user] a few seconds delay when screen recording with ffmpeg and GPU(Nvidia Tesla T4)

[kanata koyama]

hi,
I am trying to figure out how to start a chrome browser on a ubuntu 22.04
machine and take a screen recording with ffmpeg using the GPU,

1. the first few seconds of the video freezes.
2. after that, the video lags behind the audio by a few seconds.

I am having trouble with 2 in particular. Could you please tell me how to
fix it? Thanks!
here is more info👇

Dockerfile (only installation of ffmpeg)

...
RUN apt-get update \
&& apt-get install -y \
-o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" \
ffmpeg \
...


browser launch (puppeteer-core at 19.4.1)
import {launch} from 'puppeteer-core';

browser = await launch({
defaultViewport: null,
headless: false,
ignoreHTTPSErrors: true,
ignoreDefaultArgs: ['--mute-audio', '--enable-automation'],
handleSIGTERM: false,
handleSIGHUP: false,
handleSIGINT: false,
args: [
'--incognito',
'--no-sandbox',
'--disable-setuid-sandbox',
'--remote-debugging-port=9222',
'--remote-debugging-address=0.0.0.0',
'--start-fullscreen',
'--no-first-run',
'--no-default-browser-check',
'--disable-dev-shm-usage',
'--autoplay-policy=no-user-gesture-required',
'--use-fake-device-for-media-stream',
'--use-file-for-fake-audio-capture=./assets/nosound.wav',
'--use-file-for-fake-video-capture=./assets/no_video.y4m',
'--allow-file-access',
// Options that might enable usage of hardware GPU
// See:
https://ask.fedoraproject.org/t/how-to-make-google-chrome-use-nvidia-gpu/21027
'--disable-features=UseChromeOSDirectVideoDecoder',
'--disable-software-rasterizer',
'--enable-accelerated-2d-canvas',
'--enable-accelerated-video-decode',
'--enable-gpu-rasterization',
'--enable-zero-copy',
'--enable-raw-draw',
'--enable-native-gpu-memory-buffers',
'--enable-oop-rasterization',
'--canvas-oop-rasterization',
'--use-vulkan',
'--enable-features=VaapiVideoEncoder,VaapiVideoDecoder,CanvasOopRasterization,Vulkan'
,
'--ignore-gpu-blocklist',
'--use-gl=desktop',
],
userDataDir,
dumpio: true,
executablePath: '/usr/bin/google-chrome-stable',
});

a few seconds later, ffmpeg command is executed👇
const recordingProcess = spawn(
'ffmpeg',
[
'-loglevel',
'debug',
'-probesize',
'50M',
'-thread_queue_size',
'512',
// Record entire screen
'-f',
'x11grab',
'-framerate',
'25',
'-draw_mouse',
'0',
'-video_size',
'1024x768',
'-i',
':99.0',
'-hwaccel',
'cuda',
'-hwaccel_output_format',
'cuda',
// Record audio too
'-ac',
'1',
'-channel_layout',
'mono',
'-guess_layout_max',
'0',
'-f',
'pulse',
'-i',
'default',
'-ar',
'48000',
// output encoding options
'-vcodec',
'hevc_nvenc', // H.265
outputFile, // mp4
],
{
stdio: ['pipe', 'pipe', 'pipe'],
}
);

here is my experiment log👇

- thread_queue_size = 32 -> Video freezes entirely.
- thread_queue_size = 128 -> overall freeze is resolved, but the gap
between audio and video remained.
- thread_queue_size > 128 -> overall freeze is resolved, but the gap
between audio and video is not improved at all.
- vcodec = hevc_nvenc, h246_nvenc -> No improvement
- (b:v, maxrate, bufsize) = (1M, 1M, 0.5M), (2M, 2M, 1M) -> No improvement
- preset: no improvement, but may tend to
    - preset=fast -> audio is slower
    - preset=medium -> audio is faster
    - preset=slow -> audio is faster
- af =
    aresample=async=1:first_pts=0
    aresample=async=1000:min_comp=0.001:min_hard_comp=0.100:first_pts=0
    aresample=async=1000:min_comp=0.001:min_hard_comp=0.100
    aresample=async=2000:min_comp=0.001:min_hard_comp=0.100
    aresample=async=5000:min_comp=0.001:min_hard_comp=0.100
    -> No improvement.
- vsync = 0,1,2 -> no improvement.
- (async,vsync) = (1,1) -> no improvement
- crf = 18 -> no improvement
- (hwaccel,hwaccel_output_format) = (cuda,cuda), (not specified,not
specified) -> no improvement