[FFmpeg-user] overlay_qsv with hardware decode and encode

[Vladimir Mishonov]

Greetings everyone.

I am attempting to overlay 2 video streams in real-time with QuickSync 
hardware acceleration. I want to use both hardware decode and encode to 
make the most use out of the GPU and avoid excessive CPU usage.

I've found an example command-line here: 
https://github.com/intel/vaapi-fits/issues/173 and adapted it:

./ffmpeg \
  -hwaccel qsv -c:v h264_qsv -i rtmp://localhost/stream1 \
  -hwaccel qsv -c:v h264_qsv -i rtmp://localhost/stream2 \
  -filter_complex "[0:v][1:v]overlay_qsv=w=500" \
  -c:v h264_qsv -global_quality 25 -g 50 -r 25 -an test.h264

However, this gives me the following error:

ffmpeg version N-118510-gbc1a3bfd2c-20250220 Copyright (c) 2000-2025 the 
FFmpeg developers
   built with gcc 14.2.0 (crosstool-NG 1.26.0.120_4d36f27)
   configuration: --prefix=/ffbuild/prefix --pkg-config-flags=--static 
--pkg-config=pkg-config --cross-prefix=x86_64-ffbuild-linux-gnu- 
--arch=x86_64 --target-os=linux --enable-gpl --enable-version3 
--disable-debug --enable-iconv --enable-zlib --enable-libfreetype 
--enable-libfribidi --enable-gmp --enable-libxml2 --enable-openssl 
--enable-lzma --enable-fontconfig --enable-libharfbuzz 
--enable-libvorbis --enable-opencl --enable-libpulse --enable-libvmaf 
--enable-libxcb --enable-xlib --enable-amf --enable-libaom 
--enable-libaribb24 --enable-avisynth --enable-chromaprint 
--enable-libdav1d --enable-libdavs2 --enable-libdvdread 
--enable-libdvdnav --disable-libfdk-aac --enable-ffnvcodec 
--enable-cuda-llvm --enable-frei0r --enable-libgme --enable-libkvazaar 
--enable-libaribcaption --enable-libass --enable-libbluray 
--enable-libjxl --enable-libmp3lame --enable-libopus --enable-librist 
--enable-libssh --enable-libtheora --enable-libvpx --enable-libwebp 
--enable-libzmq --enable-lv2 --enable-libvpl --enable-openal 
--enable-libopencore-amrnb --enable-libopencore-amrwb 
--enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt 
--enable-librav1e --enable-librubberband --disable-schannel 
--enable-sdl2 --enable-libsnappy --enable-libsoxr --enable-libsrt 
--enable-libsvtav1 --enable-libtwolame --enable-libuavs3d 
--enable-libdrm --enable-vaapi --enable-libvidstab --enable-vulkan 
--enable-libshaderc --enable-libplacebo --disable-libvvenc 
--enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid 
--enable-libzimg --enable-libzvbi --extra-cflags=-DLIBTWOLAME_STATIC 
--extra-cxxflags= --extra-libs='-ldl -lgomp' --extra-ldflags=-pthread 
--extra-ldexeflags=-pie --cc=x86_64-ffbuild-linux-gnu-gcc 
--cxx=x86_64-ffbuild-linux-gnu-g++ --ar=x86_64-ffbuild-linux-gnu-gcc-ar 
--ranlib=x86_64-ffbuild-linux-gnu-gcc-ranlib 
--nm=x86_64-ffbuild-linux-gnu-gcc-nm --extra-version=20250220
   libavutil      59. 57.100 / 59. 57.100
   libavcodec     61. 33.102 / 61. 33.102
   libavformat    61.  9.107 / 61.  9.107
   libavdevice    61.  4.100 / 61.  4.100
   libavfilter    10.  9.100 / 10.  9.100
   libswscale      8. 13.100 /  8. 13.100
   libswresample   5.  4.100 /  5.  4.100
   libpostproc    58.  4.100 / 58.  4.100
[vist#0:1/h264 @ 0x562c70e48780] WARNING: defaulting 
hwaccel_output_format to qsv for compatibility with old commandlines. 
This behaviour is DEPRECATED and will be removed in the future. Please 
explicitly set "-hwaccel_output_format qsv".
Input #0, flv, from 'rtmp://localhost/stream1':
   Metadata:
     |RtmpSampleAccess: true
     Server          : NGINX RTMP (github.com/arut/nginx-rtmp-module)
     displayWidth    : 2560
     displayHeight   : 1440
     fps             : 25
     profile         :
     level           :
   Duration: 00:00:00.00, start: 138266.736000, bitrate: N/A
   Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
   Stream #0:1: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 
2560x1440, 25 fps, 25 tbr, 1k tbn
[vist#1:1/h264 @ 0x562c70ef3b00] WARNING: defaulting 
hwaccel_output_format to qsv for compatibility with old commandlines. 
This behaviour is DEPRECATED and will be removed in the future. Please 
explicitly set "-hwaccel_output_format qsv".
Input #1, flv, from 'rtmp://localhost/stream2':
   Metadata:
     |RtmpSampleAccess: true
     Server          : NGINX RTMP (github.com/arut/nginx-rtmp-module)
     displayWidth    : 2688
     displayHeight   : 1520
     fps             : 26
     profile         :
     level           :
   Duration: 00:00:00.00, start: 138268.361000, bitrate: N/A
   Stream #1:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
   Stream #1:1: Video: h264 (High), yuvj420p(pc, progressive), 2688x1520 
[SAR 1:1 DAR 168:95], 26 fps, 26 tbr, 1k tbn
Stream mapping:
   Stream #0:1 (h264_qsv) -> overlay_qsv
   Stream #1:1 (h264_qsv) -> overlay_qsv
   overlay_qsv:default -> Stream #0:0 (h264_qsv)
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'graph 
0 input from stream 1:1' and the filter 'auto_scale_0'
[fc#0 @ 0x562c70e30a40] Error reinitializing filters!
[fc#0 @ 0x562c70e30a40] Task finished with error code: -38 (Function not 
implemented)
[fc#0 @ 0x562c70e30a40] Terminating thread with return code -38 
(Function not implemented)
[vost#0:0/h264_qsv @ 0x562c70ede940] [enc:h264_qsv @ 0x562c70edec40] 
Could not open encoder before EOF
[vost#0:0/h264_qsv @ 0x562c70ede940] Task finished with error code: -22 
(Invalid argument)
[vost#0:0/h264_qsv @ 0x562c70ede940] Terminating thread with return code 
-22 (Invalid argument)
[out#0/h264 @ 0x562c70e31240] Nothing was written into output file, 
because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0KiB time=N/A bitrate=N/A 
speed=N/A
Conversion failed!


If I remove the filter_complex and use just one input stream, it works 
without issue:

./ffmpeg \
  -hwaccel qsv -c:v h264_qsv -i rtmp://localhost/stream1 \
  -c:v h264_qsv -global_quality 25 -g 50 -r 25 -an test.h264

Could anyone please tell me what I'm doing wrong?

Thank you very much.

-- 
Kind regards,
Vladimir