[FFmpeg-user] how to set the color background when i use overlay_qsv

[Xiang, Haihao]

On Thu, 2022-01-27 at 11:04 +0800, zhaofg at bstar.com.cn wrote:
> hello:
>     I want to splice four or more video into one video，i want to set the
> background to a fixed size and the color is black, so i must scale the image
> then overlay. 
>     
>     My environment is ubuntu and I used command is:
> 
>     ffmpeg -i h1.mp4 -i h2.mp4 -i h3.mp4 -i h4.mp4 -filter_complex
> "color=c=black:s=1920x1080[bg];[0:v]scale=iw/2:-1[img0];[1:v]scale=iw/2:-
> 1[img1];[2:v] scale=iw/2:-1[img2];[3:v]scale=iw/2:-
> 1[img3];[bg][img0]overlay=shortest=1[out0];[out0][img1]overlay=shortest=1:x=w[
> out1];[out1][img2]overlay=shortest=1:y=h[out2];[out2][img3]overlay=shortest=1:
> x=w:y=h" -y out.mp4
> 
> ------------------------------------------------------------------------------
> ------------------------------------------------------------------------------
> -----
> the comman is correct,but when i change it to qsv. It's not working correctly:
> 
>     ./ffmpeg -hwaccel qsv -c:v h264_qsv -i h1.mp4 -hwaccel qsv -c:v h264_qsv
> -i h2.mp4 -hwaccel qsv -c:v h264_qsv -i h3.mp4 -hwaccel qsv -c:v h264_qsv -i
> h4.mp4 -filter_complex "color=black:r=30:size=1920x1080:sar=1/1,
> hwupload=extra_hw_frames=64,format=qsv[bg];[0:v]scale_qsv=iw/2:ih/2[img0];[1:v
> ]scale_qsv=iw/2:ih/2[img1];[2:v]
> scale_qsv=iw/2:ih/2[img2];[3:v]scale_qsv=iw/2:ih/2[img3];[bg][img0]overlay_qsv
> =shortest=1[out0];[out0][img1]overlay_qsv=shortest=1:x=w[out1];[out1][img2]ove
> rlay_qsv=shortest=1:y=h[out2];[out2][img3]overlay_qsv=shortest=1:x=w:y=h" -y
> out.mp4
> 
>  output:
> [hwupload @ 0x5593003e07c0] A hardware device reference is required to upload
> frames to.
> [Parsed_hwupload_1 @ 0x5593003e0680] Query format failed for
> 'Parsed_hwupload_1': Invalid argument
> Error reinitializing filters!
> Failed to inject frame into filter network: Invalid argument
> Error while processing the decoded data for stream #3:0
> Conversion failed!
> ------------------------------------------------------------------------------
> ------------------------------------------------------------------------------
> ----
> ./ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -hwaccel qsv -c:v
> h264_qsv -i h1.mp4 -hwaccel qsv -c:v h264_qsv -i h2.mp4 -hwaccel qsv -c:v
> h264_qsv -i h3.mp4 -hwaccel qsv -c:v h264_qsv -i h4.mp4 -filter_complex
> "color=black:r=30:size=1920x1080:sar=1/1,
> hwupload=extra_hw_frames=64,format=qsv[bg];[0:v]scale_qsv=iw/2:ih/2[img0];[1:v
> ]scale_qsv=iw/2:ih/2[img1];[2:v]
> scale_qsv=iw/2:ih/2[img2];[3:v]scale_qsv=iw/2:ih/2[img3];[bg][img0]overlay_qsv
> =shortest=1[out0];[out0][img1]overlay_qsv=shortest=1:x=w[out1];[out1][img2]ove
> rlay_qsv=shortest=1:y=h[out2];[out2][img3]overlay_qsv=shortest=1:x=w:y=h" -y
> out.mp4
> 
> 
> output:
> [Parsed_overlay_qsv_7 @ 0x563881a99180] Inputs with different underlying QSV
> devices are forbidden.
> [Parsed_overlay_qsv_7 @ 0x563881a99180] Failed to configure output pad on
> Parsed_overlay_qsv_7
> Error reinitializing filters!
> Failed to inject frame into filter network: Invalid argument
> Error while processing the decoded data for stream #3:0
> Conversion failed!
> ------------------------------------------------------------------------------
> ------------------------------------------------------------------------------
> ----
> I want to know How can I modify my command to run correctly. Thank you.

What is the version of your FFmpeg? Your command work for me with the latest
FFmpeg (https://git.ffmpeg.org/gitweb/ffmpeg.git/shortlog/refs/heads/master).
There is a green line at the bottom of each scaled image with your command, you
may try vpp_qsv instead of scale_qsv. We will fix the green line issue in
scale_qsv

Thanks
Haihao