[FFmpeg-user] HW Acceleration 101? 2-Up Streaming from RTSP-->ffmpeg-->YouTube

[Steven Kan]

> On Jan 18, 2021, at 10:58 AM, Michael Koch <astroelectronic at t-online.de> wrote:
> 
> Am 18.01.2021 um 19:18 schrieb Steven Kan:
>> But now I want to do a “2-Up” live stream of two different cameras, side-by-side. Here’s an archive from last night (waiting for a mating pair of Barn Owls to move in):
>> 
>> https://www.youtube.com/watch?v=GDN2MjPwn0Q&feature=youtu.be <https://www.youtube.com/watch?v=GDN2MjPwn0Q&feature=youtu.be>
>> 
>> The cameras are each outputting 1920 x 1080 @ 25 fps.
>> 
>> Now that I’m actually encoding, I need a lot more CPU/GPU. I’m running this in Win10 Pro/64 on an HP Microserver with an AMD Opteron X3418 Quad-Core, and the CPU runs at about ~65-80% while the integrated GPU runs at about ~55%.
>> 
>> C:\Program Files\ffmpeg\bin> .\ffmpeg.exe -re -thread_queue_size 1024 -i rtsp://anonymous:password@192.168.1.47:554 <rtsp://anonymous:password@192.168.1.47:554> -i rtsp://anonymous:password@192.168.1.50:554 <rtsp://anonymous:password@192.168.1.50:554> -vcodec h264_amf -acodec copy -t 01:47:02 -filter_complex "nullsrc=size=3840x1080 [base]; [0:v] setpts=PTS-STARTPTS, scale=1920x1080 [upperleft]; [1:v] setpts=PTS-STARTPTS, scale=1920x1080 [upperright]; [base][upperleft] overlay=shortest=1 [tmp1]; [tmp1][upperright] overlay=shortest=1:x=1920" -f flv "rtmp://a.rtmp.youtube.com/live2/my-youtube-streaming-key <rtmp://a.rtmp.youtube.com/live2/my-youtube-streaming-key>”
> 
> Wouldn't it be easier to use something like
> [upperleft][upperright] hstack

It might be. I didn’t know hstack existed :D. When I googled ‘ffmpeg 2-up” the nullsrc/overlay examples were the first ones I found :D 

I tried hstack, and I’m not getting at all the results I expect. On my Windows machine I get "Conversion failed!” error, whereas the same command with '-filter_complex “nullsrc . . . .’ does not fail:

C:\Program Files\ffmpeg\bin> .\ffmpeg.exe -re -thread_queue_size 1024 -i rtsp://anonymous:password@192.168.1.47:554 -i rtsp://anonymous:password@192.168.1.50:554 -vcodec h264_amf -acodec copy -t 01:47:02 -filter_complex hstack=inputs=2 -f flv out.flv                   

ffmpeg version 2020-12-09-git-7777e5119a-essentials_build-www.gyan.dev Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.2.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libgme --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-librubberband
  libavutil      56. 62.100 / 56. 62.100
  libavcodec     58.115.102 / 58.115.102
  libavformat    58. 65.100 / 58. 65.100
  libavdevice    58. 11.103 / 58. 11.103
  libavfilter     7. 92.100 /  7. 92.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
Input #0, rtsp, from 'rtsp://anonymous:password@192.168.1.47:554':
  Metadata:
    title           : Media Server
  Duration: N/A, start: 0.080000, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 1920x1080, 25 fps, 25 tbr, 90k tbn, 180k tbc
Input #1, rtsp, from 'rtsp://anonymous:password@192.168.1.50:554':
  Metadata:
    title           : Media Server
  Duration: N/A, start: 0.100000, bitrate: N/A
    Stream #1:0: Video: h264 (Main), yuv420p(progressive), 1920x1080, 100 tbr, 90k tbn, 180k tbc
    Stream #1:1: Audio: aac (LC), 8000 Hz, mono, fltp
Stream mapping:
  Stream #0:0 (h264) -> hstack:input0
  Stream #1:0 (h264) -> hstack:input1
  hstack -> Stream #0:0 (h264_amf)
  Stream #1:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
[rtsp @ 000002a54cba0fc0] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
[rtsp @ 000002a54cb2dd80] max delay reached. need to consume packet
[rtsp @ 000002a54cb2dd80] RTP: missed 135 packets
[swscaler @ 000002a550941000] deprecated pixel format used, make sure you did set range correctly
[swscaler @ 000002a54ef7d380] deprecated pixel format used, make sure you did set range correctly
[h264_amf @ 000002a54d76dc80] encoder->Init() failed with error 36
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!