[FFmpeg-user] FFMPEG - AMD GPU Transcoding MJPEG to HEVC provides a skewed split screen video

Thu Jul 28 16:55:19 EEST 2022

Hello,
I have six CCTV 8MP camera which outputs CVI analog video signal.
Each camera is connected to it's own individual standalone special converter box which converts the analog signal to common HDMI output. Each of the converter boxes is then connected to it's own individual standalone USB 3.0 HDMI capture card.
So now I literally have six USB capture cards (all of them of the same exact brand and model) connected to my PC.The real time video stream plays perfectly at 60FPS and each of the output stream format is MJPG.
I would like to do AMD GPU hardware transcoding to HEVC of the six MJPG streams, all simultaneously. 

I checked some FFMPEG commands to do hardware transcoding from this link:
https://trac.ffmpeg.org/wiki/Hardware/VAAPI

I settled using this transcoding command since it is the only command which works with no crash errors:
ffmpeg -report -f v4l2 -input_format mjpeg -framerate 30 -video_size 1920x1080 -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i /dev/video0 -vf 'scale_vaapi=format=nv12,hwupload' -c:v hevc_vaapi output.mp4 -y

Primary issue I am facing:

When playback of the output video file, half of the video on the screen is the actual video and the other half is all noise data. Why is this happening and how can I resolve this issue?

Picture of the video (Please note that the video is in greyscale due to the fact that the camera is a CCTV camera and the infrared lights are switched on due to the low lighting condition):
https://i.stack.imgur.com/T86kG.png

Here is a YouTube video of the output file:
https://youtu.be/XCC4JDffTxM

Here is the report.log file if needed:

https://drive.google.com/file/d/1S7aV9cHhGUv1H0uebnt109n5V3rbU-94/view

I would also like to mention that when using the following command below (which is only encoding from RAW stream):
ffmpeg -vaapi_device /dev/dri/renderD128 -f v4l2 -framerate 30 -video_size 1920x1080 -i /dev/video0 -vf 'format=nv12,hwupload' -c:v hevc_vaapi output.mp4 -y
The video playback works properly, however it only works great when set to 30FPS and uses about 20% of CPU resource. When I set the command to 60 FPS, it uses about 2-3% of CPU resource but the video file becomes corrupted and freezes after 3-5 secs when playback.

Details of my PC:
OS: Arch Linux x86_64Host: MS-7693 3.0Kernel: 5.18.12-arch1-1Uptime: 3 hours, 37 minsPackages: 1076 (pacman)Shell: bash 5.1.16Resolution: 1920x1080Terminal: /dev/pts/0CPU: AMD FX-8350 (8) @ 4.000GHzGPU: NVIDIA GeForce GT 710GPU: AMD ATI Radeon PRO WX 2100Memory: 561MiB / 23987MiB

Details about the AMD GPU driver:
$ lspci -k | grep -A 2 -E "(VGA|3D)"
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Lexa XT [Radeon PRO WX 2100]        Subsystem: Dell Device 0b0c        Kernel driver in use: amdgpu

Details about FFMPEG:
$ ffmpeg
ffmpeg version n5.0.1 Copyright (c) 2000-2022 the FFmpeg developers  built with gcc 12.1.0 (GCC)  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3  libavutil      57. 17.100 / 57. 17.100  libavcodec     59. 18.100 / 59. 18.100  libavformat    59. 16.100 / 59. 16.100  libavdevice    59.  4.100 / 59.  4.100  libavfilter     8. 24.100 /  8. 24.100  libswscale      6.  4.100 /  6.  4.100  libswresample   4.  3.100 /  4.  3.100  libpostproc    56.  3.100 / 56.  3.100Hyper fast Audio and Video encoder

Here are the details of the video stream and the USB 3.0 capture card device with the help of V4L2:
$ v4l2-ctl --all --device /dev/video0:
Driver Info:        Driver name      : uvcvideo        Card type        : USB3.0 HD Video Capture: USB3.0        Bus info         : usb-0000:04:00.0-2        Driver version   : 5.18.12        Capabilities     : 0x84a00001                Video Capture                Metadata Capture                Streaming                Extended Pix Format                Device Capabilities        Device Caps      : 0x04200001                Video Capture                Streaming                Extended Pix FormatMedia Driver Info:        Driver name      : uvcvideo        Model            : USB3.0 HD Video Capture: USB3.0        Serial           : HU123450        Bus info         : usb-0000:04:00.0-2        Media version    : 5.18.12        Hardware revision: 0x00000328 (808)        Driver version   : 5.18.12Interface Info:        ID               : 0x03000002        Type             : V4L VideoEntity Info:        ID               : 0x00000001 (1)        Name             : USB3.0 HD Video Capture: USB3.0        Function         : V4L2 I/O        Flags            : default        Pad 0x01000007   : 0: Sink          Link 0x02000010: from remote pad 0x100000a of entity 'Processing 3' (Video Pixel Formatter): Data, Enabled, ImmutablePriority: 2Video input : 0 (Camera 1: ok)Format Video Capture:        Width/Height      : 1920/1080        Pixel Format      : 'MJPG' (Motion-JPEG)        Field             : None        Bytes per Line    : 0        Size Image        : 2073600  [Megapixels: 2.1 MP (2,073,600 pixels)]        Colorspace        : sRGB        Transfer Function : Rec. 709        YCbCr/HSV Encoding: ITU-R 601        Quantization      : Default (maps to Full Range)        Flags             :Crop Capability Video Capture:        Bounds      : Left 0, Top 0, Width 1920, Height 1080        Default     : Left 0, Top 0, Width 1920, Height 1080        Pixel Aspect: 1/1Selection Video Capture: crop_default, Left 0, Top 0, Width 1920, Height 1080, Flags:Selection Video Capture: crop_bounds, Left 0, Top 0, Width 1920, Height 1080, Flags:Streaming Parameters Video Capture:        Capabilities     : timeperframe        Frames per second: 60.000 (60/1)        Read buffers     : 0

More details about the USB 3.0 capture card device stream capabilities:
$ v4l2-ctl --list-formats-ext --device /dev/video0
ioctl: VIDIOC_ENUM_FMT        Type: Video Capture
        [0]: 'MJPG' (Motion-JPEG, compressed)                Size: Discrete 1920x1080                        Interval: Discrete 0.017s (60.000 fps)                        Interval: Discrete 0.033s (30.000 fps)                Size: Discrete 1280x720                        Interval: Discrete 0.017s (60.000 fps)                        Interval: Discrete 0.033s (30.000 fps)                Size: Discrete 960x540                        Interval: Discrete 0.017s (60.000 fps)                        Interval: Discrete 0.033s (30.000 fps)                Size: Discrete 800x450                        Interval: Discrete 0.017s (60.000 fps)                        Interval: Discrete 0.033s (30.000 fps)                Size: Discrete 640x360                        Interval: Discrete 0.017s (60.000 fps)                        Interval: Discrete 0.033s (30.000 fps)

Thanks for any advice and help.
-Rajh