[FFmpeg-user] A question on H.264 High 4:4:4 Predictive decode support on NVIDIA Pascal hardware via nvdec

[Dennis Mungai]

Hello there,

I have a sample file with the following attributes:

ffprobe -i rv.mkv
ffprobe version N-93814-g48539b62fc Copyright (c) 2007-2019 the FFmpeg
developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04)
  configuration: --pkg-config-flags=--static --prefix=/home/brainiarc7/bin
--bindir=/home/brainiarc7/bin --extra-cflags=-I/home/brainiarc7/bin/include
--extra-ldflags=-L/home/brainiarc7/bin/lib --enable-cuda-nvcc
--enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include/
--extra-ldflags=-L/usr/local/cuda/lib64/ --enable-nvenc
--extra-cflags=-I/opt/intel/mediasdk/include
--extra-ldflags=-L/opt/intel/mediasdk/lib
--extra-ldflags=-L/opt/intel/mediasdk/plugins --enable-libmfx
--enable-libass --enable-vaapi --disable-debug --enable-libvorbis
--enable-libvpx --enable-libdrm --enable-opencl --enable-gpl --cpu=native
--enable-opengl --enable-libfdk-aac --enable-libx265 --enable-openssl
--extra-libs='-lpthread -lm -lz' --enable-nonfree
  libavutil      56. 26.101 / 56. 26.101
  libavcodec     58. 52.101 / 58. 52.101
  libavformat    58. 27.103 / 58. 27.103
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 50.100 /  7. 50.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Input #0, matroska,webm, from 'rv.mkv':
  Metadata:
    ENCODER         : Lavf58.27.102
  Duration: 00:02:24.77, start: 0.000000, bitrate: 20063 kb/s
    Stream #0:0: Video: h264 (High 4:4:4 Predictive), yuv444p(progressive),
2560x1440, 60 fps, 60 tbr, 1k tbn, 120 tbc (default)
    Metadata:
      ENCODER         : Lavc58.51.100 libx264
      DURATION        : 00:02:24.767000000


Attempting to use either cuvid,nvdec or auto (which selects cuda) for
hwaccel results in the following message:

[h264 @ 0x55ff3d516880] Hardware is lacking required capabilities
[h264 @ 0x55ff3d516880] Failed setup for format cuda: hwaccel
initialisation returned error.

But the encode session runs to completion.

Explicitly setting hwaccel to vdpau fails as such:

[h264 @ 0x560d4b9f3340] hardware accelerator failed to decode picture
Error while decoding stream #0:0: Input/output error
[h264 @ 0x560d4b9f9280] hardware accelerator failed to decode picture
Error while decoding stream #0:0: Input/output error
[h264 @ 0x560d4bd5a8c0] hardware accelerator failed to decode picture
Error while decoding stream #0:0: Input/output error
[matroska,webm @ 0x560d4b9d2bc0] Element at 0x15a3f012 ending at 0x15a3f0a4
exceeds containing master element ending at 0x15a3f006
[h264 @ 0x560d4bf22d00] hardware accelerator failed to decode picture
Error while decoding stream #0:0: Input/output error
    Last message repeated 11 times
Output #0, mp4, to
'/home/brainiarc7/Desktop/src/benchmarks/final/benchmark-nvenc-file.mp4':
  Metadata:
    encoder         : Lavf58.27.103
    Stream #0:0: Video: h264 (h264_nvenc) (High 4:4:4 Predictive) (avc1 /
0x31637661), yuv444p, 2560x1440, q=-1--1, 7500 kb/s, 60 fps, 15360 tbn, 60
tbc (default)
    Metadata:
      DURATION        : 00:02:24.767000000
      encoder         : Lavc58.52.101 h264_nvenc
    Side data:
      cpb: bitrate max/min/avg: 15000000/0/7500000 buffer size: 15000000
vbv_delay: -1
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=00:00:00.00 bitrate=N/A
speed=   0x
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: unknown
Conversion failed!

In this case, with vdpau, the error is not recoverable and the encoder
fails.

The command used:

ffmpeg -y -stats -hwaccel nvdec \
-i $(pwd)/rv.mkv  \
-c:v h264_nvenc -bf:v 3 -rc:v vbr_hq  -qp:v 19 -profile:v main  \
-b:v 7.5M -maxrate:v 15M -r:v 60  -an -f mp4 $(pwd)/benchmark-nvenc-file.mp4


FFmpeg version used:

ffmpeg
ffmpeg version N-93814-g48539b62fc Copyright (c) 2000-2019 the FFmpeg
developers
  built with gcc 7 (Ubuntu 7.4.0-1ubuntu1~18.04)
  configuration: --pkg-config-flags=--static --prefix=/home/brainiarc7/bin
--bindir=/home/brainiarc7/bin --extra-cflags=-I/home/brainiarc7/bin/include
--extra-ldflags=-L/home/brainiarc7/bin/lib --enable-cuda-nvcc
--enable-cuvid --enable-libnpp --extra-cflags=-I/usr/local/cuda/include/
--extra-ldflags=-L/usr/local/cuda/lib64/ --enable-nvenc
--extra-cflags=-I/opt/intel/mediasdk/include
--extra-ldflags=-L/opt/intel/mediasdk/lib
--extra-ldflags=-L/opt/intel/mediasdk/plugins --enable-libmfx
--enable-libass --enable-vaapi --disable-debug --enable-libvorbis
--enable-libvpx --enable-libdrm --enable-opencl --enable-gpl --cpu=native
--enable-opengl --enable-libfdk-aac --enable-libx265 --enable-openssl
--extra-libs='-lpthread -lm -lz' --enable-nonfree
  libavutil      56. 26.101 / 56. 26.101
  libavcodec     58. 52.101 / 58. 52.101
  libavformat    58. 27.103 / 58. 27.103
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 50.100 /  7. 50.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options]
outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

The GPU matrix published by NVIDIA only indicates support for HEVC @ 4:4:4
: https://developer.nvidia.com/video-encode-decode-gpu-support-matrix. That
page indicates support for H.264 4:4:4 in NVENC (as from GM107+) but
information on decode capability on the same is absent.

According to the commit:
https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/83c7ac2e47efd96927127c1c385cdbb5fb53cb02

Only two codecs can handle 4:4:4: HEVC and MJPEG.
Would this imply that support for H.264 @ 4:4:4 is either unsupported (in
hardware, as implied above) or  omitted in FFmpeg? Please clarify.

Thanks for your time,

Dennis.