[FFmpeg-user] OVERLAY_CUDA and PGS Subtitle burn
Dennis Mungai
dmngaie at gmail.com
Tue Nov 23 12:46:25 EET 2021
On Thu, 1 Oct 2020 at 03:57, Panda Sing Cool <pandacoolsing at gmail.com>
wrote:
> Hi Community,
>
> Updated to the Cuda 11.1 (latest) and NVD drivers 455.23.04 (latest beta),
> FFmpeg Git and nv-codec-headers from today. Same result. no subtitles.
> So i will take a break on this option, maybe some bugs somewhere , which
> will have to wait until a dev has a look on this :)
>
> Is it something I can highlight on the Dev FFmpeg list, or the dev list is
> only reserved/exclusive for the dev ?
>
> So, i'm using now the traditional overlay filter, which give me the result
> i'm looking for:
>
> /ffmpeg -y -init_hw_device cuda=cuda -hwaccel cuda -i input.mkv
> -filter_complex
> "[0:v][0:s]overlay[vid];[vid]hwupload,scale_cuda=w=-1:h=720[vid]" -map
> "[vid]" -c:v h264_nvenc test.mkv
>
> i'm running around 400 fps (x16) on this command on my gear. Not sure if
> this is considered slow or fast, but for my own requirements it's nice
> already.
>
> Question related to this part only:
> -filter_complex
> "[0:v][0:s]overlay[vid];[vid]hwupload,scale_cuda=w=-1:h=720[vid]" :
>
> - based on this simple command, any tips/tricks to increase the FPS or
> it's considered already optimal.
>
>
> Many thanks.
>
>
>
> On Wed, 23 Sep 2020 at 08:08, Panda Sing Cool <pandacoolsing at gmail.com>
> wrote:
>
> > Hi Dennis,
> >
> > thanks for the link. I have rebuild ffmpeg with all the latest version of
> > any source i can find, update to the latest cuda 11 patch 3 and NV
> > headers...
> > Also included the latest vulkan sdk and support for ffmpeg.
> >
> > overlay_cuda and overlay_opencl, same issue: video appear but not
> subtitle.
> > overlay_vulkan -> direct crash dump from ffmpeg.
> >
> > :(
> >
> > Any change from your side ?
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Fri, 18 Sep 2020 at 22:46, Dennis Mungai <dmngaie at gmail.com> wrote:
> >
> >> On Thu, 17 Sep 2020 at 03:29, Panda Sing Cool <pandacoolsing at gmail.com>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > Changed the input format:
> >> > Video -> yuv420p
> >> > Sub -> yuv*a*420p (to include Alpha Channel)
> >> >
> >> > Now the video is showing, but still no subtitles.
> >> > Still get the error message:
> >> > *Error while add the frame to buffer source(Internal bug, should not
> >> have
> >> > happened).*
> >> >
> >> >
> >> > ./ffmpeg -threads 1 -loglevel info -nostdin -y -fflags
> +genpts-fastseek
> >> \
> >> > -ss 00:00:00 -t 00:00:15 \
> >> > -extra_hw_frames 3 -vsync 0 -async 0 -filter_threads 1
> >> > -filter_complex_threads 1 \
> >> > -init_hw_device cuda=cuda -hwaccel cuda -filter_hw_device cuda
> >> > -hwaccel_output_format cuda \
> >> > -i input.mkv \
> >> >
> >> >
> >> >
> >> >
> >> > * -filter_complex \
> >> > "[0:v]scale_npp=w=-1:h=720:interp_algo=lanczos:format=yuv420p[vid]; \
> >> > [0:s]format=yuva420p,hwupload[sub]; \
> >> [vid][sub]overlay_cuda[v]"
> >> > \* -map "[v]" -map 0:a \
> >> > -force_key_frames "expr:gte(t,n_forced*5)" \
> >> > -c:v h264_nvenc -preset:v slow -profile:v high -level:v 51 \
> >> > -rc:v cbr_hq -rc-lookahead:v 32 -refs:v 16 -cq:v 16 -bf:v 3 -b:v
> >> 2000K
> >> > -minrate:v 2000K -maxrate:v 2000k -bufsize:v 8M -coder:v cabac
> >> > -b_ref_mode:v middle \
> >> > -c:a libfdk_aac -ac 2 -ar 48000 -b:a 128k \
> >> > output.mkv
> >> >
> >> >
> >> >
> >> >
> >> > On Thu, 17 Sep 2020 at 07:08, Panda Sing Cool <
> pandacoolsing at gmail.com>
> >> > wrote:
> >> >
> >> > > Hi everyone,
> >> > >
> >> > > i'm trying to use the OVERLAY_CUDA function to burn PGS titles over
> a
> >> > > video and the result is not working.
> >> > > i might misunderstand the usage of this function, so some help is
> >> > welcome.
> >> > >
> >> > > The result of this command is a black screen with audio, using
> >> 'standard'
> >> > > overlay filter is working fine, but slow ...
> >> > >
> >> > > Notice this message at the end of the log file (ffmpeg version
> >> > > N-99194-g142ae27b1d ( compiled myself from git) ):
> >> > > *Error while add the frame to buffer source(Internal bug, should not
> >> have
> >> > > happened).*
> >> > >
> >> > > Thanks for any help.
> >> > >
> >> > >
> >> > >
> >> >
> >>
> ***************************************************************************************************************
> >> > >
> >> > > ./ffmpeg -threads 1 -loglevel info -nostdin -y -fflags
> >> > +genpts-fastseek
> >> > > \
> >> > > -ss 00:00:00 -t 00:01:00 \
> >> > > -extra_hw_frames 3 -vsync 0 -async 0 -filter_threads 1
> >> > > -filter_complex_threads 1 \
> >> > > -init_hw_device cuda=cuda -hwaccel cuda -filter_hw_device cuda
> >> > > -hwaccel_output_format cuda \
> >> > > -i input.mkv \
> >> > > -filter_complex \
> >> > >
> >> > >
> >> > > *
> "[0:v]scale_npp=w=-1:h=720:interp_algo=lanczos:format=nv12[vid]; \
> >> > > [0:s]format=nv12,hwupload_cuda[sub]; \ [vid][sub]overlay_cuda[v]"
> >> \*
> >> > > -map "[v]" -map 0:a \
> >> > > -force_key_frames "expr:gte(t,n_forced*5)" \
> >> > > -c:v h264_nvenc -preset:v slow -profile:v high -level:v 51 \
> >> > > -rc:v cbr_hq -rc-lookahead:v 32 -refs:v 16 -cq:v 16 -bf:v 3 -b:v
> >> 2000K
> >> > > -minrate:v 2000K -maxrate:v 2000k -bufsize:v 8M -coder:v cabac
> >> > > -b_ref_mode:v middle \
> >> > > -c:a libfdk_aac -ac 2 -ar 48000 -b:a 128k \
> >> > > output.mkv
> >> > >
> >> > >
> >> > > ***** LOG ****
> >> > >
> >> > > f*fmpeg version N-99194-g142ae27b1d *Copyright (c) 2000-2020 the
> >> FFmpeg
> >> > > developers
> >> > > built with gcc 10 (GCC)
> >> > > configuration: --prefix=/home/users/work/ffmpeg_build
> >> > > --pkg-config-flags=--static --extra-libs=-lpthread --extra-libs=-lm
> >> > > --bindir=/home/users/work/ffmpeg_build/bin --enable-gpl
> >> > --enable-libfdk_aac
> >> > > --enable-libfreetype --enable-libmp3lame --enable-libopus
> >> --enable-libvpx
> >> > > --enable-libx264 --enable-libx265 --enable-vulkan --enable-nonfree
> >> > > --enable-libnpp --enable-nvenc --enable-cuvid --enable-libass
> >> > > --enable-libfontconfig --enable-libfreetype --enable-libfribidi
> >> > > --enable-cuda
> >> > > libavutil 56. 59.100 / 56. 59.100
> >> > > libavcodec 58.106.100 / 58.106.100
> >> > > libavformat 58. 56.100 / 58. 56.100
> >> > > libavdevice 58. 11.102 / 58. 11.102
> >> > > libavfilter 7. 87.100 / 7. 87.100
> >> > > libswscale 5. 8.100 / 5. 8.100
> >> > > libswresample 3. 8.100 / 3. 8.100
> >> > > libpostproc 55. 8.100 / 55. 8.100
> >> > > Input #0, matroska,webm, from 'input.mkv':
> >> > > Metadata:
> >> > > encoder : libebml v1.3.10 + libmatroska v1.5.2
> >> > > creation_time : 2020-08-13T06:58:46.000000Z
> >> > > Duration: 00:57:53.06, start: 0.000000, bitrate: 12993 kb/s
> >> > > Chapter #0:0: start 0.000000, end 508.424583
> >> > > Metadata:
> >> > > title : Chapter 01
> >> > > Chapter #0:1: start 508.424583, end 1037.202833
> >> > > Metadata:
> >> > > title : Chapter 02
> >> > > Chapter #0:2: start 1037.202833, end 1510.175333
> >> > > Metadata:
> >> > > title : Chapter 03
> >> > > Chapter #0:3: start 1510.175333, end 2231.896333
> >> > > Metadata:
> >> > > title : Chapter 04
> >> > > Chapter #0:4: start 2231.896333, end 2908.530625
> >> > > Metadata:
> >> > > title : Chapter 05
> >> > > Chapter #0:5: start 2908.530625, end 3473.052917
> >> > > Metadata:
> >> > > title : Chapter 06
> >> > > Stream #0:0: Video: h264 (High), yuv420p(progressive),
> 1920x1080,
> >> SAR
> >> > > 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
> >> > > Metadata:
> >> > > BPS-eng : 10946248
> >> > > DURATION-eng : 00:57:53.053000000
> >> > > NUMBER_OF_FRAMES-eng: 83270
> >> > > NUMBER_OF_BYTES-eng: 4752112489
> >> > > _STATISTICS_WRITING_APP-eng: mkvmerge v46.0.0 ('No Deeper
> >> Escape')
> >> > > 64-bit
> >> > > _STATISTICS_WRITING_DATE_UTC-eng: 2020-08-13 06:58:46
> >> > > _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES
> >> NUMBER_OF_BYTES
> >> > > Stream #0:1(eng): Audio: dts (DTS-HD MA), 48000 Hz, 5.1(side),
> >> s16p
> >> > > Metadata:
> >> > > BPS-eng : 2022940
> >> > > DURATION-eng : 00:57:53.056000000
> >> > > NUMBER_OF_FRAMES-eng: 325599
> >> > > NUMBER_OF_BYTES-eng: 878223124
> >> > > _STATISTICS_WRITING_APP-eng: mkvmerge v46.0.0 ('No Deeper
> >> Escape')
> >> > > 64-bit
> >> > > _STATISTICS_WRITING_DATE_UTC-eng: 2020-08-13 06:58:46
> >> > > _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES
> >> NUMBER_OF_BYTES
> >> > > Stream #0:2(eng): Subtitle: hdmv_pgs_subtitle, 1920x1080
> >> > > Metadata:
> >> > > BPS-eng : 44076
> >> > > DURATION-eng : 00:56:53.660000000
> >> > > NUMBER_OF_FRAMES-eng: 1450
> >> > > NUMBER_OF_BYTES-eng: 18807980
> >> > > _STATISTICS_WRITING_APP-eng: mkvmerge v46.0.0 ('No Deeper
> >> Escape')
> >> > > 64-bit
> >> > > _STATISTICS_WRITING_DATE_UTC-eng: 2020-08-13 06:58:46
> >> > > _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES
> >> NUMBER_OF_BYTES
> >> > > Stream mapping:
> >> > > Stream #0:0 (h264) -> scale_npp (graph 0)
> >> > > Stream #0:2 (pgssub) -> format (graph 0)
> >> > > overlay_cuda (graph 0) -> Stream #0:0 (h264_nvenc)
> >> > > Stream #0:1 -> #0:1 (dts (dca) -> aac (libfdk_aac))
> >> > > [matroska,webm @ 0x3a39bc0] sub2video: using 1920x1080 canvas
> >> > > Output #0, matroska, to 'output.mkv':
> >> > > Metadata:
> >> > > encoder : Lavf58.56.100
> >> > > Chapter #0:0: start 0.000000, end 508.424583
> >> > > Metadata:
> >> > > title : Chapter 01
> >> > > Chapter #0:1: start 508.424583, end 1037.202833
> >> > > Metadata:
> >> > > title : Chapter 02
> >> > > Chapter #0:2: start 1037.202833, end 1510.175333
> >> > > Metadata:
> >> > > title : Chapter 03
> >> > > Chapter #0:3: start 1510.175333, end 2231.896333
> >> > > Metadata:
> >> > > title : Chapter 04
> >> > > Chapter #0:4: start 2231.896333, end 2908.530625
> >> > > Metadata:
> >> > > title : Chapter 05
> >> > > Chapter #0:5: start 2908.530625, end 3473.052917
> >> > > Metadata:
> >> > > title : Chapter 06
> >> > > Stream #0:0: Video: h264 (h264_nvenc) (High) (H264 /
> 0x34363248),
> >> > > cuda, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 23.98 fps, 1k tbn, 23.98
> >> tbc
> >> > > (default)
> >> > > Metadata:
> >> > > encoder : Lavc58.106.100 h264_nvenc
> >> > > Side data:
> >> > > cpb: bitrate max/min/avg: 2000000/0/0 buffer size: 8000000
> >> > > vbv_delay: N/A
> >> > > Stream #0:1(eng): Audio: aac (libfdk_aac) ([255][0][0][0] /
> >> 0x00FF),
> >> > > 48000 Hz, stereo, s16, 128 kb/s
> >> > > Metadata:
> >> > > BPS-eng : 2022940
> >> > > DURATION-eng : 00:57:53.056000000
> >> > > NUMBER_OF_FRAMES-eng: 325599
> >> > > NUMBER_OF_BYTES-eng: 878223124
> >> > > _STATISTICS_WRITING_APP-eng: mkvmerge v46.0.0 ('No Deeper
> >> Escape')
> >> > > 64-bit
> >> > > _STATISTICS_WRITING_DATE_UTC-eng: 2020-08-13 06:58:46
> >> > > _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES
> >> NUMBER_OF_BYTES
> >> > > encoder : Lavc58.106.100 libfdk_aac
> >> > > *Error while add the frame to buffer source(Internal bug, should not
> >> have
> >> > > happened).*
> >> > > frame= 360 fps=241 q=11.0 Lsize= 3813kB time=00:00:15.01
> >> > > bitrate=2079.9kbits/s speed=10.1x
> >> > > video:3568kB audio:236kB subtitle:0kB other streams:0kB global
> >> > headers:0kB
> >> > > muxing overhead: 0.243563%
> >> > > ************************
> >> > >
> >> > >
> >> > > Infos:
> >> > > ******** lspci *******
> >> > > 0e:00.0 VGA compatible controller: NVIDIA Corporation TU102
> [*GeForce
> >> RTX
> >> > > 2080 Ti Rev. A]* (rev a1) (prog-if 00 [VGA controller])
> >> > > Subsystem: ZOTAC International (MCO) Ltd. Device 1503
> >> > > Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> >> > > Stepping- SERR- FastB2B- DisINTx+
> >> > > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> >> <TAbort-
> >> > > <MAbort- >SERR- <PERR- INTx-
> >> > > *Kernel driver in use: nvidia*
> >> > > Kernel modules: nouveau, nvidia_drm, nvidia
> >> > > **********************
> >> > >
> >> > > ******** nvidia-smi *******
> >> > >
> >> > >
> >> >
> >>
> +-----------------------------------------------------------------------------+
> >> > > | NVIDIA-SMI *450.66 Driver Version: 450.66 CUDA
> Version:
> >> > > 11.0* |
> >> > >
> >> > >
> >> >
> >>
> |-------------------------------+----------------------+----------------------+
> >> > > **********************
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> >
> >> >
> >> Related, see: https://trac.ffmpeg.org/ticket/8831
> >>
> >> I ran into the same issue approximately seven weeks ago, as documented
> >> above.
> >> Even downgrading device drivers (from R450, R440 to below) did not
> resolve
> >> the issue.
> >> Another filter affected by the same issue is the overlay_opencl
> >> implementation, wherein the video canvas on overlay turns up black.
> >>
> >> Regards,
> >>
> >> Dennis.
> >> _______________________________________________
> >> ffmpeg-user mailing list
> >> ffmpeg-user at ffmpeg.org
> >> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >>
> >> To unsubscribe, visit link above, or email
> >> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> >
> >
>
>
Tested with the latest git tip (as of today), here is your overlay command,
using overlay_cuda:
/ffmpeg -y cuda=cuda:0,primary_ctx=1 -filter_hw_device cuda -threads 1
-extra_hw_frames 3 \
-i input.mkv \
-filter_complex "[0:v]format=yuv420p,hwupload[main]; \
[0:s]format=yuva420p,hwupload[subs]; \
[main][subs]overlay_cuda[v]; \
[v]scale_cuda=w=-1:h=720[vid]" \
-map "[vid]" -c:v h264_nvenc test.mkv
More information about the ffmpeg-user
mailing list