[FFmpeg-user] Clarification for decoding AV1 input on the GPU
Mario Roy
marioeroy at gmail.com
Wed Jul 27 04:50:59 EEST 2022
Thanks for the explanation regarding cuvid.
Mario
On Wed, Jul 20, 2022 at 11:27 PM Xiang, Haihao <
haihao.xiang-at-intel.com at ffmpeg.org> wrote:
> On Wed, 2022-07-20 at 08:02 -0500, Mario Roy wrote:
> > Thank you Haihao, for the URL to patchset.
> >
> > Previously, I applied only the patch contained in the email thread. This
> > time, I applied both patches to FFmpeg master and confirmed that
> > av1(native) is used for nvdec, vaapi, and vdpau. I observed 100% and 50%
> ~
> > 68% video engine utilization for VP9 and AV1 respectively. Unix time is
> > captured for output.ts.
> >
> > Video Engine Utilization: 100%
> >
> > $ ffmpeg -y -hwaccel nvdec -i input_vp9.webm -f null - # GPU fps=1441
> > $ ffmpeg -y -hwaccel vaapi -i input_vp9.webm -f null - # GPU fps=1320
> > $ ffmpeg -y -hwaccel vdpau -i input_vp9.webm -f null - # GPU fps=1308
> >
> > Video Engine Utilization: 50% ~ 68%
> >
> > $ ffmpeg -y -hwaccel nvdec -i input_av1.mp4 -f null - # GPU fps=900
> > $ ffmpeg -y -hwaccel vaapi -i input_av1.mp4 -f null - # GPU fps=760
> > $ ffmpeg -y -hwaccel vdpau -i input_av1.mp4 -f null - # GPU fps=799
>
> Glad to see the patchset works for you, I'll resend the patchset to FFmpeg
> ML.
>
> >
> > Unix Time output.ts:
> > -hwaccel vdpau is unexpectedly slow for av1
> >
> > $ time ffmpeg -y -hwaccel nvdec -i input_vp9.webm output.ts # GPU
> 6.176s
> > $ time ffmpeg -y -hwaccel vaapi -i input_vp9.webm output.ts # GPU
> 7.754s
> > $ time ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts # GPU
> 7.213s
> >
> > $ time ffmpeg -y -hwaccel nvdec -i input_av1.mp4 output.ts # GPU
> 8.709s
> > $ time ffmpeg -y -hwaccel vaapi -i input_av1.mp4 output.ts # GPU
> 10.816s
> > $ time ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts # GPU
> 15.162s
> >
> > The next two commands core dumps using FFmpeg master, ditto for FFmpeg
> 5.1.
> >
> > $ ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm -f null - #
> > core dumped
> > $ ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4 -f null - #
> > core dumped
>
> I am not familiar with cuvid decoders, but according to
>
> https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/cuviddec.c#L1099-L1110,
>
> cuvid decoders use CUDA device. I guess you should use -hwaccel cuvid
> which is
> remapped to -hwaccel cuda in FFmpeg, see
>
>
> https://github.com/FFmpeg/FFmpeg/blob/master/fftools/ffmpeg_opt.c#L1000-L1001
>
> BRs
> Haihao
>
>
> >
> > No issues for nvdec. Passing -c:v av1_cuvid is swift, compared to 900 fps
> > above without -c:v av1_cuvid.
> >
> > $ ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm -f null - #
> > fps=1435
> > $ ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4 -f null - #
> > fps=1642
> >
> >
> > Thanks,
> > Mario
> >
> >
> > On Wed, Jul 20, 2022 at 4:30 AM Xiang, Haihao <
> > haihao.xiang-at-intel.com at ffmpeg.org> wrote:
> >
> > >
> > > > Thank you, Haihao. I applied the patchset to FFmpeg 5.1 branch.
> Passing
> > > > -hwaccel {nvdec|vdpau} without -c:v av1 still decoded on the CPU. I
> > >
> > > double
> > > > checked to be sure the patchset was applied.
> > > >
> > >
> > > libdav1d is still used for format probing but av1(native) should be
> used
> > > for
> > > decoding when -hwaccel xxx is specified.
> > >
> > > I rebased the patches on the master(
> > > https://github.com/xhaihao/FFmpeg/tree/ffmpeg-choose-dec) however I
> don't
> > > have
> > > the environment to verify nvdec/vdpau. I verified it with vaapi
> > >
> > > $ ffmpeg -hwaccel vaapi -i av1.ivf -f null -
> > >
> > > [...]
> > > Stream mapping:
> > > Stream #0:0 -> #0:0 (av1 (native) -> wrapped_avframe (native))
> > >
> > > $ ffmpeg -i av1.ivf -f null -
> > >
> > > [...]
> > > Stream mapping:
> > > Stream #0:0 -> #0:0 (av1 (libdav1d) -> wrapped_avframe (native))
> > >
> > > Thanks
> > > Haihao
> > >
> > >
> > > > I obtained AV1 and VP9 samples via yt-dlp: 1920x1080 resolution.
> > > >
> > > > $ yt-dlp -F https://youtu.be/mIAfxj7nd9k
> > > > $ yt-dlp -f 399 https://youtu.be/mIAfxj7nd9k -o input_av1.mp4
> > > > $ yt-dlp -f 248 https://youtu.be/mIAfxj7nd9k -o input_vp9.webm
> > > >
> > > > Hardware: GeForce RTX 3070 FE
> > > >
> > > > AV1 Observations.
> > > >
> > > > $ ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4
> output.ts #
> > > > GPU fps=653
> > > > $ ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4
> output.ts #
> > > > Core Dumps
> > > > $ ffmpeg -y -hwaccel nvdec -c:v av1 -i input_av1.mp4 output.ts
> #
> > > > GPU fps=454
> > > > $ ffmpeg -y -hwaccel vdpau -c:v av1 -i input_av1.mp4 output.ts
> #
> > > > GPU fps=246
> > > > $ ffmpeg -y -hwaccel nvdec -i input_av1.mp4 output.ts
> #
> > > > CPU libdav1d
> > > > $ ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts
> #
> > > > CPU libdav1d
> > > >
> > > > VP9 Observations.
> > > >
> > > > $ ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm
> output.ts #
> > > > GPU fps=669
> > > > $ ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm
> output.ts #
> > > > Core Dumps
> > > > $ ffmpeg -y -hwaccel nvdec -c:v vp9 -i input_vp9.webm output.ts
> #
> > > > GPU fps=662
> > > > $ ffmpeg -y -hwaccel vdpau -c:v vp9 -i input_vp9.webm output.ts
> #
> > > > GPU fps=529
> > > > $ ffmpeg -y -hwaccel nvdec -i input_vp9.webm output.ts
> #
> > > > GPU fps=655
> > > > $ ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts
> #
> > > > GPU fps=533
> > > >
> > > > Looking at the AV1 results, -hwaccel vdpau -c:v av1 is nearly 2x
> slower
> > > > than -hwaccel nvdec -c:v av1.
> > > >
> > > > Should FFmpeg exit gracefully if passing unsupported options for
> -hwaccel
> > > > vdpau? Or is this a bug?
> > > > -hwaccel vdpau -c:v av1_cuvid # Crashes
> > > > -hwaccel vdpau -c:v vp9_cuvid # Crashes
> > > >
> > > > Thanks,
> > > > Mario
> > > >
> > > > On Tue, Jul 19, 2022 at 8:42 PM Xiang, Haihao <
> > > > haihao.xiang-at-intel.com at ffmpeg.org> wrote:
> > > >
> > > > > On Tue, 2022-07-19 at 16:54 -0500, Mario Roy wrote:
> > > > > > Greetings,
> > > > > >
> > > > > > With the recent FFmpeg 5.1 supporting AV1 using VDPAU, should
> > >
> > > decoding
> > > > >
> > > > > AV1
> > > > > > occur on the GPU automatically (NVIDIA 3000 series), similarly to
> > > > >
> > > > > decoding
> > > > > > VP9?
> > > > > >
> > > > > > ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts # decodes
> on
> > >
> > > the
> > > > >
> > > > > CPU
> > > > >
> > > > >
> > > > > libdav1d has higher priority than the the native av1 decoder in
> > >
> > > FFmpeg, you
> > > > > should specify the native av1 in your command line if you want to
> use
> > > > > vdpau.
> > > > >
> > > > > $ ffmpeg -y -hwaccel vdpau -c:v av1 -i input_av1.mp4 output.ts
> > > > >
> > > > > Or you may try
> > > > >
> > > > >
> > >
> > >
>
> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210922074207.19451-2-haihao.xiang@intel.com/
> > > > > which can decode AV1 on the GPU automatically when -hwaccel xxx is
> > > > > specified in
> > > > > the command line. (You need to rebase the patchset).
> > > > >
> > > > > Thanks
> > > > > Haihao
> > > > >
> > > > >
> > > > > > ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts # decodes
> on
> > >
> > > the
> > > > >
> > > > > GPU
> > > > > >
> > > > > > Are the following incorrect usage? Both segfaults.
> > > > > >
> > > > > > ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4
> output.ts
> > >
> > > #
> > > > > > segfaults
> > > > > > ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm
> output.ts
> > >
> > > #
> > > > > > segfaults
> > > > > >
> > > > > > Thanks,
> > > > > > Mario
> > > > > >
> > > > > > On Wed, Jul 13, 2022 at 9:44 AM Mario Roy <marioeroy at gmail.com>
> > >
> > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > What are the recommended arguments for folks using NVIDIA 3000
> > >
> > > series
> > > > > > > graphics and decoding AV1 media? I tried the same with VP9
> media.
> > > > > > >
> > > > > > > The following was done using FFmpeg 5.1.
> > > > > > >
> > > > > > > ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4
> > > > > > > output.ts segfaults
> > > > > > > ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm
> > > > > > > output.ts segfaults
> > > > > > >
> > > > > > > ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts decodes
> on
> > >
> > > the
> > > > >
> > > > > CPU
> > > > > > > ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts decodes
> on
> > >
> > > the
> > > > >
> > > > > GPU
> > > > > > >
> > > > > > > ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4
> output.ts
> > > > > > > ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm
> output.ts
> > > > > > >
> > > > > > > ffmpeg -y -c:v av1_cuvid -i input_av1.mp4 output.ts
> > > > > > > ffmpeg -y -c:v vp9_cuvid -i input_vp9.webm output.ts
> > > > > > >
> > > > > > > Best,
> > > > > > > Mario
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > ffmpeg-user mailing list
> > > > > > ffmpeg-user at ffmpeg.org
> > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > > > > >
> > > > > > To unsubscribe, visit link above, or email
> > > > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > > > >
> > > > > _______________________________________________
> > > > > ffmpeg-user mailing list
> > > > > ffmpeg-user at ffmpeg.org
> > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > > > >
> > > > > To unsubscribe, visit link above, or email
> > > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > > > >
> > > >
> > > > _______________________________________________
> > > > ffmpeg-user mailing list
> > > > ffmpeg-user at ffmpeg.org
> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > > >
> > > > To unsubscribe, visit link above, or email
> > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > >
> > > _______________________________________________
> > > ffmpeg-user mailing list
> > > ffmpeg-user at ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > >
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > >
> >
> > _______________________________________________
> > ffmpeg-user mailing list
> > ffmpeg-user at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> >
> > To unsubscribe, visit link above, or email
> > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>
More information about the ffmpeg-user
mailing list