[FFmpeg-user] Clarification for decoding AV1 input on the GPU
Mario Roy
marioeroy at gmail.com
Fri Aug 5 07:57:57 EEST 2022
Hi,
I missed checking the LIBVA_DRIVER_NAME env in my prior email. Apologies.
LIBVA_DRIVER_NAME=*vdpau* \
ffmpeg -y -hwaccel vaapi -i input_vp9.webm -f null - # *segfault*, though
okay with -c:v vp9
LIBVA_DRIVER_NAME=*nvdec* \
ffmpeg -y -hwaccel vaapi -i input_vp9.webm -f null - # decodes on GPU
I updated my repos: *nvidia_drv_video.so* now points to *nvdec_drv_video.so*
going forward. Running ffmpeg without env (e.g. unset LIBVA_DRIVER_NAME)
picks up nvidia by default.
https://github.com/marioroy/nvidia-driver-on-clear-linux
https://github.com/marioroy/nvidia-acceleration-on-ubuntu
Regards,
Mario
On Thu, Aug 4, 2022 at 11:52 AM Mario Roy <marioeroy at gmail.com> wrote:
> Hi Haihao,
>
> Congrats on the patchset to FFmpeg. I saw patches df5defe and ad67ea9 were
> recently committed and thought to retest FFmpeg (on master branch), using
> NVIDIA 3070 RTX.
>
>
> https://github.com/FFmpeg/FFmpeg/commit/df5defe15a29dfca5f13224eb5da821095f6c745
>
> https://github.com/FFmpeg/FFmpeg/commit/ad67ea9eeed2cc8edab8662b7503ce4a49247b1d
>
>
> *First, I tried -hwaccel without -c:v.*
>
> ffmpeg -y -hwaccel nvdec -i input_vp9.webm -f null - # decodes on GPU
> ffmpeg -y -hwaccel vaapi -i input_vp9.webm -f null - # *segfault*,
> though okay with -c:v vp9
> ffmpeg -y -hwaccel vdpau -i input_vp9.webm -f null - # decodes on GPU
>
> ffmpeg -y -hwaccel nvdec -i input_av1.mp4 -f null - # decodes on GPU
> ffmpeg -y -hwaccel vaapi -i input_av1.mp4 -f null - # decodes on GPU
> ffmpeg -y -hwaccel vdpau -i input_av1.mp4 -f null - # decodes on GPU
>
> *Next, I tried incorrect arguments.* Why does FFmpeg not exit gracefully?
>
> ffmpeg -y -hwaccel vaapi -c:v vp9_cuvid -i input_vp9.webm -f null - #
> *segfault*
> ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm -f null - #
> *segfault*
> ffmpeg -y -hwaccel vaapi -c:v av1_cuvid -i input_av1.mp4 -f null - #
> *segfault*
> ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4 -f null - #
> *segfault*
>
> *Then, I compared -hwaccel nvdec with/without -c:v* {vp9_cuvid,
> av1_cuvid}.
>
> ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm -f null - #
> 1436 fps
> ffmpeg -y -hwaccel nvdec -i input_vp9.webm -f null - #
> 1428 fps
>
> ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4 -f null - #
> 1629 fps
> ffmpeg -y -hwaccel nvdec -i input_av1.mp4 -f null - # *895
> fps*
>
> I cannot understand why ffmpeg does not automatically choose the cuvid
> path for -hwaccel nvdec. There is a noticeable difference for av1.
>
> Best regards,
> Mario
>
>
> On Tue, Jul 26, 2022 at 8:50 PM Mario Roy <marioeroy at gmail.com> wrote:
>
>> Thanks for the explanation regarding cuvid.
>>
>> Mario
>>
>>
>> On Wed, Jul 20, 2022 at 11:27 PM Xiang, Haihao <
>> haihao.xiang-at-intel.com at ffmpeg.org> wrote:
>>
>>> On Wed, 2022-07-20 at 08:02 -0500, Mario Roy wrote:
>>> > Thank you Haihao, for the URL to patchset.
>>> >
>>> > Previously, I applied only the patch contained in the email thread.
>>> This
>>> > time, I applied both patches to FFmpeg master and confirmed that
>>> > av1(native) is used for nvdec, vaapi, and vdpau. I observed 100% and
>>> 50% ~
>>> > 68% video engine utilization for VP9 and AV1 respectively. Unix time is
>>> > captured for output.ts.
>>> >
>>> > Video Engine Utilization: 100%
>>> >
>>> > $ ffmpeg -y -hwaccel nvdec -i input_vp9.webm -f null - # GPU fps=1441
>>> > $ ffmpeg -y -hwaccel vaapi -i input_vp9.webm -f null - # GPU fps=1320
>>> > $ ffmpeg -y -hwaccel vdpau -i input_vp9.webm -f null - # GPU fps=1308
>>> >
>>> > Video Engine Utilization: 50% ~ 68%
>>> >
>>> > $ ffmpeg -y -hwaccel nvdec -i input_av1.mp4 -f null - # GPU fps=900
>>> > $ ffmpeg -y -hwaccel vaapi -i input_av1.mp4 -f null - # GPU fps=760
>>> > $ ffmpeg -y -hwaccel vdpau -i input_av1.mp4 -f null - # GPU fps=799
>>>
>>> Glad to see the patchset works for you, I'll resend the patchset to
>>> FFmpeg ML.
>>>
>>> >
>>> > Unix Time output.ts:
>>> > -hwaccel vdpau is unexpectedly slow for av1
>>> >
>>> > $ time ffmpeg -y -hwaccel nvdec -i input_vp9.webm output.ts # GPU
>>> 6.176s
>>> > $ time ffmpeg -y -hwaccel vaapi -i input_vp9.webm output.ts # GPU
>>> 7.754s
>>> > $ time ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts # GPU
>>> 7.213s
>>> >
>>> > $ time ffmpeg -y -hwaccel nvdec -i input_av1.mp4 output.ts # GPU
>>> 8.709s
>>> > $ time ffmpeg -y -hwaccel vaapi -i input_av1.mp4 output.ts # GPU
>>> 10.816s
>>> > $ time ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts # GPU
>>> 15.162s
>>> >
>>> > The next two commands core dumps using FFmpeg master, ditto for FFmpeg
>>> 5.1.
>>> >
>>> > $ ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm -f null -
>>> #
>>> > core dumped
>>> > $ ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4 -f null - #
>>> > core dumped
>>>
>>> I am not familiar with cuvid decoders, but according to
>>>
>>> https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/cuviddec.c#L1099-L1110,
>>>
>>> cuvid decoders use CUDA device. I guess you should use -hwaccel cuvid
>>> which is
>>> remapped to -hwaccel cuda in FFmpeg, see
>>>
>>>
>>> https://github.com/FFmpeg/FFmpeg/blob/master/fftools/ffmpeg_opt.c#L1000-L1001
>>>
>>> BRs
>>> Haihao
>>>
>>>
>>> >
>>> > No issues for nvdec. Passing -c:v av1_cuvid is swift, compared to 900
>>> fps
>>> > above without -c:v av1_cuvid.
>>> >
>>> > $ ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm -f null -
>>> #
>>> > fps=1435
>>> > $ ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4 -f null - #
>>> > fps=1642
>>> >
>>> >
>>> > Thanks,
>>> > Mario
>>> >
>>> >
>>> > On Wed, Jul 20, 2022 at 4:30 AM Xiang, Haihao <
>>> > haihao.xiang-at-intel.com at ffmpeg.org> wrote:
>>> >
>>> > >
>>> > > > Thank you, Haihao. I applied the patchset to FFmpeg 5.1 branch.
>>> Passing
>>> > > > -hwaccel {nvdec|vdpau} without -c:v av1 still decoded on the CPU. I
>>> > >
>>> > > double
>>> > > > checked to be sure the patchset was applied.
>>> > > >
>>> > >
>>> > > libdav1d is still used for format probing but av1(native) should be
>>> used
>>> > > for
>>> > > decoding when -hwaccel xxx is specified.
>>> > >
>>> > > I rebased the patches on the master(
>>> > > https://github.com/xhaihao/FFmpeg/tree/ffmpeg-choose-dec) however I
>>> don't
>>> > > have
>>> > > the environment to verify nvdec/vdpau. I verified it with vaapi
>>> > >
>>> > > $ ffmpeg -hwaccel vaapi -i av1.ivf -f null -
>>> > >
>>> > > [...]
>>> > > Stream mapping:
>>> > > Stream #0:0 -> #0:0 (av1 (native) -> wrapped_avframe (native))
>>> > >
>>> > > $ ffmpeg -i av1.ivf -f null -
>>> > >
>>> > > [...]
>>> > > Stream mapping:
>>> > > Stream #0:0 -> #0:0 (av1 (libdav1d) -> wrapped_avframe (native))
>>> > >
>>> > > Thanks
>>> > > Haihao
>>> > >
>>> > >
>>> > > > I obtained AV1 and VP9 samples via yt-dlp: 1920x1080 resolution.
>>> > > >
>>> > > > $ yt-dlp -F https://youtu.be/mIAfxj7nd9k
>>> > > > $ yt-dlp -f 399 https://youtu.be/mIAfxj7nd9k -o input_av1.mp4
>>> > > > $ yt-dlp -f 248 https://youtu.be/mIAfxj7nd9k -o input_vp9.webm
>>> > > >
>>> > > > Hardware: GeForce RTX 3070 FE
>>> > > >
>>> > > > AV1 Observations.
>>> > > >
>>> > > > $ ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4
>>> output.ts #
>>> > > > GPU fps=653
>>> > > > $ ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4
>>> output.ts #
>>> > > > Core Dumps
>>> > > > $ ffmpeg -y -hwaccel nvdec -c:v av1 -i input_av1.mp4 output.ts
>>> #
>>> > > > GPU fps=454
>>> > > > $ ffmpeg -y -hwaccel vdpau -c:v av1 -i input_av1.mp4 output.ts
>>> #
>>> > > > GPU fps=246
>>> > > > $ ffmpeg -y -hwaccel nvdec -i input_av1.mp4 output.ts
>>> #
>>> > > > CPU libdav1d
>>> > > > $ ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts
>>> #
>>> > > > CPU libdav1d
>>> > > >
>>> > > > VP9 Observations.
>>> > > >
>>> > > > $ ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm
>>> output.ts #
>>> > > > GPU fps=669
>>> > > > $ ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm
>>> output.ts #
>>> > > > Core Dumps
>>> > > > $ ffmpeg -y -hwaccel nvdec -c:v vp9 -i input_vp9.webm output.ts
>>> #
>>> > > > GPU fps=662
>>> > > > $ ffmpeg -y -hwaccel vdpau -c:v vp9 -i input_vp9.webm output.ts
>>> #
>>> > > > GPU fps=529
>>> > > > $ ffmpeg -y -hwaccel nvdec -i input_vp9.webm output.ts
>>> #
>>> > > > GPU fps=655
>>> > > > $ ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts
>>> #
>>> > > > GPU fps=533
>>> > > >
>>> > > > Looking at the AV1 results, -hwaccel vdpau -c:v av1 is nearly 2x
>>> slower
>>> > > > than -hwaccel nvdec -c:v av1.
>>> > > >
>>> > > > Should FFmpeg exit gracefully if passing unsupported options for
>>> -hwaccel
>>> > > > vdpau? Or is this a bug?
>>> > > > -hwaccel vdpau -c:v av1_cuvid # Crashes
>>> > > > -hwaccel vdpau -c:v vp9_cuvid # Crashes
>>> > > >
>>> > > > Thanks,
>>> > > > Mario
>>> > > >
>>> > > > On Tue, Jul 19, 2022 at 8:42 PM Xiang, Haihao <
>>> > > > haihao.xiang-at-intel.com at ffmpeg.org> wrote:
>>> > > >
>>> > > > > On Tue, 2022-07-19 at 16:54 -0500, Mario Roy wrote:
>>> > > > > > Greetings,
>>> > > > > >
>>> > > > > > With the recent FFmpeg 5.1 supporting AV1 using VDPAU, should
>>> > >
>>> > > decoding
>>> > > > >
>>> > > > > AV1
>>> > > > > > occur on the GPU automatically (NVIDIA 3000 series), similarly
>>> to
>>> > > > >
>>> > > > > decoding
>>> > > > > > VP9?
>>> > > > > >
>>> > > > > > ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts #
>>> decodes on
>>> > >
>>> > > the
>>> > > > >
>>> > > > > CPU
>>> > > > >
>>> > > > >
>>> > > > > libdav1d has higher priority than the the native av1 decoder in
>>> > >
>>> > > FFmpeg, you
>>> > > > > should specify the native av1 in your command line if you want
>>> to use
>>> > > > > vdpau.
>>> > > > >
>>> > > > > $ ffmpeg -y -hwaccel vdpau -c:v av1 -i input_av1.mp4 output.ts
>>> > > > >
>>> > > > > Or you may try
>>> > > > >
>>> > > > >
>>> > >
>>> > >
>>>
>>> https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210922074207.19451-2-haihao.xiang@intel.com/
>>> > > > > which can decode AV1 on the GPU automatically when -hwaccel xxx
>>> is
>>> > > > > specified in
>>> > > > > the command line. (You need to rebase the patchset).
>>> > > > >
>>> > > > > Thanks
>>> > > > > Haihao
>>> > > > >
>>> > > > >
>>> > > > > > ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts #
>>> decodes on
>>> > >
>>> > > the
>>> > > > >
>>> > > > > GPU
>>> > > > > >
>>> > > > > > Are the following incorrect usage? Both segfaults.
>>> > > > > >
>>> > > > > > ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4
>>> output.ts
>>> > >
>>> > > #
>>> > > > > > segfaults
>>> > > > > > ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm
>>> output.ts
>>> > >
>>> > > #
>>> > > > > > segfaults
>>> > > > > >
>>> > > > > > Thanks,
>>> > > > > > Mario
>>> > > > > >
>>> > > > > > On Wed, Jul 13, 2022 at 9:44 AM Mario Roy <marioeroy at gmail.com
>>> >
>>> > >
>>> > > wrote:
>>> > > > > >
>>> > > > > > > Hi,
>>> > > > > > >
>>> > > > > > > What are the recommended arguments for folks using NVIDIA
>>> 3000
>>> > >
>>> > > series
>>> > > > > > > graphics and decoding AV1 media? I tried the same with VP9
>>> media.
>>> > > > > > >
>>> > > > > > > The following was done using FFmpeg 5.1.
>>> > > > > > >
>>> > > > > > > ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4
>>> > > > > > > output.ts segfaults
>>> > > > > > > ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm
>>> > > > > > > output.ts segfaults
>>> > > > > > >
>>> > > > > > > ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts
>>> decodes on
>>> > >
>>> > > the
>>> > > > >
>>> > > > > CPU
>>> > > > > > > ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts
>>> decodes on
>>> > >
>>> > > the
>>> > > > >
>>> > > > > GPU
>>> > > > > > >
>>> > > > > > > ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4
>>> output.ts
>>> > > > > > > ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm
>>> output.ts
>>> > > > > > >
>>> > > > > > > ffmpeg -y -c:v av1_cuvid -i input_av1.mp4 output.ts
>>> > > > > > > ffmpeg -y -c:v vp9_cuvid -i input_vp9.webm output.ts
>>> > > > > > >
>>> > > > > > > Best,
>>> > > > > > > Mario
>>> > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > > > _______________________________________________
>>> > > > > > ffmpeg-user mailing list
>>> > > > > > ffmpeg-user at ffmpeg.org
>>> > > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>> > > > > >
>>> > > > > > To unsubscribe, visit link above, or email
>>> > > > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>>> > > > >
>>> > > > > _______________________________________________
>>> > > > > ffmpeg-user mailing list
>>> > > > > ffmpeg-user at ffmpeg.org
>>> > > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>> > > > >
>>> > > > > To unsubscribe, visit link above, or email
>>> > > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>>> > > > >
>>> > > >
>>> > > > _______________________________________________
>>> > > > ffmpeg-user mailing list
>>> > > > ffmpeg-user at ffmpeg.org
>>> > > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>> > > >
>>> > > > To unsubscribe, visit link above, or email
>>> > > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>>> > >
>>> > > _______________________________________________
>>> > > ffmpeg-user mailing list
>>> > > ffmpeg-user at ffmpeg.org
>>> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>> > >
>>> > > To unsubscribe, visit link above, or email
>>> > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>>> > >
>>> >
>>> > _______________________________________________
>>> > ffmpeg-user mailing list
>>> > ffmpeg-user at ffmpeg.org
>>> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>> >
>>> > To unsubscribe, visit link above, or email
>>> > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>>> _______________________________________________
>>> ffmpeg-user mailing list
>>> ffmpeg-user at ffmpeg.org
>>> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>>>
>>> To unsubscribe, visit link above, or email
>>> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>>>
>>
More information about the ffmpeg-user
mailing list