[FFmpeg-user] Clarification for decoding AV1 input on the GPU

[Xiang, Haihao]

> Thank you, Haihao. I applied the patchset to FFmpeg 5.1 branch.  Passing
> -hwaccel {nvdec|vdpau} without -c:v av1 still decoded on the CPU. I double
> checked to be sure the patchset was applied.
> 

libdav1d is still used for format probing but av1(native) should be used for
decoding when -hwaccel xxx is specified. 

I rebased the patches on the master(
https://github.com/xhaihao/FFmpeg/tree/ffmpeg-choose-dec) however I don't have
the environment to verify nvdec/vdpau. I verified it with vaapi

$ ffmpeg -hwaccel vaapi -i av1.ivf -f null -

[...]
Stream mapping:
Stream #0:0 -> #0:0 (av1 (native) -> wrapped_avframe (native))

$ ffmpeg -i av1.ivf -f null -

[...]
Stream mapping:
Stream #0:0 -> #0:0 (av1 (libdav1d) -> wrapped_avframe (native))

Thanks
Haihao


> I obtained AV1 and VP9 samples via yt-dlp: 1920x1080 resolution.
> 
> $ yt-dlp -F https://youtu.be/mIAfxj7nd9k
> $ yt-dlp -f 399 https://youtu.be/mIAfxj7nd9k -o input_av1.mp4
> $ yt-dlp -f 248 https://youtu.be/mIAfxj7nd9k -o input_vp9.webm
> 
> Hardware: GeForce RTX 3070 FE
> 
> AV1 Observations.
> 
> $ ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4 output.ts   #
> GPU fps=653
> $ ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4 output.ts   #
> Core Dumps
> $ ffmpeg -y -hwaccel nvdec -c:v av1 -i input_av1.mp4 output.ts         #
> GPU fps=454
> $ ffmpeg -y -hwaccel vdpau -c:v av1 -i input_av1.mp4 output.ts         #
> GPU fps=246
> $ ffmpeg -y -hwaccel nvdec -i input_av1.mp4 output.ts                  #
> CPU libdav1d
> $ ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts                  #
> CPU libdav1d
> 
> VP9 Observations.
> 
> $ ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm output.ts  #
> GPU fps=669
> $ ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm output.ts  #
> Core Dumps
> $ ffmpeg -y -hwaccel nvdec -c:v vp9 -i input_vp9.webm output.ts        #
> GPU fps=662
> $ ffmpeg -y -hwaccel vdpau -c:v vp9 -i input_vp9.webm output.ts        #
> GPU fps=529
> $ ffmpeg -y -hwaccel nvdec -i input_vp9.webm output.ts                 #
> GPU fps=655
> $ ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts                 #
> GPU fps=533
> 
> Looking at the AV1 results, -hwaccel vdpau -c:v av1 is nearly 2x slower
> than -hwaccel nvdec -c:v av1.
> 
> Should FFmpeg exit gracefully if passing unsupported options for -hwaccel
> vdpau? Or is this a bug?
> -hwaccel vdpau -c:v av1_cuvid  # Crashes
> -hwaccel vdpau -c:v vp9_cuvid  # Crashes
> 
> Thanks,
> Mario
> 
> On Tue, Jul 19, 2022 at 8:42 PM Xiang, Haihao <
> haihao.xiang-at-intel.com at ffmpeg.org> wrote:
> 
> > On Tue, 2022-07-19 at 16:54 -0500, Mario Roy wrote:
> > > Greetings,
> > > 
> > > With the recent FFmpeg 5.1 supporting AV1 using VDPAU, should decoding
> > 
> > AV1
> > > occur on the GPU automatically (NVIDIA 3000 series), similarly to
> > 
> > decoding
> > > VP9?
> > > 
> > > ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts   # decodes on the
> > 
> > CPU
> > 
> > 
> > libdav1d has higher priority than the the native av1 decoder in FFmpeg, you
> > should specify the native av1 in your command line if you want to use
> > vdpau.
> > 
> > $ ffmpeg -y -hwaccel vdpau -c:v av1 -i input_av1.mp4 output.ts
> > 
> > Or you may try
> > 
> > 
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210922074207.19451-2-haihao.xiang@intel.com/
> >  which can decode AV1 on the GPU automatically when -hwaccel xxx is
> > specified in
> > the command line. (You need to rebase the patchset).
> > 
> > Thanks
> > Haihao
> > 
> > 
> > > ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts  # decodes on the
> > 
> > GPU
> > > 
> > >  Are the following incorrect usage? Both segfaults.
> > > 
> > > ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4 output.ts   #
> > > segfaults
> > > ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm output.ts  #
> > > segfaults
> > > 
> > > Thanks,
> > > Mario
> > > 
> > > On Wed, Jul 13, 2022 at 9:44 AM Mario Roy <marioeroy at gmail.com> wrote:
> > > 
> > > > Hi,
> > > > 
> > > > What are the recommended arguments for folks using NVIDIA 3000 series
> > > > graphics and decoding AV1 media? I tried the same with VP9 media.
> > > > 
> > > > The following was done using FFmpeg 5.1.
> > > > 
> > > > ffmpeg -y -hwaccel vdpau -c:v av1_cuvid -i input_av1.mp4
> > > > output.ts   segfaults
> > > > ffmpeg -y -hwaccel vdpau -c:v vp9_cuvid -i input_vp9.webm
> > > > output.ts  segfaults
> > > > 
> > > > ffmpeg -y -hwaccel vdpau -i input_av1.mp4 output.ts   decodes on the
> > 
> > CPU
> > > > ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts  decodes on the
> > 
> > GPU
> > > > 
> > > > ffmpeg -y -hwaccel nvdec -c:v av1_cuvid -i input_av1.mp4 output.ts
> > > > ffmpeg -y -hwaccel nvdec -c:v vp9_cuvid -i input_vp9.webm output.ts
> > > > 
> > > > ffmpeg -y -c:v av1_cuvid -i input_av1.mp4 output.ts
> > > > ffmpeg -y -c:v vp9_cuvid -i input_vp9.webm output.ts
> > > > 
> > > > Best,
> > > > Mario
> > > > 
> > > > 
> > > 
> > > _______________________________________________
> > > ffmpeg-user mailing list
> > > ffmpeg-user at ffmpeg.org
> > > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > > 
> > > To unsubscribe, visit link above, or email
> > > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > 
> > _______________________________________________
> > ffmpeg-user mailing list
> > ffmpeg-user at ffmpeg.org
> > https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> > 
> > To unsubscribe, visit link above, or email
> > ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> > 
> 
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> 
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".