[FFmpeg-devel] [PATCH v4 1/1] avutils/hwcontext: When deriving a hwdevice, search for existing device in both directions

[Xiang, Haihao]

On Mon, 2022-01-10 at 21:16 +0000, Mark Thompson wrote:
> On 10/01/2022 06:47, Xiang, Haihao wrote:
> > 
> > Hi Mark,
> > 
> > We want to provide a more user friendly command-line to share gfx memory
> > between
> > QSV, VAAPI and other HW methods.
> > 
> > E.g. VAAPI provides sharpness_vaapi but QSV doesn't provide a corresponding
> > filter, we want to use sharpness_vaapi filter on the output from QSV
> > decoders.
> > Currently the first command-line below may work, however the second command
> > line
> > below can't work because QSV device is not derived from a VAAPI device
> > explicitly, so ffmpeg fails to derive VAAPI device from QSV device (it may
> > derive VAAPI device from QSV device in the first case)
> > 
> > $ ffmpeg -init_hw_device vaapi=intel -init_hw_device qsv=qsv at intel -hwaccel
> > qsv
> > -c:v h264_qsv -i input.mp4 -vf hwmap=derive_device=vaapi,sharpness_vaapi -f
> > null
> > -
> 
> With explicit device selection:
> 
> $ ffmpeg -init_hw_device vaapi=intel -init_hw_device qsv=qsv at intel -hwaccel
> qsv -c:v h264_qsv -i input.mp4 -vf hwmap{intel},sharpness_vaapi -f null -
> 
> (Exact syntax unknown, but I was intending something like that.)

I got the error below if using '-vf hwmap{intel}':

[AVFilterGraph @ 0x556e3141f750] No such filter: 'hwmap{intel}'. 

I also went through the source code for the right syntax, but failed to find it.

The working syntax for me is to use '-vf hwmap=derive_device=vaapi'

> 
> > 
> > $ ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf
> > hwmap=derive_device=vaapi,sharpness_vaapi -f null -
> 
> This is really wanting the reverse case of device derivation.  I guess this
> does want the libmfx hwcontext to always have the source device there, as
> suggested above.


Always setting the source device for qsv device will break 'make fate-hwdevice'. 

(source device can't be set when qsv device is created via '-init_hw_device
qsv=intel)

> 
> > After applying Softworks' patch, the above two command-lines may work well.
> > In
> > addition, we may use other HW methods on QSV output without copy for gfx
> > memory,
> > e.g.
> > 
> > $ ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf
> > "hwmap=derive_device=vaapi,format=vaapi,hwmap=derive_device=vulkan,scale_vul
> > kan=
> > w=1920:h=1080" -f null -
> 
> So, we move the derivation out of the graph and get:
> 
> $ ffmpeg -init_hw_device vaapi=vadev -init_hw_device qsv=qsvdev at vadev
> -init_hw_device vulkan=vkdev at vadev \
>    -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf
> "hwmap{vadev},format=vaapi,hwmap{vkdev},scale_vulkan=w=1920:h=1080" -f null -

See above, -vf hwmap{avdev} doesn't work. 

BTW you prefer to derive a device via -init_hw_device option, right ? It seems
the command line using derive_device option is more straightforward. 

Thanks
Haihao

> 
> (Maybe it would help to have some shorter options for making devices like the
> first line there, because making a set derived in the right sequence is
> probably quite a common operation.  If ffmpeg knows the possible-derivations
> this wouldn't be hard to make.)
> 
> Making it all explicit also lets multiple physical devices work together
> straightforwardly, as Intel seems likely to want to support in the not-too-
> distant future.
> 
> - Mark
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".