[FFmpeg-devel] [PATCH v4 1/1] avutils/hwcontext: When deriving a hwdevice, search for existing device in both directions

[Mark Thompson]

On 10/01/2022 06:47, Xiang, Haihao wrote:
> 
> Hi Mark,
> 
> We want to provide a more user friendly command-line to share gfx memory between
> QSV, VAAPI and other HW methods.
> 
> E.g. VAAPI provides sharpness_vaapi but QSV doesn't provide a corresponding
> filter, we want to use sharpness_vaapi filter on the output from QSV decoders.
> Currently the first command-line below may work, however the second command line
> below can't work because QSV device is not derived from a VAAPI device
> explicitly, so ffmpeg fails to derive VAAPI device from QSV device (it may
> derive VAAPI device from QSV device in the first case)
> 
> $ ffmpeg -init_hw_device vaapi=intel -init_hw_device qsv=qsv at intel -hwaccel qsv
> -c:v h264_qsv -i input.mp4 -vf hwmap=derive_device=vaapi,sharpness_vaapi -f null
> -

With explicit device selection:

$ ffmpeg -init_hw_device vaapi=intel -init_hw_device qsv=qsv at intel -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf hwmap{intel},sharpness_vaapi -f null -

(Exact syntax unknown, but I was intending something like that.)

> 
> $ ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf
> hwmap=derive_device=vaapi,sharpness_vaapi -f null -

This is really wanting the reverse case of device derivation.  I guess this does want the libmfx hwcontext to always have the source device there, as suggested above.

> After applying Softworks' patch, the above two command-lines may work well. In
> addition, we may use other HW methods on QSV output without copy for gfx memory,
> e.g.
> 
> $ ffmpeg -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf
> "hwmap=derive_device=vaapi,format=vaapi,hwmap=derive_device=vulkan,scale_vulkan=
> w=1920:h=1080" -f null -

So, we move the derivation out of the graph and get:

$ ffmpeg -init_hw_device vaapi=vadev -init_hw_device qsv=qsvdev at vadev -init_hw_device vulkan=vkdev at vadev \
   -hwaccel qsv -c:v h264_qsv -i input.mp4 -vf "hwmap{vadev},format=vaapi,hwmap{vkdev},scale_vulkan=w=1920:h=1080" -f null -

(Maybe it would help to have some shorter options for making devices like the first line there, because making a set derived in the right sequence is probably quite a common operation.  If ffmpeg knows the possible-derivations this wouldn't be hard to make.)

Making it all explicit also lets multiple physical devices work together straightforwardly, as Intel seems likely to want to support in the not-too-distant future.

- Mark