[FFmpeg-devel] [PATCH] lavc/qsvdec: allow qsv decoders to use initialized device

[Xiang, Haihao]

On Wed, 2020-09-02 at 20:37 +0100, Mark Thompson wrote:
> On 02/09/2020 15:36, Rogozhkin, Dmitry V wrote:
> > On Wed, 2020-09-02 at 14:21 +0000, Rogozhkin, Dmitry V wrote:
> > > On Wed, 2020-09-02 at 08:41 +0000, Soft Works wrote:
> > > > ...
> > 
> > Small suggestion: let's move discussion around -qsv_device and
> > -hwaccel_device options entirely to the "ffmpeg_qsv: use
> > -hwaccel_device to specify a device for VAAPI backend" thread and a
> > return focus on the original patch which is not about -qsv_device, but
> > about this command line:
> > 
> > ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device
> > qsv=hw at va -c:v h264_qsv -i input.h264 -c:v hevc_qsv -y output.h264
> 
> I wouldn't recommend doing this; using the appropriate acceleration API is
> preferred to messing with the vendor-specific full-offload decoders:
> 
> ffmpeg -init_hw_device vaapi:/dev/dri/renderD129 -hwaccel vaapi -i input.h264
> -vf hwmap=derive_device=qsv -c:v hevc_qsv -y output.h264
> 
> (Or ... -init_hw_device dxva2:1 -hwaccel dxva2 ...)
> 
> > For what I see it uses valid device specifications and still in
> > original ffmpeg implementation it results in:
> > - decoder is being run on /dev/dri/renderD128
> > - encoder is being run on /dev/dri/renderD129
> 
> The libmfx decoders need you to set -qsv_device, which is a hack option for
> ffmpeg to allow selecting a device on them because they doesn't support the
> normal device selection.
> 

How about to extend -hwaccel_device to accept drm path for QSV on Linux in
another thread? According to https://trac.ffmpeg.org/ticket/7649, -qsv_device
was added to workaround device select issue on Linux. We may deprecate
-qsv_device once we allow -hwaccel_device to accept drm path.

> The libmfx encoders do support the normal setup in libavcodec, so ffmpeg
> matched hevc_qsv to the libmfx device you created.
> 
> > In general, per my taste, I would try to use the following device
> > specification working with QSV on Linux across all command lines:
> > 
> > -init_hw_device vaapi=va:/dev/dri/renderD129 -init_hw_device qsv=hw at va
> > -filter_hw_device hw
> > 
> > My primary goal is to make it workable for all pipelines. Hence current
> > patch.
> 
> Implementing AV_HW_CONFIG_METHOD_HW_DEVICE_CTX would make sense to allow the
> legacy decoders to use normal device selection, but your patch seems to just
> set the flag indicating support without actually implementing the feature?
> 
> (If you do implement it then you can delete all of the ad-hoc treatment in
> ffmpeg, like has been done for the other hardware codecs.)

AVCodecContext.hw_device_ctx is set in ffmpeg_hw.c as soon as
AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX is set in AVCodecHWConfig. And
AVCodecContext.hw_device_ctx has been taken into account in the qsv path.

Thanks
Haihao

> 
> > > > ...
> > > 
> > > I still don't see the full picture. What I am looking for at the
> > > first
> > > place is how maintainers and architects envision hwaccel to work in
> > > general. Basically, there are few ways to specify the device (-
> > > hwaccel,
> > > -init_hw_device, -filter_hw_device), questions are:
> > > 1. What's scope of applicability of each option w/ explanation why
> > > each
> > > of the option is actually needed (as one of the examples: why -
> > > filter_hw_device is needed and why -init_hw_device can't be used
> > > instead?
> 
> -init_hw_device is a global option which defines a device and adds it to the
> list in ffmpeg; you can have as many of these as you like.
> 
> When setting up a codec which might be able to use a device it tries to match
> against that list, unless you have an explicit -hwaccel_device option for that
> input.  If you do, then it looks for the device with that name.
> 
> To maintain compatibility with old command-lines where -hwaccel_device took a
> string interpreted by special code for each API in ffmpeg (now removed), if it
> doesn't find a device with the given name then it will try to use it to create
> a new one.
> 
> (So "-hwaccel foo -hwaccel_device bar" when "bar" hasn't been previously
> defined ends up with the effect of "-init_hw_device foo=quux:bar -hwaccel foo
> -hwaccel_device quux".)
> 
> This doesn't apply to the libmfx decoders, because they have their own ad-hoc
> code.  Getting rid of that so they work in the same way as everything else
> would be a good thing.
> 
> -filter_hw_device selects a device from the list to use while filtering (e.g.
> by hwupload).
> 
> If there is exactly one device in the list when making a filter graph then it
> will use that, but if there are multiple then it has no way to guess which one
> you meant so the option is needed.
> 
> > > 2. Since there are few methods how component can get a device: what's
> > > the priority order? (for example, if device can be deducted from
> > > incoming frames, is there a way to override it w/ some command line
> > > options?)
> 
> Devices from incoming frames have to win, because they are stored on that
> device - it doesn't make sense to select a different device at that point (you
> would need a separate download/upload or mapping step to do that).
> 
> - Mark
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".