[FFmpeg-devel] [PATCH] ffmpeg_qsv: use -hwaccel_device to specify a device for VAAPI backend

[mypopy at gmail.com]

On Thu, Sep 3, 2020 at 9:21 AM Xiang, Haihao <haihao.xiang at intel.com> wrote:
>
> On Wed, 2020-09-02 at 14:04 +0000, Soft Works wrote:
> > > -----Original Message-----
> > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > > Rogozhkin, Dmitry V
> > > Sent: Wednesday, September 2, 2020 3:54 PM
> > > To: ffmpeg-devel at ffmpeg.org
> > > Subject: Re: [FFmpeg-devel] [PATCH] ffmpeg_qsv: use -hwaccel_device to
> > > specify a device for VAAPI backend
> > >
> > > On Wed, 2020-09-02 at 07:31 +0000, Soft Works wrote:
> > > > > -----Original Message-----
> > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > > > > Rogozhkin, Dmitry V
> > > > > Sent: Wednesday, September 2, 2020 9:00 AM
> > > > > To: ffmpeg-devel at ffmpeg.org
> > > > > Subject: Re: [FFmpeg-devel] [PATCH] ffmpeg_qsv: use -hwaccel_device
> > > > > to specify a device for VAAPI backend
> > > > >
> > > > > On Wed, 2020-09-02 at 04:28 +0000, Xiang, Haihao wrote:
> > > > > > On Wed, 2020-09-02 at 04:20 +0000, Soft Works wrote:
> > > > > > > > -----Original Message-----
> > > > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On
> > >
> > > Behalf
> > > > >
> > > > > Of
> > > > > > > > Haihao Xiang
> > > > > > > > Sent: Wednesday, September 2, 2020 6:01 AM
> > > > > > > > To: ffmpeg-devel at ffmpeg.org
> > > > > > > > Cc: Haihao Xiang <haihao.xiang at intel.com>
> > > > > > > > Subject: [FFmpeg-devel] [PATCH] ffmpeg_qsv: use
> > > > > > > > -hwaccel_device to specify a device for VAAPI backend
> > > > > > > >
> > > > > > > > According to the FFmpeg documentation, -hwaccel_device can be
> > > > > > > > used to select a device for hardware acceleration
> > > > > > > >
> > > > > > > > Example:
> > > > > > > > ffmpeg -hwaccel qsv -hwaccel_device /dev/dri/renderD129 -c:v
> > > > > > > > h264_qsv \ -i
> > > > > > > > in.h264 -c:v hevc_qsv out.h265
> > > > > > >
> > > > > > > No. The docs say about hwaccel_device
> > > > > > >
> > > > > > > It can either refer to an existing device created with
> > > > > > > -init_hw_device by name,
> > > > > > >
> > > > > > > => /dev/dri/renderD129 is not an existing device name
> > > > > > >
> > > > > > > or it can create a new device as if ‘-init_hw_device’
> > > > > > > type:hwaccel_device were called immediately before.
> > > > > > >
> > > > > > > => /dev/dri/renderD129 is not a valid string for init_hw_device
> > > > >
> > > > > Well, it seems that's where documentation does not reflect the
> > > > > reality since (as pointed by Haihao below)
> > > > > https://trac.ffmpeg.org/wiki/Hardware/VAAPI
> > > > > is full of contradictory examples.
> > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hence, the patch doesn't make sense.
> > > > > >
> > > > > > in https://trac.ffmpeg.org/wiki/Hardware/VAAPI,
> > > > > >
> > > > > > If only one stream is being used, -hwaccel_device can also accept
> > > > > > a device path directly
> > > > > >
> > > > > > ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -i ...
> > > > > >
> > > > > > I think it can be applied to qsv too.
> > > > > >
> > > > >
> > > > > I agree. Part of ffmpeg help:
> > > > >     -hwaccel hwaccel name  use HW accelerated decoding
> > > > >     -hwaccel_device devicename  select a device for HW acceleration
> > > > >
> > > > > To me specifying device via -hwaccel_device in the way it is being
> > > > > suggest in commit is pretty intuitive and is better than explain
> > > > > people that there is - qsv_device for that purpose. Niceness of
> > > > > -hwaccel_device is that it is not tight to particular acceleration,
> > > > > hence easier to work with for end users.
> > > >
> > > > In case of QSV, the usage of hwaccel_device is already defined:
> > > >
> > > > device selects a value in ‘MFX_IMPL_*’. Allowed values are:
> > > > auto sw  hw  auto_any hw_any hw2 hw3 hw4
> > > >
> > > > I agree that it would be more intuitive, but you can't just give it a
> > > > different meaning from one day to another and break functionality.
> > >
> > > Adding new values does not break functionality! No one  suggests to drop
> > > existing set of values (sw, hw, etc.) - these are remaining. But basically
> > > suggestion is to extend allowed set to accept drm device specification on
> > > Linux, that's it. Honestly, I don't see problem here.
> > > Some background story: hw,hw2,hw3 allows to select adapter for msdk
> > > library on Windows, but they don't allow to do that on Linux. On Linux all
> > > these hw options are basically the same for the reason that device is being
> > > specified externally via SetHandle call. On Linux only sw/hw makes a
> > > difference (other story that sw msdk library is not actually released
> > > currently
> > > on Linux). That's the origin of the request: we need a good way to specify
> > > adapter on Linux.
> >
> > As mentioned, there are cases where both needs to be specified on Windows:
> > hw, hw2, hw3,.. plus adapter number.
> >
> > 'qsv_device' is for specifying adapter number on Windows and DRI on Linux
> > 'hwaccel_device' is for specifying hw, hw2, hw3 (everywhere)
>
> According to https://trac.ffmpeg.org/ticket/7649, -qsv_device was added to
> workaround device selection on Linux.
>
> On Windows, we should not use another option to specify the adapter number, the
> adapter is 1:1 mapped to the hw impl. I think we should fix it if the current
> QSV depends on -qsv_device on Window.
>
> https://github.com/Intel-Media-SDK/MediaSDK/blob/master/samples/sample_decode/src/pipeline_decode.cpp#L245-L264
>
> >
> > I see no reason to start mixing that and allowing to specify DRI via
> > 'hwaccel_device' parameter.
> >
> > That would make things just worse instead of improving anything.
>
> From user's perspective, we may use the same device selection for both QSV and
> VAAPI on Linux. It is easy for user to move from QSV to VAAPI or inverse.
>
Which case want to mix the  QSV and VAAPI in one FFmpeg pipeline? Do
you have a real case for this?

I know we used  VAAPI as decoder and QSV as encoder before when QSV
decoder have some limitation in FFmpeg

But I think we are better to fix QSV decoder in FFmpeg part or MSDK,
do you have any other case to mix VAAPI and QSV?