[FFmpeg-devel] [PATCH] ffmpeg_qsv: use -hwaccel_device to specify a device for VAAPI backend

Thu Sep 3 10:13:58 EEST 2020

On Thu, 2020-09-03 at 12:59 +0800, mypopy at gmail.com wrote:
> On Thu, Sep 3, 2020 at 9:21 AM Xiang, Haihao <haihao.xiang at intel.com>
> wrote:
> > 
> > On Wed, 2020-09-02 at 14:04 +0000, Soft Works wrote:
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf
> > > > Of
> > > > Rogozhkin, Dmitry V
> > > > Sent: Wednesday, September 2, 2020 3:54 PM
> > > > To: ffmpeg-devel at ffmpeg.org
> > > > Subject: Re: [FFmpeg-devel] [PATCH] ffmpeg_qsv: use
> > > > -hwaccel_device to
> > > > specify a device for VAAPI backend
> > > > 
> > > > On Wed, 2020-09-02 at 07:31 +0000, Soft Works wrote:
> > > > > > -----Original Message-----
> > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On
> > > > > > Behalf Of
> > > > > > Rogozhkin, Dmitry V
> > > > > > Sent: Wednesday, September 2, 2020 9:00 AM
> > > > > > To: ffmpeg-devel at ffmpeg.org
> > > > > > Subject: Re: [FFmpeg-devel] [PATCH] ffmpeg_qsv: use
> > > > > > -hwaccel_device
> > > > > > to specify a device for VAAPI backend
> > > > > > 
> > > > > > On Wed, 2020-09-02 at 04:28 +0000, Xiang, Haihao wrote:
> > > > > > > On Wed, 2020-09-02 at 04:20 +0000, Soft Works wrote:
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org>
> > > > > > > > > On
> > > > 
> > > > Behalf
> > > > > > 
> > > > > > Of
> > > > > > > > > Haihao Xiang
> > > > > > > > > Sent: Wednesday, September 2, 2020 6:01 AM
> > > > > > > > > To: ffmpeg-devel at ffmpeg.org
> > > > > > > > > Cc: Haihao Xiang <haihao.xiang at intel.com>
> > > > > > > > > Subject: [FFmpeg-devel] [PATCH] ffmpeg_qsv: use
> > > > > > > > > -hwaccel_device to specify a device for VAAPI backend
> > > > > > > > > 
> > > > > > > > > According to the FFmpeg documentation,
> > > > > > > > > -hwaccel_device can be
> > > > > > > > > used to select a device for hardware acceleration
> > > > > > > > > 
> > > > > > > > > Example:
> > > > > > > > > ffmpeg -hwaccel qsv -hwaccel_device
> > > > > > > > > /dev/dri/renderD129 -c:v
> > > > > > > > > h264_qsv \ -i
> > > > > > > > > in.h264 -c:v hevc_qsv out.h265
> > > > > > > > 
> > > > > > > > No. The docs say about hwaccel_device
> > > > > > > > 
> > > > > > > > It can either refer to an existing device created with
> > > > > > > > -init_hw_device by name,
> > > > > > > > 
> > > > > > > > => /dev/dri/renderD129 is not an existing device name
> > > > > > > > 
> > > > > > > > or it can create a new device as if ‘-init_hw_device’
> > > > > > > > type:hwaccel_device were called immediately before.
> > > > > > > > 
> > > > > > > > => /dev/dri/renderD129 is not a valid string for
> > > > > > > > init_hw_device
> > > > > > 
> > > > > > Well, it seems that's where documentation does not reflect
> > > > > > the
> > > > > > reality since (as pointed by Haihao below)
> > > > > > https://trac.ffmpeg.org/wiki/Hardware/VAAPI
> > > > > > is full of contradictory examples.
> > > > > > 
> > > > > > > > 
> > > > > > > > 
> > > > > > > > Hence, the patch doesn't make sense.
> > > > > > > 
> > > > > > > in https://trac.ffmpeg.org/wiki/Hardware/VAAPI,
> > > > > > > 
> > > > > > > If only one stream is being used, -hwaccel_device can
> > > > > > > also accept
> > > > > > > a device path directly
> > > > > > > 
> > > > > > > ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
> > > > > > > -i ...
> > > > > > > 
> > > > > > > I think it can be applied to qsv too.
> > > > > > > 
> > > > > > 
> > > > > > I agree. Part of ffmpeg help:
> > > > > >     -hwaccel hwaccel name  use HW accelerated decoding
> > > > > >     -hwaccel_device devicename  select a device for HW
> > > > > > acceleration
> > > > > > 
> > > > > > To me specifying device via -hwaccel_device in the way it
> > > > > > is being
> > > > > > suggest in commit is pretty intuitive and is better than
> > > > > > explain
> > > > > > people that there is - qsv_device for that purpose.
> > > > > > Niceness of
> > > > > > -hwaccel_device is that it is not tight to particular
> > > > > > acceleration,
> > > > > > hence easier to work with for end users.
> > > > > 
> > > > > In case of QSV, the usage of hwaccel_device is already
> > > > > defined:
> > > > > 
> > > > > device selects a value in ‘MFX_IMPL_*’. Allowed values are:
> > > > > auto sw  hw  auto_any hw_any hw2 hw3 hw4
> > > > > 
> > > > > I agree that it would be more intuitive, but you can't just
> > > > > give it a
> > > > > different meaning from one day to another and break
> > > > > functionality.
> > > > 
> > > > Adding new values does not break functionality! No
> > > > one  suggests to drop
> > > > existing set of values (sw, hw, etc.) - these are remaining.
> > > > But basically
> > > > suggestion is to extend allowed set to accept drm device
> > > > specification on
> > > > Linux, that's it. Honestly, I don't see problem here.
> > > > Some background story: hw,hw2,hw3 allows to select adapter for
> > > > msdk
> > > > library on Windows, but they don't allow to do that on Linux.
> > > > On Linux all
> > > > these hw options are basically the same for the reason that
> > > > device is being
> > > > specified externally via SetHandle call. On Linux only sw/hw
> > > > makes a
> > > > difference (other story that sw msdk library is not actually
> > > > released
> > > > currently
> > > > on Linux). That's the origin of the request: we need a good way
> > > > to specify
> > > > adapter on Linux.
> > > 
> > > As mentioned, there are cases where both needs to be specified on
> > > Windows:
> > > hw, hw2, hw3,.. plus adapter number.
> > > 
> > > 'qsv_device' is for specifying adapter number on Windows and DRI
> > > on Linux
> > > 'hwaccel_device' is for specifying hw, hw2, hw3 (everywhere)
> > 
> > According to https://trac.ffmpeg.org/ticket/7649, -qsv_device was
> > added to
> > workaround device selection on Linux.
> > 
> > On Windows, we should not use another option to specify the adapter
> > number, the
> > adapter is 1:1 mapped to the hw impl. I think we should fix it if
> > the current
> > QSV depends on -qsv_device on Window.
> > 
> > https://github.com/Intel-Media-SDK/MediaSDK/blob/master/samples/sample_decode/src/pipeline_decode.cpp#L245-L264
> > 
> > > 
> > > I see no reason to start mixing that and allowing to specify DRI
> > > via
> > > 'hwaccel_device' parameter.
> > > 
> > > That would make things just worse instead of improving anything.
> > 
> > From user's perspective, we may use the same device selection for
> > both QSV and
> > VAAPI on Linux. It is easy for user to move from QSV to VAAPI or
> > inverse.
> > 
> 
> Which case want to mix the  QSV and VAAPI in one FFmpeg pipeline? Do
> you have a real case for this?
> 
> I know we used  VAAPI as decoder and QSV as encoder before when QSV
> decoder have some limitation in FFmpeg
> 
> But I think we are better to fix QSV decoder in FFmpeg part or MSDK,
> do you have any other case to mix VAAPI and QSV?

MSDK on Linux requires VAAPI device (same as on Windows it requires DX9
or DX11). For this reason VAAPI device can be used to initialized QSV
device in ffmpeg. So, this is not necessarily mixing QSV and VAAPI
components in the pipeline (though should be possible, not talking why
though), but just a way to initialize QSV device...

> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".