[FFmpeg-devel] [PATCH v7 1/8] libavcodec/qsv: enabling d3d11va support, added mfxhdlpair

Fri Jul 16 04:25:19 EEST 2021

On Thu, 2021-07-15 at 09:32 +0000, Soft Works wrote:
> > -----Original Message-----
> > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > Xiang, Haihao
> > Sent: Thursday, 15 July 2021 09:56
> > To: ffmpeg-devel at ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v7 1/8] libavcodec/qsv: enabling
> > d3d11va support, added mfxhdlpair
> > 
> > On Thu, 2021-07-15 at 06:43 +0000, Soft Works wrote:
> > > > -----Original Message-----
> > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> > > > Xiang, Haihao
> > > > Sent: Thursday, 15 July 2021 07:43
> > > > To: ffmpeg-devel at ffmpeg.org
> > > > Subject: Re: [FFmpeg-devel] [PATCH v7 1/8] libavcodec/qsv: enabling
> > > > d3d11va support, added mfxhdlpair
> > > > 
> > > > On Thu, 2021-07-15 at 03:49 +0000, Soft Works wrote:
> > > > > > -----Original Message-----
> > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On
> > 
> > Behalf
> > > > > > Of James Almer
> > > > > > Sent: Thursday, 15 July 2021 05:21
> > > > > > To: ffmpeg-devel at ffmpeg.org
> > > > > > Subject: Re: [FFmpeg-devel] [PATCH v7 1/8] libavcodec/qsv:
> > > > > > enabling d3d11va support, added mfxhdlpair
> > > > > > 
> > > > > > On 7/15/2021 12:10 AM, Soft Works wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > > -----Original Message-----
> > > > > > > > From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On
> > > > 
> > > > Behalf
> > > > > > > > Of Xiang, Haihao
> > > > > > > > Sent: Thursday, 15 July 2021 04:35
> > > > > > > > To: ffmpeg-devel at ffmpeg.org
> > > > > > > > Subject: Re: [FFmpeg-devel] [PATCH v7 1/8] libavcodec/qsv:
> > > > > > > > enabling d3d11va support, added mfxhdlpair
> > > > > > > > 
> > > > > > > > On Tue, 2021-07-13 at 11:53 -0300, James Almer wrote:
> > > > > > > > > On 11/14/2020 1:49 PM, Max Dmitrichenko wrote:
> > > > > > > > > > On Tue, Nov 3, 2020 at 7:47 PM Artem Galin
> > > > > > > > > > <artem.galin at gmail.com>
> > > > > > > > 
> > > > > > > > wrote:
> > > > > > > > > > 
> > > > > > > > > > > Adding DX11 relevant device type checks and adjusting
> > > > > > > > > > > callbacks with proper MediaSDK pair type support.
> > > > > > > > > > > 
> > > > > > > > > > > Extending structure for proper MediaSDK pair type support.
> > > > > > > > > > > 
> > > > > > > > > > > Signed-off-by: Artem Galin <artem.galin at intel.com> .....
> > > > > > > > > > 
> > > > > > > > > > 
> > > > > > > > > > Patchset obviously closes the gap of DirectX 11 support
> > > > > > > > > > and already checked with several apps that use FFMPEG.
> > > > > > > > > > 
> > > > > > > > > > Any particular review comments should be yet expected?
> > > > > > > > > > 
> > > > > > > > > > Changes seem to be straight forward and incorporate prev.
> > > > 
> > > > comments.
> > > > > > > > > > 
> > > > > > > > > > thank in advance
> > > > > > > > > > 
> > > > > > > > > > regards
> > > > > > > > > > Max
> > > > > > > > > 
> > > > > > > > > There are some issues pointed out by Soft Works related to
> > > > > > > > > switching the default to DX11 breaking existing command
> > > > > > > > > lines with DX9, plus an OpenCL<->QSV interop regression
> > > > > > > > > this would introduce that according to him should be easy to
> > > > > > > > > fix.
> > > > > > > > > 
> > > > > > > > > If those are addressed, the set should be good.
> > > > > > > > 
> > > > > > > > If we may apply
> > > > > > > > http://ffmpeg.org/pipermail/ffmpeg-devel/2021-
> > > > > > > > June/281778.html
> > > > > > > > (qsvdec: add support for HW_DEVICE_CTX method) first, we
> > > > > > > > should be able to use the common device stuff to initialize
> > > > > > > > d3d11va device for QSV and needn't use child_device_type to
> > > > > > > > specify child
> > > > 
> > > > device.
> > > > > > > 
> > > > > > > There's no doubt that the new method will be better, but the
> > > > > > > point is not
> > > > > > 
> > > > > > to break existing command lines. From my experience, ffmpeg
> > > > > > usually avoids to break established command line patterns, which
> > > > > > is a good thing
> > > > 
> > > > imo.
> > > > > > > 
> > > > > > > Anything more official than my perception? Please feel free to
> > > > > > > chime in... ;-)
> > > > > > 
> > > > > > That patch appears to attempt to maintain support for existing
> > > > > > command lines for a deprecation period, something we've done
> > > > > > before with other hwaccels like cuvid, so if it's really better,
> > > > > > it's fine and
> > > > 
> > > > acceptable.
> > > > > 
> > > > > ^
> > > > > 
> > > > > Hi James,
> > > > > 
> > > > > "That patch" that preserves compatibility doesn't exist yet. We
> > > > > need to make sure that command lines using the '--child_device'
> > > > > parameter will continue to work, while Haihao's patch will be the
> > 
> > "future way":
> > > > > 
> > > > > > -init_hw_device d3d11va=d3d11va:xxx -init_hw_device
> > 
> > qsv at d3d11va
> > > > > 
> > > > > The new behaviour that is introduced by this patch is that an
> > > > > initialized hw device can now be applied to qsv decoders
> > > > > (currently only to filter graphs and qsv encoders).
> > > > 
> > > > '-init_hw_device qsv=qsv:hw,child_device=xxx' still works,
> > > 
> > > Sure - that needs to work anyway. What I meant is the -qsv_device
> > > parameter, which is a global-scope param and takes a device path on
> > > Linux or the D3D adapter ID on Windows.
> > > 
> > > ffmpeg -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v h264_qsv -i
> > > input.mp4 -c:v h264_qsv output.mp4
> > 
> > Yes, -qsv_device still works, I added opt_qsv_device() to handle qsv_device
> > option.
> > 
> > > 
> > > For my implementation I had added an additional global parameter (
> > > --qsv_use_dx11). When it is set, the -qsv_device parameter indicates a
> > > DXGI adapter id rather than a D3D adapter. But let's forget about
> > > that. The init_hw_device approach is much better, and -qsv_device
> > > should just keep working like before.
> > > 
> > > I'm currently rebasing all our patches to the latest ffmpeg; probably
> > > tomorrow I'll check out your patch.
> > 
> > Thanks for checking out the patch.
> > 
> > > 
> > > > > Additionally, the default should remain to be D3D9 (if none of the
> > > > > above is
> > > > > specified) as laid out earlier - again for maintaining
> > > > > compatibility but also for better robustness and performance.
> > > > 
> > > > We may refine Artem's patch to make sure the default is D3D9 on
> > > > Microsoft windows
> > > 
> > > Sounds good.
> > > 
> > > > when both dxva2 and d3d11va are enabled if applying my patch first.
> > > 
> > > Not sure what you mean by that.
> > 
> > What I meant is we needn't use parameter child_device_type (which was
> > added in Artem's patch) to specify the child device type after applying my
> > patch, but we should make sure the default child device type is DXVA2 when
> > both DXVA2 and D3D11VA are available however user doesn't specify the
> > type. E.g. '- init_hw_device qsv=qsv:hw' should work with DXVA2, not
> > D3D11VA if both DXVA2 and D3D11VA are availabe.
> 
> Ah, thanks for the clarification, but that doesn't work out. The parameter -
> child_device_type is essential:
> 
> 1. The device numbers that you specify via qsv_device or child_device are
> totally different between D3D9 and D3D11/DXGI. In the latter case, you specify
> the index of a graphics adapter/hardware device, no matter how many displays
> you _can_ connect to it and no matter how many displays _are_ connected.
> In case of D3D9, you specify the index in a collection of connected
> displays  which belong to the context of the active user session.
> => without specifying a device type, an index number is meaningless
> 
> 2. It's important that the behavior is deterministic: when you specify D3D9,
> you need to be able to rely on getting D3D9 (or fail otherwise), same for
> D3D11. 
> 
> 3. When you want to combine a DXVA2/D3D11VA decoder with a qsv encoder or
> filter context, you need to make a choice between these two, which also means
> that you need to be able to control D3D9/11 at the QSV side.
> 
> 4. OpenCL interop support is different between D3D9 and D3D11 and you might
> need to adjust your command line accordingly. That's only possible when you
> can be sure up-front about which kind of context you'll get.
> 
> 5. Drivers for Gen 3, 4 and 5 are no longer getting updates for QSV. There are
> still driver updates being published, but the QSV/MSDK part remains unchanged.
> These versions have issues that Artem's patch doesn't address
> 
> Summing up: a selection mechanism between D3D9 and D3D11 is  inevitable, but
> it must not be automatic.
> Of course - it would be possible to create a DXVA2 or D3D11VA context first
> and derive from that one, but that's technically not the same and too
> complicated for the average user.

Thanks for explanation, I'm fine to keep child_device_type (my original thought
was to ask user to derive a QSV device from D3D11VA for D3D11VA support). 

BR
Haihao
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".