[FFmpeg-devel] [RFC] amfenc: Add support for OpenCL input

[Mark Thompson]

On 23/01/18 15:31, Mironov, Mikhail wrote:
>> -----Original Message-----
>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf
>> Of Mark Thompson
>> Sent: January 23, 2018 10:21 AM
>> To: ffmpeg-devel at ffmpeg.org
>> Subject: Re: [FFmpeg-devel] [RFC] amfenc: Add support for OpenCL input
>>
>> On 23/01/18 15:04, Mironov, Mikhail wrote:
>>>> -----Original Message-----
>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
>> Behalf
>>>> Of Mark Thompson
>>>> Sent: January 22, 2018 6:57 PM
>>>> To: FFmpeg development discussions and patches <ffmpeg-
>>>> devel at ffmpeg.org>
>>>> Subject: [FFmpeg-devel] [RFC] amfenc: Add support for OpenCL input
>>>>
>>>> ---
>>>> This allows passing OpenCL frames to AMF without a download/upload
>>>> step to get around AMD's lack of support for D3D11 mapping.
>>>>
>>>> For example:
>>>>
>>>> ./ffmpeg -hwaccel dxva2 -hwaccel_output_format dxva2_vld -i input.mp4
>>>> -an -vf
>>>>
>> 'hwmap=derive_device=opencl,program_opencl=source=examples.cl:kernel=
>>>> rotate_image' -c:v h264_amf output.mp4
>>>>
>>>> * I can't find any documentation or examples for these functions, so
>>>> I'm guessing a bit exactly how they are meant to work.  In
>>>> particular, there are some locking functions which I have ignored
>>>> because I have no idea under what circumstances something might want
>> to be locked.
>>>> * I tried to write common parts with D3D11, but I might well have
>>>> broken
>>>> D3D11 support in the process - it doesn't work at all for me so I can't test
>> it.
>>>> * Not sure how to get non-NV12 to work.  I may be missing something,
>>>> or it may just not be there - the trace messages suggest it doesn't
>>>> like the width of
>>>> RGB0 or the second plane of GRAY8.
>>>>
>>>> - Mark
>>>>
>>>>
>>>>  libavcodec/amfenc.c | 178
>>>> +++++++++++++++++++++++++++++++++++---------
>>>> --------
>>>>  libavcodec/amfenc.h |   1 +
>>>>  2 files changed, 123 insertions(+), 56 deletions(-)
>>>>
>>>> ...
>>>
>>> AMF encoder works via D3D9 or D3D11 only. AMF OpenCL support is done
>>> for possible integration with external image processing. Passing regular
>> OpenCL 2D images will cause mapping to system memory and copy.
>>> The fast way is to use interop:
>>> - Allocate last processing NV12 surface as D3D11 texture
>>> - iterop it into OpenCL
>>
>> This step doesn't work - AMD has no support for mapping NV12 D3D11
>> textures to OpenCL.  (That's why I wrote the patch at all.)
> 
> It does via OCL D3D11 Media extension. I use it inside AMF. 
> But for fast results the surface must be allocated with shared handle.

Where can I find details of that extension?  Currently we use the standard cl_khr_d3d11_sharing extension, which combines with Intel's cl_intel_d3d11_nv12_media_sharing for NV12 support (see code in hwcontext_opencl.c).

Allocating with shared handle (MISC_SHARED flags) is fine, Intel has that as a hard requirement and fails without it.

>>
>> Does D3D9 also work?  That does have the necessary support, but would
>> admittedly end up with the same multiple-mapping ugliness as does D3D9 ->
>> OpenCL -> D3D9 -> libmfx on Intel.
> 
> D3D9 also works more or less the same as D3D11.

Ok, thank you.

- Mark