[FFmpeg-devel] [PATCH] Vulkan hwcontext and filters

Fri Jan 10 23:05:21 EET 2020

Patches attached
Also pushed to https://github.com/cyanreg/FFmpeg/ master branch because they're 9 and they add about 7000 lines.
Filtering won't work without a recent glslang version since they moved a header and broke API because they felt like it.

Git log:

commit aa9f0ea2cf210234ed26df349d3a1562b7de1110
Author: Philip Langdale <philipl at overt.org>
Date:   Tue Dec 31 09:41:57 2019 -0800

    lavu/hwcontext_cuda: refactor context initialisation

    There's enough going on here now that it should not be duplicated
    between cuda_device_create and cuda_device_derive.

commit bb734edb10b6f1853e1f2f5735b7ffd0a1d48468
Author: Lynne <dev at lynne.ee>
Date:   Sun Oct 27 14:48:16 2019 +0000

    lavfi: add an chromaber_vulkan filter

    This commit adds a chromatic aberration filter for Vulkan that attempts to
    emulate a lens chromatic aberration effect.
    For a YUV frame it will instead shift the chroma channels, providing a
    simple approximation.

commit 1e3a50fbe4399f76c0ab0f62bf6d6c65b8565db4
Author: Lynne <dev at lynne.ee>
Date:   Sun Oct 27 14:47:18 2019 +0000

    lavfi: add an avgblur_vulkan filter

    This commit adds a fast avgblur Vulkan filter.
    This will reset Intel GPUs on Windows due to a known, year-old driver bug.

commit f4c77d10e5e2c37ec1bf305773ec94898b99a5e5
Author: Lynne <dev at lynne.ee>
Date:   Sun Oct 27 14:46:16 2019 +0000

    lavfi: add an overlay_vulkan filter

    This commit adds a basic, non-converting overlay filter for Vulkan.

commit 0badbf31effc16cf8f0be86f1de4fbdd029cebe4
Author: Lynne <dev at lynne.ee>
Date:   Sun Oct 27 14:45:36 2019 +0000

    lavfi: add an scale_vulkan filter

    This commit adds a basic, non-converting Vulkan scaling filter.

commit 04c1836f89d89dcdc892cef66ee82afbcfda9f2d
Author: Lynne <dev at lynne.ee>
Date:   Sun Oct 27 14:44:00 2019 +0000

    lavfi: add Vulkan filtering framework

    This commit adds a Vulkan filtering infrastructure for libavfilter.
    It attempts to abstract as much as possible of the Vulkan API from filters.

    The way the hwcontext and the framework are designed permits for parallel,
    non-CPU-blocking filtering throughout, with the exception of up/downloading
    and mapping.

commit e2d18e03e3a5fa8ef82159c68212b720198a9b91
Author: Philip Langdale <philipl at overt.org>
Date:   Wed Oct 23 18:11:37 2019 -0700

    lavfi/vf_hwupload: Add support for HW -> HW transfers

    As we find ourselves wanting a way to transfer frames between
    HW devices (or more realistically, between APIs on the same device),
    it's desirable to have a way to describe the relationship. While
    we could imagine introducing a `hwtransfer` filter, there is
    almost no difference from `hwupload`. The main new feature we need
    is a way to specify the target device. Having a single device
    for the filter chain is obviously insufficient if we're dealing
    with two devices.

    So let's add a way to specify the upload target device, and if none
    is specified, continue with the existing behaviour.

    We must also correctly preserve the sw_format on such a transfer.

commit d5f1bbc61fab452803443511b1241931169359b7
Author: Lynne <dev at lynne.ee>
Date:   Wed Aug 28 21:58:10 2019 +0100

    lavu: add Vulkan hwcontext code

    This commit adds the necessary code to initialize and use a Vulkan device
    within the hwcontext libavutil framework.
    Currently direct mapping to VAAPI and DRM frames is functional, and
    transfers to CUDA and native frames are supported.

    Lets hope the future Vulkan video decode extension fits well within this
    framework.

commit 2fefb0b7ff760f2fb019751da8c37cfd0578ef00
Author: Philip Langdale <philipl at overt.org>
Date:   Wed Oct 23 18:01:52 2019 -0700

    lavu/hwcontext: Add support for HW -> HW transfers

    We are beginning to consider scenarios where a given HW Context
    may be able to transfer frames to another HW Context without
    passing via system memory - this would usually be when two
    contexts represent different APIs on the same device (eg: Vulkan
    and CUDA).

    This is modelled as a transfer, as we have today, but where both
    the src and the dst are hardware frames with hw contexts. We need
    to be careful to ensure the contexts are compatible - particularly,
    we cannot do transfers where one of the frames has been mapped via
    a derived frames context - we can only do transfers for frames that
    were directly allocated by the specified context.

    Additionally, as we have two hardware contexts, the transfer function
    could be implemented by either (or indeed both). To handle this
    uncertainty, we explicitly look for ENOSYS as an indicator to try
    the transfer in the other direction before giving up.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-lavu-hwcontext-Add-support-for-HW-HW-transfers.patch
Type: text/x-patch
Size: 3920 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200110/1a59264d/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-lavu-add-Vulkan-hwcontext-code.patch
Type: text/x-patch
Size: 118979 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200110/1a59264d/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0003-lavfi-vf_hwupload-Add-support-for-HW-HW-transfers.patch
Type: text/x-patch
Size: 6450 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200110/1a59264d/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0004-lavfi-add-Vulkan-filtering-framework.patch
Type: text/x-patch
Size: 67408 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200110/1a59264d/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0005-lavfi-add-an-scale_vulkan-filter.patch
Type: text/x-patch
Size: 14852 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200110/1a59264d/attachment-0004.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0006-lavfi-add-an-overlay_vulkan-filter.patch
Type: text/x-patch
Size: 19675 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200110/1a59264d/attachment-0005.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0007-lavfi-add-an-avgblur_vulkan-filter.patch
Type: text/x-patch
Size: 19108 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200110/1a59264d/attachment-0006.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0008-lavfi-add-an-chromaber_vulkan-filter.patch
Type: text/x-patch
Size: 16377 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200110/1a59264d/attachment-0007.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0009-lavu-hwcontext_cuda-refactor-context-initialisation.patch
Type: text/x-patch
Size: 6160 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200110/1a59264d/attachment-0008.bin>