[FFmpeg-devel] [PATCH] lavfi: add a libplacebo filter

[Lynne]

Mar 9, 2021, 23:36 by ffmpeg at haasn.xyz:

> On Tue, 09 Mar 2021 22:53:18 +0100 Lynne <dev at lynne.ee> wrote:
>
>> Sorry, as we discussed on IRC the Vulkan hwcontext should be
>> improved first and this filter should be kept as a pure hardware
>> filter.
>>
>
> I changed my mind on this because:
>
> - I think it's reasonably self-contained as is, and actually simpler to
>  implement this way.
>

It's simpler because it makes no effort to integrate with anything
we use.


> - There's no duplication of code within this patch.
>

There is, it's just hidden in libplacebo. If you integrate with the
framework code like in the previous PR, there's no duplication.
And no, we will not be removing the current Vulkan filters and
we intend to continue expanding them. Not even if you add similar
filters to libplacebo.


> - Sharing the VkImages directly is simply not a good approach. It
>  creates an unnecessary API hassle (as demonstrated by the multitude of
>  solved and unsolved issues surrounding device negotiation,
>  synchronization, etc.)
>

It's a good approach since API users work with VkImages. It's more
complex because it's explicit.


> - Therefore, it's better to share using an agreed-upon mechanism like
>  a shared memory object exported/imported using the Vulkan API.
>

There is a generic shared hardware memory export/import format,
it's called Vulkan, this is partly why it was created.


> - A good API for this already exists (dmabufs).
>

See above.


> - You can already derive a drm device from a vulkan device and export
>  vulkan hwframes as drmprime frames.
>

With an extension. Only supported on Linux. Only supported on recent
AMD cards and drivers. Which is also buggy and crashes drivers. And
the Intel implementation has been sitting as a PR for more than 3 years!
You may have the privilege of using it because you lucked out on
hardware, but the rest of us are not so lucky.


> I plan on adding support for drmprime frames to this filter, which I
> think is the cleaner way to avoid the upload/download. But I wanted to
> get feedback on the current version, since it's *functionally* complete
> as-is.
>

I won't accept adding drmprime filtering functionality either.
This is a pure hardware Vulkan filtering infrastructure. We keep nothing
hidden from users when it comes to filtering.


> If you're worried about dmabufs being a Linux-only API, I would still
> rather see an approach based on Vulkan memory object imports/exports
> rather than attempting to share and synchronize the VkImages directly.
>

Windows still accounts for a huge amount of our userbase. Even on
servers which do encoding. And so does Nvidia hardware.
In fact, Nvidia transcoding setups are the major users of hardware filtering.


> Even just the fact that we can't each enable extensions/features we want
> enabled independently is a reason that trying to share a single vulkan
> device is a bad idea. Right now the result of that is that vf_libplacebo
> atop hwcontext_vulkan would simply be missing out on features and
> performance, compared to this version.
>

That's plain wrong. We wouldn't be missing out on features.

I'm sorry, but I'm not going to change my mind on this. I'd really appreciate
your suggestions elsewhere, particularly in the are of making our libraries
interoperable.