[FFmpeg-devel] [PATCH] Vulkan hwcontext and filters

[Mark Thompson]

On 10/01/2020 21:05, Lynne wrote:
> From 2fefb0b7ff760f2fb019751da8c37cfd0578ef00 Mon Sep 17 00:00:00 2001
> From: Philip Langdale <philipl at overt.org>
> Date: Wed, 23 Oct 2019 18:01:52 -0700
> Subject: [PATCH 1/9] lavu/hwcontext: Add support for HW -> HW transfers
> 
> We are beginning to consider scenarios where a given HW Context
> may be able to transfer frames to another HW Context without
> passing via system memory - this would usually be when two
> contexts represent different APIs on the same device (eg: Vulkan
> and CUDA).
> 
> This is modelled as a transfer, as we have today, but where both
> the src and the dst are hardware frames with hw contexts. We need
> to be careful to ensure the contexts are compatible - particularly,
> we cannot do transfers where one of the frames has been mapped via
> a derived frames context - we can only do transfers for frames that
> were directly allocated by the specified context.

Why?  A derived frames context should be mostly treatable as a normal frames context in the mapped-to device, so I'm not seeing where this restriction is coming from.

(Is there some particular case which doesn't work that this helps to avoid?)

> 
> Additionally, as we have two hardware contexts, the transfer function
> could be implemented by either (or indeed both). To handle this
> uncertainty, we explicitly look for ENOSYS as an indicator to try
> the transfer in the other direction before giving up.
> ---
>  libavutil/hwcontext.c | 53 +++++++++++++++++++++++++++++++++++--------
>  1 file changed, 43 insertions(+), 10 deletions(-)
> 
> diff --git a/libavutil/hwcontext.c b/libavutil/hwcontext.c
> index f1e404ab20..3189391c07 100644
> --- a/libavutil/hwcontext.c
> +++ b/libavutil/hwcontext.c
> @@ -444,21 +444,54 @@ int av_hwframe_transfer_data(AVFrame *dst, const AVFrame *src, int flags)
>      if (!dst->buf[0])
>          return transfer_data_alloc(dst, src, flags);
>  
> -    if (src->hw_frames_ctx) {
> -        ctx = (AVHWFramesContext*)src->hw_frames_ctx->data;
> +    /*
> +     * Hardware -> Hardware Transfer.
> +     * Unlike Software -> Hardware or Hardware -> Software, the transfer
> +     * function could be provided by either the src or dst, depending on
> +     * the specific combination of hardware.
> +     */
> +    if (src->hw_frames_ctx && dst->hw_frames_ctx) {
> +        AVHWFramesContext *src_ctx =
> +            (AVHWFramesContext*)src->hw_frames_ctx->data;
> +        AVHWFramesContext *dst_ctx =
> +            (AVHWFramesContext*)dst->hw_frames_ctx->data;
> +
> +        if (src_ctx->internal->source_frames) {
> +            av_log(src_ctx, AV_LOG_ERROR,
> +                   "A device with a derived frame context cannot be used as "
> +                   "the source of a HW -> HW transfer.");
> +            return AVERROR(ENOSYS);
> +        }
>  
> -        ret = ctx->internal->hw_type->transfer_data_from(ctx, dst, src);
> -        if (ret < 0)
> -            return ret;
> -    } else if (dst->hw_frames_ctx) {
> -        ctx = (AVHWFramesContext*)dst->hw_frames_ctx->data;
> +        if (dst_ctx->internal->source_frames) {
> +            av_log(src_ctx, AV_LOG_ERROR,
> +                   "A device with a derived frame context cannot be used as "
> +                   "the destination of a HW -> HW transfer.");
> +            return AVERROR(ENOSYS);
> +        }
>  
> -        ret = ctx->internal->hw_type->transfer_data_to(ctx, dst, src);
> +        ret = src_ctx->internal->hw_type->transfer_data_from(src_ctx, dst, src);
> +        if (ret == AVERROR(ENOSYS))
> +            ret = dst_ctx->internal->hw_type->transfer_data_to(dst_ctx, dst, src);
>          if (ret < 0)
>              return ret;
> -    } else
> -        return AVERROR(ENOSYS);
> +    } else {
> +        if (src->hw_frames_ctx) {
> +            ctx = (AVHWFramesContext*)src->hw_frames_ctx->data;
> +
> +            ret = ctx->internal->hw_type->transfer_data_from(ctx, dst, src);
> +            if (ret < 0)
> +                return ret;
> +        } else if (dst->hw_frames_ctx) {
> +            ctx = (AVHWFramesContext*)dst->hw_frames_ctx->data;
>  
> +            ret = ctx->internal->hw_type->transfer_data_to(ctx, dst, src);
> +            if (ret < 0)
> +                return ret;
> +        } else {
> +            return AVERROR(ENOSYS);
> +        }
> +    }
>      return 0;
>  }
>  
> -- 
> 2.25.0.rc2
> 

This looks fine with the intent of the extra condition clarified.

Thanks,

- Mark