[FFmpeg-devel] [PATCH 1/2] avcodec: Add metadata to identify hardware backed codecs

[wm4]

On Mon, 27 Nov 2017 20:46:00 -0800
Philip Langdale <philipl at overt.org> wrote:

> If hardware acceleration is implemented through an HWAccel, then it's
> easy to recognise, but some hardware implementations are best suited
> to implementation as full decoders, and these are not easy to identify.
> Clients typically need hardcoded lists of codecs, and usually need to
> rely on related codecs using a particular naming convention.
> 
> To make it easier to discover these codecs dynamically, we can
> introduce an explicit AV_CODEC_CAP_HARDWARE that indicates the codec
> is hardware-backed. We also introduce a 'provider' field into AVCodec
> that allows for an identifier to be attached to these codecs for
> simple matching. The provider could also be used to identify the
> use of an external software library.
> 
> Unfortunately, we know of at least one case where an external interface
> might silently end up using a separate software implementation (libmfx)
> and so we introduce AV_CODEC_CAP_MAYBE_NOT_HARDWARE so say that you
> can't trust it 100%. It's silly but what can you do?
> 
> The expected use-case here is allowing a client application to
> enumerate hardware decoding options for a given format and to allow
> a user to select one based on a recognisable identifier (the provider).
> 
> Signed-off-by: Philip Langdale <philipl at overt.org>
> ---
>  Changelog            |  1 +
>  doc/APIchanges       |  5 +++++
>  libavcodec/avcodec.h | 19 +++++++++++++++++++
>  3 files changed, 25 insertions(+)
> 
> diff --git a/Changelog b/Changelog
> index 76d6fad56b..a99edbf4b7 100644
> --- a/Changelog
> +++ b/Changelog
> @@ -21,6 +21,7 @@ version <next>:
>  - video mix filter
>  - video normalize filter
>  - audio lv2 wrapper filter
> +- Added codec metadata to identify hardware backed decoders
>  
>  
>  version 3.4:
> diff --git a/doc/APIchanges b/doc/APIchanges
> index 44a740e51f..7856cb13a1 100644
> --- a/doc/APIchanges
> +++ b/doc/APIchanges
> @@ -15,6 +15,11 @@ libavutil:     2017-10-21
>  
>  API changes, most recent first:
>  
> +2017-12-xx - xxxxxxx - lavc 58.7.100 - avcodec.h
> +  Add AV_CODEC_CAP_HARDWARE
> +  Add AV_CODEC_CAP_MAYBE_NOT_HARDWARE
> +  Add AVCodec.provider
> +
>  2017-xx-xx - xxxxxxx - lavc 58.6.100 - avcodec.h
>    Add const to AVCodecContext.hwaccel.
>  
> diff --git a/libavcodec/avcodec.h b/libavcodec/avcodec.h
> index c1e68b1d13..04cfa48665 100644
> --- a/libavcodec/avcodec.h
> +++ b/libavcodec/avcodec.h
> @@ -1036,6 +1036,16 @@ typedef struct RcOverride{
>   * choice for probing.
>   */
>  #define AV_CODEC_CAP_AVOID_PROBING       (1 << 17)
> +/**
> + * Codec is backed by a hardware implementation. Typically used to
> + * identify a non-hwaccel hardware decoder.
> + */
> +#define AV_CODEC_CAP_HARDWARE            (1 << 18)
> +/**
> + * Codec is normally backed by a hardware implementation, but the
> + * external abstraction could be software backed.
> + */
> +#define AV_CODEC_CAP_MAYBE_NOT_HARDWARE  (1 << 19)

One could argue that every hw-backed decoder might fallback to software
fully or partially. For example, Intel is known to have hybrid drivers
(that sometimes even tend to be slower than libavcodec), and
videotoolbox has a software decoder (though we disable it). So I'm not
sure if this flag is really necessary, but I'm not opposed either.

>  /**
>   * Codec is intra only.
>   */
> @@ -3475,6 +3485,15 @@ typedef struct AVCodec {
>       * The user can only access this field via avcodec_get_hw_config().
>       */
>      const struct AVCodecHWConfigInternal **hw_configs;
> +
> +    /**
> +     * String that helps identify the external provider for the codec.
> +     * For example, if some external hardware decoder provides support
> +     * for a variety of codecs, they could each set the same provider
> +     * value to signify their relationship. Could also be used to
> +     * identify the use of an external software implementation.
> +     */
> +   const char *provider;
>  } AVCodec;
>  
>  #if FF_API_CODEC_GET_SET

Generally LGTM. It's pretty useless if it doesn't land in libav,
though. Can you post it there too?