[FFmpeg-devel] [PATCH v3 2/2] ffmpeg_opt: consider HW acceleration method when selecting decoder

[Xiang, Haihao]

On Mon, 2022-07-25 at 12:30 +0800, Xiang, Haihao wrote:
> From: Haihao Xiang <haihao.xiang at intel.com>
> 
> Usually a HW decoder is expected when user specifies a HW acceleration
> method via -hwaccel option, however the current implementation doesn't
> take HW acceleration method into account, it is possible to select a SW
> decoder.
> 
> For example:
> $ ffmpeg -hwaccel vaapi -i av1.mp4 -f null -
> $ ffmpeg -hwaccel nvdec -i av1.mp4 -f null -
> $ ffmpeg -hwaccel vdpau -i av1.mp4 -f null -
> [...]
> Stream #0:0 -> #0:0 (av1 (libdav1d) -> wrapped_avframe (native))
> 
> libdav1d is selected in this case even if vaapi, nvdec or vdpau is
> specified.
> 
> After applying this patch, the native av1 decoder (with vaapi, nvdec or
> vdpau support) is selected for decoding(libdav1d is still used for
> probing format).
> $ ffmpeg -hwaccel vaapi -i av1.mp4 -f null -
> $ ffmpeg -hwaccel nvdec -i av1.mp4 -f null -
> $ ffmpeg -hwaccel vdpau -i av1.mp4 -f null -
> [...]
> Stream #0:0 -> #0:0 (av1 (native) -> wrapped_avframe (native))
> 
> Tested-by: Mario Roy <marioeroy at gmail.com>
> Signed-off-by: Haihao Xiang <haihao.xiang at intel.com>
> ---
>  fftools/ffmpeg_opt.c | 44 +++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 43 insertions(+), 1 deletion(-)
> 
> diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
> index 2771c5d993..614bfeea80 100644
> --- a/fftools/ffmpeg_opt.c
> +++ b/fftools/ffmpeg_opt.c
> @@ -861,6 +861,48 @@ static const AVCodec *choose_decoder(OptionsContext *o,
> AVFormatContext *s, AVSt
>          return avcodec_find_decoder(st->codecpar->codec_id);
>  }
>  
> +static const AVCodec *choose_decoder2(InputStream *ist, OptionsContext *o,
> AVFormatContext *s, AVStream *st)
> +{
> +    char *codec_name = NULL;
> +
> +    MATCH_PER_STREAM_OPT(codec_names, str, codec_name, s, st);
> +    if (codec_name) {
> +        const AVCodec *codec = find_codec_or_die(codec_name, st->codecpar-
> >codec_type, 0);
> +        st->codecpar->codec_id = codec->id;
> +        if (recast_media && st->codecpar->codec_type != codec->type)
> +            st->codecpar->codec_type = codec->type;
> +        return codec;
> +    } else {
> +        if (st->codecpar->codec_type == AVMEDIA_TYPE_VIDEO &&
> +            ist->hwaccel_id == HWACCEL_GENERIC &&
> +            ist->hwaccel_device_type != AV_HWDEVICE_TYPE_NONE) {
> +            const AVCodec *p;
> +            void *i = 0;
> +
> +            while ((p = av_codec_iterate(&i))) {
> +                int j;
> +
> +                if (p->id != st->codecpar->codec_id ||
> +                    !av_codec_is_decoder(p) ||
> +                    !avcodec_get_hw_config(p, 0))
> +                    continue;
> +
> +                for (j = 0; ;j++) {
> +                    const AVCodecHWConfig *config = avcodec_get_hw_config(p,
> j);
> +
> +                    if (!config)
> +                        break;
> +
> +                    if (config->device_type == ist->hwaccel_device_type)
> +                        return p;
> +                }
> +            }
> +        }
> +
> +        return avcodec_find_decoder(st->codecpar->codec_id);
> +    }
> +}
> +
>  /* Add all the streams from the given input file to the global
>   * list of input streams. */
>  static void add_input_streams(OptionsContext *o, AVFormatContext *ic)
> @@ -973,7 +1015,7 @@ static void add_input_streams(OptionsContext *o,
> AVFormatContext *ic)
>              ist->hwaccel_pix_fmt = AV_PIX_FMT_NONE;
>          }
>  
> -        ist->dec = choose_decoder(o, ic, st);
> +        ist->dec = choose_decoder2(ist, o, ic, st);
>          ist->decoder_opts = filter_codec_opts(o->g->codec_opts, ist->st-
> >codecpar->codec_id, ic, st, ist->dec);
>  
>          ist->reinit_filters = -1;

Hi,

libdav1d is the preferred AV1 decoder in FFmpeg, libdav1d is always used when
running the command below even if user expects vaapi or other HW acceleration
methods.

$ ffmpeg -hwaccel vaapi -i av1.ivf -f null -

A similar command works as expected for other codecs, e.g. GPU is used when
running the command below (see 
http://ffmpeg.org/pipermail/ffmpeg-user/2022-July/055227.html)

$ ffmpeg -y -hwaccel vdpau -i input_vp9.webm output.ts

This patchset can ensure FFmpeg chooses a HW accelerated decoder when '-hwaccel
arg' is specified in the command line. 

Is there any comment for this patchset ?

Thanks
Haihao