[FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a specific decode format
Fu, Linjie
linjie.fu at intel.com
Mon Feb 18 07:05:19 EET 2019
> -----Original Message-----
> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf
> Of Fu, Linjie
> Sent: Friday, November 16, 2018 16:37
> To: FFmpeg development discussions and patches <ffmpeg-
> devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a
> specific decode format
>
> > -----Original Message-----
> > From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
> Behalf
> > Of Mark Thompson
> > Sent: Thursday, November 15, 2018 05:48
> > To: ffmpeg-devel at ffmpeg.org
> > Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a
> > specific decode format
> >
> > On 14/11/18 01:35, Fu, Linjie wrote:
> > >> -----Original Message-----
> > >> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
> > Behalf
> > >> Of Mark Thompson
> > >> Sent: Wednesday, November 14, 2018 09:11
> > >> To: ffmpeg-devel at ffmpeg.org
> > >> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a
> > >> specific decode format
> > >>
> > >> On 14/11/18 00:50, Fu, Linjie wrote:
> > >>>> -----Original Message-----
> > >>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
> > >> Behalf
> > >>>> Of Mark Thompson
> > >>>> Sent: Wednesday, November 14, 2018 07:44
> > >>>> To: ffmpeg-devel at ffmpeg.org
> > >>>> Subject: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a
> > >> specific
> > >>>> decode format
> > >>>>
> > >>>> Fixes #7519.
> > >>>> ---
> > >>>> doc/ffmpeg.texi | 13 ++++++++++++
> > >>>> fftools/ffmpeg.c | 10 ++++++++++
> > >>>> fftools/ffmpeg.h | 4 ++++
> > >>>> fftools/ffmpeg_opt.c | 47
> > >>>> ++++++++++++++++++++++++++++++++++++++++++++
> > >>>> 4 files changed, 74 insertions(+)
> > >>>>
> > >>>> diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi
> > >>>> index 3717f22d42..d127bc0f0d 100644
> > >>>> --- a/doc/ffmpeg.texi
> > >>>> +++ b/doc/ffmpeg.texi
> > >>>> @@ -920,6 +920,19 @@ would be more efficient.
> > >>>> When doing stream copy, copy also non-key frames found at the
> > >>>> beginning.
> > >>>>
> > >>>> + at item -decode_format[:@var{stream_specifier}]
> > >>>> @var{pixfmt}[, at var{pixfmt}...] (@emph{input,per-stream})
> > >>>> +Set the possible output formats to be used by the decoder for this
> > >> stream.
> > >>>> +If the decoder does not natively support any format in the given list
> > for
> > >>>> +the input stream then decoding will fail rather than continuing with a
> > >>>> +different format.
> > >>>> +
> > >>>> +In general this should not be set - the decoder will select an output
> > >>>> +format based on the input stream parameters and available
> > components,
> > >>>> and
> > >>>> +that will be automatically converted to whatever the output
> requires.
> > It
> > >>>> +may be useful to force a hardware decoder supporting output in
> > >> multiple
> > >>>> +different memory types to pick a desired one, or to ensure that a
> > >> hardware
> > >>>> +decoder is used when software fallback is also available.
> > >>>> +
> > >>>> @item -init_hw_device
> > >>>> @var{type}[=@var{name}][:@var{device}[, at var{key=value}...]]
> > >>>> Initialise a new hardware device of type @var{type} called
> > @var{name},
> > >>>> using the
> > >>>> given device parameters.
> > >>>> diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
> > >>>> index 38c21e944a..c651c8d3a8 100644
> > >>>> --- a/fftools/ffmpeg.c
> > >>>> +++ b/fftools/ffmpeg.c
> > >>>> @@ -598,6 +598,7 @@ static void ffmpeg_cleanup(int ret)
> > >>>> avsubtitle_free(&ist->prev_sub.subtitle);
> > >>>> av_frame_free(&ist->sub2video.frame);
> > >>>> av_freep(&ist->filters);
> > >>>> + av_freep(&ist->decode_formats);
> > >>>> av_freep(&ist->hwaccel_device);
> > >>>> av_freep(&ist->dts_buffer);
> > >>>>
> > >>>> @@ -2800,6 +2801,15 @@ static enum AVPixelFormat
> > >>>> get_format(AVCodecContext *s, const enum AVPixelFormat
> > >>>> const AVCodecHWConfig *config = NULL;
> > >>>> int i;
> > >>>>
> > >>>> + if (ist->decode_formats) {
> > >>>> + for (i = 0; ist->decode_formats[i] != AV_PIX_FMT_NONE; i++)
> {
> > >>>> + if (ist->decode_formats[i] == *p)
> > >>>> + break;
> > >>>> + }
> > >>>> + if (ist->decode_formats[i] != *p)
> > >>>> + continue;
> > >>>> + }
> > >>>> +
> > >>>> if (!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL))
> > >>>> break;
> > >>>>
> > >>>> diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
> > >>>> index eb1eaf6363..b06fd18b1c 100644
> > >>>> --- a/fftools/ffmpeg.h
> > >>>> +++ b/fftools/ffmpeg.h
> > >>>> @@ -125,6 +125,8 @@ typedef struct OptionsContext {
> > >>>> int nb_ts_scale;
> > >>>> SpecifierOpt *dump_attachment;
> > >>>> int nb_dump_attachment;
> > >>>> + SpecifierOpt *decode_formats;
> > >>>> + int nb_decode_formats;
> > >>>> SpecifierOpt *hwaccels;
> > >>>> int nb_hwaccels;
> > >>>> SpecifierOpt *hwaccel_devices;
> > >>>> @@ -334,6 +336,8 @@ typedef struct InputStream {
> > >>>> int top_field_first;
> > >>>> int guess_layout_max;
> > >>>>
> > >>>> + enum AVPixelFormat *decode_formats;
> > >>>> +
> > >>>> int autorotate;
> > >>>>
> > >>>> int fix_sub_duration;
> > >>>> diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
> > >>>> index d4851a2cd8..63bb05053b 100644
> > >>>> --- a/fftools/ffmpeg_opt.c
> > >>>> +++ b/fftools/ffmpeg_opt.c
> > >>>> @@ -701,6 +701,7 @@ static void
> add_input_streams(OptionsContext
> > *o,
> > >>>> AVFormatContext *ic)
> > >>>> AVStream *st = ic->streams[i];
> > >>>> AVCodecParameters *par = st->codecpar;
> > >>>> InputStream *ist = av_mallocz(sizeof(*ist));
> > >>>> + char *decode_formats = NULL;
> > >>>> char *framerate = NULL, *hwaccel_device = NULL;
> > >>>> const char *hwaccel = NULL;
> > >>>> char *hwaccel_output_format = NULL;
> > >>>> @@ -797,6 +798,49 @@ static void
> add_input_streams(OptionsContext
> > >> *o,
> > >>>> AVFormatContext *ic)
> > >>>> ist->top_field_first = -1;
> > >>>> MATCH_PER_STREAM_OPT(top_field_first, i, ist-
> >top_field_first,
> > ic,
> > >>>> st);
> > >>>>
> > >>>> + MATCH_PER_STREAM_OPT(decode_formats, str,
> > >> decode_formats, ic,
> > >>>> st);
> > >>>> + if (decode_formats) {
> > >>>> + const char *p, *q;
> > >>>> + int i, nb_formats;
> > >>>> + char tmp[32];
> > >>>> +
> > >>>> + nb_formats = 0;
> > >>>> + for (p = decode_formats; p; p = strchr(p + 1, ','))
> > >>>> + ++nb_formats;
> > >>>> +
> > >>>> + ist->decode_formats =
> > >>>> + av_malloc_array(nb_formats + 1, sizeof(*ist-
> > >>> decode_formats));
> > >>>> + if (!ist->decode_formats)
> > >>>> + exit_program(1);
> > >>>> +
> > >>>> + p = decode_formats;
> > >>>> + for (i = 0; i < nb_formats; i++) {
> > >>>> + q = strchr(p, ',');
> > >>>> + if (!q) {
> > >>>> + ist->decode_formats[i] = av_get_pix_fmt(p);
> > >>>> + if (ist->decode_formats[i] == AV_PIX_FMT_NONE)
> > >>>> + break;
> > >>>> + continue;
> > >>>> + }
> > >>>> + if (q - p > sizeof(tmp) - 1)
> > >>>> + break;
> > >>>> + memcpy(tmp, p, q - p);
> > >>>> + tmp[q - p] = 0;
> > >>>> + ist->decode_formats[i] = av_get_pix_fmt(tmp);
> > >>>> + if (ist->decode_formats[i] == AV_PIX_FMT_NONE)
> > >>>> + break;
> > >>>> + p = q + 1;
> > >>>> + }
> > >>>> + if (i < nb_formats) {
> > >>>> + av_log(NULL, AV_LOG_FATAL,
> > >>>> + "Unrecognised decode format: %s.\n", p);
> > >>>> + exit_program(1);
> > >>>> + }
> > >>>> + ist->decode_formats[nb_formats] = AV_PIX_FMT_NONE;
> > >>>> + } else {
> > >>>> + ist->decode_formats = NULL;
> > >>>> + }
> > >>>> +
> > >>>> MATCH_PER_STREAM_OPT(hwaccels, str, hwaccel, ic, st);
> > >>>> if (hwaccel) {
> > >>>> // The NVDEC hwaccels use a CUDA device, so remap the
> > name
> > >> here.
> > >>>> @@ -3583,6 +3627,9 @@ const OptionDef options[] = {
> > >>>> "audio bitrate (please use -b:a)", "bitrate" },
> > >>>> { "b", OPT_VIDEO | HAS_ARG | OPT_PERFILE | OPT_OUTPUT,
> > >>>> { .func_arg = opt_bitrate },
> > >>>> "video bitrate (please use -b:v)", "bitrate" },
> > >>>> + { "decode_format", OPT_VIDEO | OPT_STRING | HAS_ARG |
> > >>>> OPT_EXPERT |
> > >>>> + OPT_SPEC | OPT_INPUT, { .off =
> > >>>> OFFSET(decode_formats) },
> > >>>> + "set output format(s) to be used by decoder, fail if none of
> these
> > >>>> formats are available", "format" },
> > >>>> { "hwaccel", OPT_VIDEO | OPT_STRING | HAS_ARG |
> > OPT_EXPERT
> > >> |
> > >>>> OPT_SPEC | OPT_INPUT, { .off =
> > >>>> OFFSET(hwaccels) },
> > >>>> "use HW accelerated decoding", "hwaccel name" },
> > >>>> --
> > >>>> 2.19.1
> > >>>>
> > >>>
> > >>> Is there any other conditions which will cause the hwaccel failed and
> > >> fallback to software path?
> > >>> Like profile was not supported by hardware?
> > >>
> > >> If this option is set then get_format will never choose a format not on
> the
> > list
> > >> you provide, so it shouldn't be able to use software decode unless a
> > >> software format is on the list.
> > >>
> > >>> I applied the patch and tested some cases, like yuv422 VAAPI
> decoding
> > >> (currently not supported), HEVC decoding with unsupported profile.
> > >>> It reports an error like "hwaccel initialization returned error", and
> > continue
> > >> to decode and produce less frames than expected. (which I thought
> > fallback
> > >> still happened).
> > >>
> > >> Can you give an example? With -v debug there will be output saying
> > exactly
> > >> what happened in ff_get_format().
> > >>
> > >> - Mark
> > >
> > >
> > > Got information like this:
> > >
> > > [hevc @ 0x55b87457dbc0] Unknown HEVC profile: 0
> > > [hevc @ 0x55b87457dbc0] Decoding SPS
> > > [hevc @ 0x55b87457dbc0] Unknown HEVC profile: 0
> > > [hevc @ 0x55b87457dbc0] Decoding PPS
> > > [hevc @ 0x55b87457dbc0] Format vaapi_vld chosen by get_format().
> > > [hevc @ 0x55b87457dbc0] Format vaapi_vld requires hwaccel initialisation.
> > > [hevc @ 0x55b87457dbc0] Codec hevc profile 0 not supported for
> hardware
> > decode.
> > > [hevc @ 0x55b87457dbc0] Failed setup for format vaapi_vld: hwaccel
> > initialisation returned error.
> > > [hevc @ 0x55b87457dbc0] Format vaapi_vld not usable, retrying
> > get_format() without it.
> > > [hevc @ 0x55b87457dbc0] Error parsing NAL unit #3.
> > > cur_dts is invalid (this is harmless if it occurs once at the start per stream)
> > > [hevc @ 0x55b87458b7c0] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0,
> > temporal_id: 0
> > > [hevc @ 0x55b87458b7c0] Could not find ref with POC 0
> > >
> > > See the attached report file for more details.
> >
> > This is a decoder bug - get_format() is not correctly called again after it fails,
> > and the decoder then uses the software pixfmt derived from the SPS on
> > subsequent frames.
> >
> > (I did pretty much all of my testing with H.264, which does not have this
> > problem.)
> >
> Works for me on forcing hwaccel in HEVC vaapi decoding after applying the
> hevcdec patch.
> Thanks for answering my questions.
>
Ping?
Can this be merged?
https://trac.ffmpeg.org/ticket/7519#comment:4
---
Linjie
More information about the ffmpeg-devel
mailing list