[FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a specific decode format

Eoff, Ullysses A ullysses.a.eoff at intel.com
Tue Apr 23 19:53:44 EEST 2019


*bump*

> -----Original Message-----
> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf Of Mark Thompson
> Sent: Wednesday, February 20, 2019 1:52 PM
> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a specific decode format
> 
> On 18/02/2019 05:05, Fu, Linjie wrote:
> >> -----Original Message-----
> >> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On Behalf
> >> Of Fu, Linjie
> >> Sent: Friday, November 16, 2018 16:37
> >> To: FFmpeg development discussions and patches <ffmpeg-
> >> devel at ffmpeg.org>
> >> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a
> >> specific decode format
> >>
> >>> -----Original Message-----
> >>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
> >> Behalf
> >>> Of Mark Thompson
> >>> Sent: Thursday, November 15, 2018 05:48
> >>> To: ffmpeg-devel at ffmpeg.org
> >>> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a
> >>> specific decode format
> >>>
> >>> On 14/11/18 01:35, Fu, Linjie wrote:
> >>>>> -----Original Message-----
> >>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
> >>> Behalf
> >>>>> Of Mark Thompson
> >>>>> Sent: Wednesday, November 14, 2018 09:11
> >>>>> To: ffmpeg-devel at ffmpeg.org
> >>>>> Subject: Re: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a
> >>>>> specific decode format
> >>>>>
> >>>>> On 14/11/18 00:50, Fu, Linjie wrote:
> >>>>>>> -----Original Message-----
> >>>>>>> From: ffmpeg-devel [mailto:ffmpeg-devel-bounces at ffmpeg.org] On
> >>>>> Behalf
> >>>>>>> Of Mark Thompson
> >>>>>>> Sent: Wednesday, November 14, 2018 07:44
> >>>>>>> To: ffmpeg-devel at ffmpeg.org
> >>>>>>> Subject: [FFmpeg-devel] [PATCH v2] ffmpeg: Add option to force a
> >>>>> specific
> >>>>>>> decode format
> >>>>>>>
> >>>>>>> Fixes #7519.
> >>>>>>> ---
> >>>>>>>  doc/ffmpeg.texi      | 13 ++++++++++++
> >>>>>>>  fftools/ffmpeg.c     | 10 ++++++++++
> >>>>>>>  fftools/ffmpeg.h     |  4 ++++
> >>>>>>>  fftools/ffmpeg_opt.c | 47
> >>>>>>> ++++++++++++++++++++++++++++++++++++++++++++
> >>>>>>>  4 files changed, 74 insertions(+)
> >>>>>>>
> >>>>>>> diff --git a/doc/ffmpeg.texi b/doc/ffmpeg.texi
> >>>>>>> index 3717f22d42..d127bc0f0d 100644
> >>>>>>> --- a/doc/ffmpeg.texi
> >>>>>>> +++ b/doc/ffmpeg.texi
> >>>>>>> @@ -920,6 +920,19 @@ would be more efficient.
> >>>>>>>  When doing stream copy, copy also non-key frames found at the
> >>>>>>>  beginning.
> >>>>>>>
> >>>>>>> + at item -decode_format[:@var{stream_specifier}]
> >>>>>>> @var{pixfmt}[, at var{pixfmt}...] (@emph{input,per-stream})
> >>>>>>> +Set the possible output formats to be used by the decoder for this
> >>>>> stream.
> >>>>>>> +If the decoder does not natively support any format in the given list
> >>> for
> >>>>>>> +the input stream then decoding will fail rather than continuing with a
> >>>>>>> +different format.
> >>>>>>> +
> >>>>>>> +In general this should not be set - the decoder will select an output
> >>>>>>> +format based on the input stream parameters and available
> >>> components,
> >>>>>>> and
> >>>>>>> +that will be automatically converted to whatever the output
> >> requires.
> >>> It
> >>>>>>> +may be useful to force a hardware decoder supporting output in
> >>>>> multiple
> >>>>>>> +different memory types to pick a desired one, or to ensure that a
> >>>>> hardware
> >>>>>>> +decoder is used when software fallback is also available.
> >>>>>>> +
> >>>>>>>  @item -init_hw_device
> >>>>>>> @var{type}[=@var{name}][:@var{device}[, at var{key=value}...]]
> >>>>>>>  Initialise a new hardware device of type @var{type} called
> >>> @var{name},
> >>>>>>> using the
> >>>>>>>  given device parameters.
> >>>>>>> diff --git a/fftools/ffmpeg.c b/fftools/ffmpeg.c
> >>>>>>> index 38c21e944a..c651c8d3a8 100644
> >>>>>>> --- a/fftools/ffmpeg.c
> >>>>>>> +++ b/fftools/ffmpeg.c
> >>>>>>> @@ -598,6 +598,7 @@ static void ffmpeg_cleanup(int ret)
> >>>>>>>          avsubtitle_free(&ist->prev_sub.subtitle);
> >>>>>>>          av_frame_free(&ist->sub2video.frame);
> >>>>>>>          av_freep(&ist->filters);
> >>>>>>> +        av_freep(&ist->decode_formats);
> >>>>>>>          av_freep(&ist->hwaccel_device);
> >>>>>>>          av_freep(&ist->dts_buffer);
> >>>>>>>
> >>>>>>> @@ -2800,6 +2801,15 @@ static enum AVPixelFormat
> >>>>>>> get_format(AVCodecContext *s, const enum AVPixelFormat
> >>>>>>>          const AVCodecHWConfig  *config = NULL;
> >>>>>>>          int i;
> >>>>>>>
> >>>>>>> +        if (ist->decode_formats) {
> >>>>>>> +            for (i = 0; ist->decode_formats[i] != AV_PIX_FMT_NONE; i++)
> >> {
> >>>>>>> +                if (ist->decode_formats[i] == *p)
> >>>>>>> +                    break;
> >>>>>>> +            }
> >>>>>>> +            if (ist->decode_formats[i] != *p)
> >>>>>>> +                continue;
> >>>>>>> +        }
> >>>>>>> +
> >>>>>>>          if (!(desc->flags & AV_PIX_FMT_FLAG_HWACCEL))
> >>>>>>>              break;
> >>>>>>>
> >>>>>>> diff --git a/fftools/ffmpeg.h b/fftools/ffmpeg.h
> >>>>>>> index eb1eaf6363..b06fd18b1c 100644
> >>>>>>> --- a/fftools/ffmpeg.h
> >>>>>>> +++ b/fftools/ffmpeg.h
> >>>>>>> @@ -125,6 +125,8 @@ typedef struct OptionsContext {
> >>>>>>>      int        nb_ts_scale;
> >>>>>>>      SpecifierOpt *dump_attachment;
> >>>>>>>      int        nb_dump_attachment;
> >>>>>>> +    SpecifierOpt *decode_formats;
> >>>>>>> +    int        nb_decode_formats;
> >>>>>>>      SpecifierOpt *hwaccels;
> >>>>>>>      int        nb_hwaccels;
> >>>>>>>      SpecifierOpt *hwaccel_devices;
> >>>>>>> @@ -334,6 +336,8 @@ typedef struct InputStream {
> >>>>>>>      int top_field_first;
> >>>>>>>      int guess_layout_max;
> >>>>>>>
> >>>>>>> +    enum AVPixelFormat *decode_formats;
> >>>>>>> +
> >>>>>>>      int autorotate;
> >>>>>>>
> >>>>>>>      int fix_sub_duration;
> >>>>>>> diff --git a/fftools/ffmpeg_opt.c b/fftools/ffmpeg_opt.c
> >>>>>>> index d4851a2cd8..63bb05053b 100644
> >>>>>>> --- a/fftools/ffmpeg_opt.c
> >>>>>>> +++ b/fftools/ffmpeg_opt.c
> >>>>>>> @@ -701,6 +701,7 @@ static void
> >> add_input_streams(OptionsContext
> >>> *o,
> >>>>>>> AVFormatContext *ic)
> >>>>>>>          AVStream *st = ic->streams[i];
> >>>>>>>          AVCodecParameters *par = st->codecpar;
> >>>>>>>          InputStream *ist = av_mallocz(sizeof(*ist));
> >>>>>>> +        char *decode_formats = NULL;
> >>>>>>>          char *framerate = NULL, *hwaccel_device = NULL;
> >>>>>>>          const char *hwaccel = NULL;
> >>>>>>>          char *hwaccel_output_format = NULL;
> >>>>>>> @@ -797,6 +798,49 @@ static void
> >> add_input_streams(OptionsContext
> >>>>> *o,
> >>>>>>> AVFormatContext *ic)
> >>>>>>>              ist->top_field_first = -1;
> >>>>>>>              MATCH_PER_STREAM_OPT(top_field_first, i, ist-
> >>> top_field_first,
> >>> ic,
> >>>>>>> st);
> >>>>>>>
> >>>>>>> +            MATCH_PER_STREAM_OPT(decode_formats, str,
> >>>>> decode_formats, ic,
> >>>>>>> st);
> >>>>>>> +            if (decode_formats) {
> >>>>>>> +                const char *p, *q;
> >>>>>>> +                int i, nb_formats;
> >>>>>>> +                char tmp[32];
> >>>>>>> +
> >>>>>>> +                nb_formats = 0;
> >>>>>>> +                for (p = decode_formats; p; p = strchr(p + 1, ','))
> >>>>>>> +                    ++nb_formats;
> >>>>>>> +
> >>>>>>> +                ist->decode_formats =
> >>>>>>> +                    av_malloc_array(nb_formats + 1, sizeof(*ist-
> >>>>>> decode_formats));
> >>>>>>> +                if (!ist->decode_formats)
> >>>>>>> +                    exit_program(1);
> >>>>>>> +
> >>>>>>> +                p = decode_formats;
> >>>>>>> +                for (i = 0; i < nb_formats; i++) {
> >>>>>>> +                    q = strchr(p, ',');
> >>>>>>> +                    if (!q) {
> >>>>>>> +                        ist->decode_formats[i] = av_get_pix_fmt(p);
> >>>>>>> +                        if (ist->decode_formats[i] == AV_PIX_FMT_NONE)
> >>>>>>> +                            break;
> >>>>>>> +                        continue;
> >>>>>>> +                    }
> >>>>>>> +                    if (q - p > sizeof(tmp) - 1)
> >>>>>>> +                        break;
> >>>>>>> +                    memcpy(tmp, p, q - p);
> >>>>>>> +                    tmp[q - p] = 0;
> >>>>>>> +                    ist->decode_formats[i] = av_get_pix_fmt(tmp);
> >>>>>>> +                    if (ist->decode_formats[i] == AV_PIX_FMT_NONE)
> >>>>>>> +                        break;
> >>>>>>> +                    p = q + 1;
> >>>>>>> +                }
> >>>>>>> +                if (i < nb_formats) {
> >>>>>>> +                    av_log(NULL, AV_LOG_FATAL,
> >>>>>>> +                           "Unrecognised decode format: %s.\n", p);
> >>>>>>> +                    exit_program(1);
> >>>>>>> +                }
> >>>>>>> +                ist->decode_formats[nb_formats] = AV_PIX_FMT_NONE;
> >>>>>>> +            } else {
> >>>>>>> +                ist->decode_formats = NULL;
> >>>>>>> +            }
> >>>>>>> +
> >>>>>>>              MATCH_PER_STREAM_OPT(hwaccels, str, hwaccel, ic, st);
> >>>>>>>              if (hwaccel) {
> >>>>>>>                  // The NVDEC hwaccels use a CUDA device, so remap the
> >>> name
> >>>>> here.
> >>>>>>> @@ -3583,6 +3627,9 @@ const OptionDef options[] = {
> >>>>>>>          "audio bitrate (please use -b:a)", "bitrate" },
> >>>>>>>      { "b",            OPT_VIDEO | HAS_ARG | OPT_PERFILE | OPT_OUTPUT,
> >>>>>>> { .func_arg = opt_bitrate },
> >>>>>>>          "video bitrate (please use -b:v)", "bitrate" },
> >>>>>>> +    { "decode_format",  OPT_VIDEO | OPT_STRING | HAS_ARG |
> >>>>>>> OPT_EXPERT |
> >>>>>>> +                        OPT_SPEC | OPT_INPUT,                                    { .off =
> >>>>>>> OFFSET(decode_formats) },
> >>>>>>> +        "set output format(s) to be used by decoder, fail if none of
> >> these
> >>>>>>> formats are available", "format" },
> >>>>>>>      { "hwaccel",          OPT_VIDEO | OPT_STRING | HAS_ARG |
> >>> OPT_EXPERT
> >>>>> |
> >>>>>>>                            OPT_SPEC | OPT_INPUT,                                  { .off =
> >>>>>>> OFFSET(hwaccels) },
> >>>>>>>          "use HW accelerated decoding", "hwaccel name" },
> >>>>>>> --
> >>>>>>> 2.19.1
> >>>>>>>
> >>>>>>
> >>>>>> Is there any other conditions which will cause the hwaccel failed and
> >>>>> fallback to software path?
> >>>>>> Like profile was not supported by hardware?
> >>>>>
> >>>>> If this option is set then get_format will never choose a format not on
> >> the
> >>> list
> >>>>> you provide, so it shouldn't be able to use software decode unless a
> >>>>> software format is on the list.
> >>>>>
> >>>>>> I applied the patch and tested some cases, like yuv422  VAAPI
> >> decoding
> >>>>> (currently not supported), HEVC decoding with unsupported profile.
> >>>>>> It reports an error like "hwaccel initialization returned error", and
> >>> continue
> >>>>> to decode and produce less frames than expected. (which I thought
> >>> fallback
> >>>>> still happened).
> >>>>>
> >>>>> Can you give an example?  With -v debug there will be output saying
> >>> exactly
> >>>>> what happened in ff_get_format().
> >>>>>
> >>>>> - Mark
> >>>>
> >>>>
> >>>> Got information like this:
> >>>>
> >>>> [hevc @ 0x55b87457dbc0] Unknown HEVC profile: 0
> >>>> [hevc @ 0x55b87457dbc0] Decoding SPS
> >>>> [hevc @ 0x55b87457dbc0] Unknown HEVC profile: 0
> >>>> [hevc @ 0x55b87457dbc0] Decoding PPS
> >>>> [hevc @ 0x55b87457dbc0] Format vaapi_vld chosen by get_format().
> >>>> [hevc @ 0x55b87457dbc0] Format vaapi_vld requires hwaccel initialisation.
> >>>> [hevc @ 0x55b87457dbc0] Codec hevc profile 0 not supported for
> >> hardware
> >>> decode.
> >>>> [hevc @ 0x55b87457dbc0] Failed setup for format vaapi_vld: hwaccel
> >>> initialisation returned error.
> >>>> [hevc @ 0x55b87457dbc0] Format vaapi_vld not usable, retrying
> >>> get_format() without it.
> >>>> [hevc @ 0x55b87457dbc0] Error parsing NAL unit #3.
> >>>> cur_dts is invalid (this is harmless if it occurs once at the start per stream)
> >>>> [hevc @ 0x55b87458b7c0] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0,
> >>> temporal_id: 0
> >>>> [hevc @ 0x55b87458b7c0] Could not find ref with POC 0
> >>>>
> >>>> See the attached report file  for more details.
> >>>
> >>> This is a decoder bug - get_format() is not correctly called again after it fails,
> >>> and the decoder then uses the software pixfmt derived from the SPS on
> >>> subsequent frames.
> >>>
> >>> (I did pretty much all of my testing with H.264, which does not have this
> >>> problem.)
> >>>
> >> Works for me on forcing hwaccel  in HEVC vaapi decoding after applying the
> >> hevcdec patch.
> >> Thanks for answering my questions.
> >>
> > Ping?
> > Can this be merged?
> > https://trac.ffmpeg.org/ticket/7519#comment:4
> 
> I think this got lost because I was feeling it loosely depended on <https://lists.ffmpeg.org/pipermail/ffmpeg-devel/2018-
> November/236376.html> in order to actually work in the intended use-case.  I wasn't particularly confident in that patch being the
> best way to do it and it isn't my code, so I was hoping for any thoughts from someone more familiar with it.
> 
> Shall I apply this part anyway?
> 
> Thanks,
> 
> - Mark
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel


More information about the ffmpeg-devel mailing list