[FFmpeg-devel] [PATCH] avcodec/av1dec: check avctx->hwaccel when hwaccel pix_fmt selected

[James Almer]

On 9/18/2020 2:40 AM, Wang, Fei W wrote:
> 
> 
>> -----Original Message-----
>> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
>> Hendrik Leppkes
>> Sent: Thursday, September 17, 2020 5:21 PM
>> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
>> Subject: Re: [FFmpeg-devel] [PATCH] avcodec/av1dec: check avctx->hwaccel
>> when hwaccel pix_fmt selected
>>
>> On Thu, Sep 17, 2020 at 10:38 AM Fei Wang <fei.w.wang at intel.com> wrote:
>>>
>>> Pix fmt with hwaccel flag may not be chosen in format probing, in this
>>> case avctx->hwaccel will not be inited.
>>>
>>> Signed-off-by: Fei Wang <fei.w.wang at intel.com>
>>> ---
>>>  libavcodec/av1dec.c | 12 ++++++++----
>>>  1 file changed, 8 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c index
>>> 0bb04a3e44..cdcc618013 100644
>>> --- a/libavcodec/av1dec.c
>>> +++ b/libavcodec/av1dec.c
>>> @@ -251,6 +251,7 @@ static int get_pixel_format(AVCodecContext *avctx)
>>> {
>>>      AV1DecContext *s = avctx->priv_data;
>>>      const AV1RawSequenceHeader *seq = s->raw_seq;
>>> +    const AVPixFmtDescriptor *desc;
>>>      uint8_t bit_depth;
>>>      int ret;
>>>      enum AVPixelFormat pix_fmt = AV_PIX_FMT_NONE; @@ -327,10 +328,13
>>> @@ static int get_pixel_format(AVCodecContext *avctx)
>>>       * Since now the av1 decoder doesn't support native decode, if it will be
>>>       * implemented in the future, need remove this check.
>>>       */
>>> -    if (!avctx->hwaccel) {
>>> -        av_log(avctx, AV_LOG_ERROR, "Your platform doesn't suppport"
>>> -               " hardware accelerated AV1 decoding.\n");
>>> -        return AVERROR(ENOSYS);
>>> +    desc = av_pix_fmt_desc_get(ret);
>>> +    if (desc && (desc->flags & AV_PIX_FMT_FLAG_HWACCEL)) {
>>> +        if (!avctx->hwaccel) {
>>> +            av_log(avctx, AV_LOG_ERROR, "Your platform doesn't suppport"
>>> +                   " hardware accelerated AV1 decoding.\n");
>>> +            return AVERROR(ENOSYS);
>>> +        }
>>>      }
>>>
>>
>> Isn't it supposed to quit here, because we do not have software decoding?
> 
> Since now av1 decoder allow probe, that will try to decode first frame to 
> find stream info in open_file stage. While the hwaccel will not be inited.
> If without change here, the error log will be print out but later during
> transcode stage, it can gives out the correct output.

If you let it choose a software pix_fmt, it will think the decoder can
actually output something, which is not true.
Probing works fine even if it fails at this point. It will have set
enough AVCodecContext fields to allow things like remuxing, hence the
relevant fate tests succeeding as is.