[FFmpeg-devel] [PATCH] lavc/cuvid: fail early if GPU can't handle given video parameters

[wm4]

On Mon, 9 Jan 2017 11:55:38 -0700
Pavel Koshevoy <pkoshevoy at gmail.com> wrote:

> On Mon, Jan 9, 2017 at 3:12 AM, wm4 <nfxjfg at googlemail.com> wrote:
> > On Mon,  2 Jan 2017 14:26:45 -0700
> > pkoshevoy at gmail.com wrote:
> >  
> >> From: Pavel Koshevoy <pkoshevoy at gmail.com>  
> 
> >> +    ret = convert_to_cuda_video_chroma_format(avctx->pix_fmt, &probed_chroma_format);
> >> +    if (ret < 0) {
> >> +        // pixel format is not supported:
> >> +        return ret;
> >> +    }
> >> +    probed_width = avctx->coded_width ? avctx->coded_width : 1280;
> >> +    probed_height = avctx->coded_height ? avctx->coded_height : 720;  
> >
> > IMO relying on these fields is a bit of a problem, since they don't
> > even need to be set (the h264 bitstream can contain them).
> >
> > It's especially bad for pix_fmt, since my application for example
> > doesn't set this field at all in many cases.  
> 
> I've posted a new version of the patch which assumes
> AV_PIX_FMT_YUV420P if avctx->pix_fmt is unset.  This version of the
> patch should preserve the existing behavior for those use cases where
> pix_fmt is AV_PIX_FMT_NONE, width = 0, height = 0...  although it
> won't help if those fields are un-initialized a contain random values.
> 
> The real problem I am trying to solve -- I need to find the "best"
> decoder for the incoming stream, or file.  There are multiple decoders
> available for H264 ... Selecting based on supported output pixel
> formats would eliminate some.  Selecting based on AVCodecParameters
> would allow to eliminate those decoders that can't handle given
> parameters, although that appears to be insufficient.  Perhaps
> selecting based on given codec extradata (PPS/SPS) would be more
> robust.  extradata is usually part of the bitstream... is there an API
> to feed extradata to the decoder to see if it likes it?

You still can do it by e.g. invoking ffprobe (or just using
libavformat). But please don't do this kind of logic in the decoder
itself.

In this case, API users will probably have to live with the cuvid
decoder erroring out only after a few packets of input data (instead on
opening) if the stream uses unsupported features.

Don't assume that every API user will give you exactly the same
parameters as libavformat if the codec doesn't actually require these
parameters.

So I think especially checking the user-set pix_fmt in a h264 decoder is
a No.