[FFmpeg-devel] [PATCH] lavc/cuvid: fail early if GPU can't handle given video resolution

[Pavel Koshevoy]

On Sun, Jan 1, 2017 at 6:00 PM,  <pkoshevoy at gmail.com> wrote:
> From: Pavel Koshevoy <pkoshevoy at gmail.com>
>
> NVDEC (CUVID) does not support unlimited video resolutions, so if the
> resolution of the source is known it can be used during avcodec_open2
> call to fail early, rather than failing later during
> avcodec_send_packet call.
>
> This problem surfaced when trying to decode 5120x2700 h246 video on
> Geforce GT 730 -- avcodec_open2 succeeded but decoding failed.
> ---
>  libavcodec/cuvid.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/libavcodec/cuvid.c b/libavcodec/cuvid.c
> index 8fc713d..2e70b62 100644
> --- a/libavcodec/cuvid.c
> +++ b/libavcodec/cuvid.c
> @@ -625,8 +625,8 @@ static int cuvid_test_dummy_decoder(AVCodecContext *avctx, CUVIDPARSERPARAMS *cu
>      cuinfo.ChromaFormat = cudaVideoChromaFormat_420;
>      cuinfo.OutputFormat = cudaVideoSurfaceFormat_NV12;
>
> -    cuinfo.ulWidth = 1280;
> -    cuinfo.ulHeight = 720;
> +    cuinfo.ulWidth = avctx->coded_width ? avctx->coded_width : 1280;
> +    cuinfo.ulHeight = avctx->coded_height ? avctx->coded_height : 720;
>      cuinfo.ulTargetWidth = cuinfo.ulWidth;
>      cuinfo.ulTargetHeight = cuinfo.ulHeight;
>
> --
> 2.9.2
>


Perhaps a more comprehensive solution is needed.  I've run into the
same problem again (avcodec_open2 succeeds, decoding spits errors)
when I tried to play back an mpeg2 stream with YUV422P pixel format.

I have a new patch but I'd like to test it on a Pascal GPU first.  I
am still hoping that 422 decoding will work.  It looks like it should
be supported, but I've observed on Geforce GT 730 that even though I
can instantiate a cuvid decoder with cudaVideoChromaFormat_422 without
error -- the decoded video is garbage everywhere except for 16-pixel
wide column along the left edge of the frame.

In ee88dcb2 cuvid_handle_video_sequence was hard-coded to error-out if
the chroma format is not 420.  This looks like an NVIDIA driver/cuvid
bug... does anyone know how to bring it to their attention?

    Pavel