[FFmpeg-devel] [PATCH 3/4] avcodec/nvdec: add support for 12 bit formats

[Carl Eugen Hoyos]

2017-11-11 16:21 GMT+01:00 Timo Rothenpieler <timo at rothenpieler.org>:
> ---
>  libavcodec/nvdec.c | 18 +++++++++++++++---
>  1 file changed, 15 insertions(+), 3 deletions(-)
>
> diff --git a/libavcodec/nvdec.c b/libavcodec/nvdec.c
> index 780c2e67cf..d3620f0d1f 100644
> --- a/libavcodec/nvdec.c
> +++ b/libavcodec/nvdec.c
> @@ -234,9 +234,21 @@ int ff_nvdec_decode_init(AVCodecContext *avctx, unsigned int dpb_size)
>          frames_ctx->format            = AV_PIX_FMT_CUDA;
>          frames_ctx->width             = (avctx->width + 1) & ~1;
>          frames_ctx->height            = (avctx->height + 1) & ~1;
> -        frames_ctx->sw_format         = AV_PIX_FMT_NV12;
> -        frames_ctx->sw_format         = sw_desc->comp[0].depth > 8 ?
> -                                        AV_PIX_FMT_P010 : AV_PIX_FMT_NV12;
> +
> +        switch (sw_desc->comp[0].depth) {
> +        case 8:
> +            frames_ctx->sw_format = AV_PIX_FMT_NV12;
> +            break;
> +        case 10:
> +            frames_ctx->sw_format = AV_PIX_FMT_P010;
> +            break;
> +        case 12:
> +            frames_ctx->sw_format = AV_PIX_FMT_P016;

Ideally, this should also set avctx->bits_per_raw_sample
or an equivalent, if it exists.

Carl Eugen