[FFmpeg-devel] [PATCH] avutil/imgutils: av_image_get_buffer_size(): do not insert padding between stride*height and palette

[wm4]

On Sat, 13 Feb 2016 19:38:01 +0100
Michael Niedermayer <michael at niedermayer.cc> wrote:

> This fixes the layout that is stored in pal8 nut with odd width*height
> 
> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> ---
>  libavcodec/avpicture.c |    2 ++
>  libavutil/imgutils.c   |    4 ++++
>  2 files changed, 6 insertions(+)
> 
> diff --git a/libavcodec/avpicture.c b/libavcodec/avpicture.c
> index 56435f4..cac9287 100644
> --- a/libavcodec/avpicture.c
> +++ b/libavcodec/avpicture.c
> @@ -51,6 +51,8 @@ int avpicture_layout(const AVPicture* src, enum AVPixelFormat pix_fmt, int width
>  
>  int avpicture_get_size(enum AVPixelFormat pix_fmt, int width, int height)
>  {
> +    if (pix_fmt == AV_PIX_FMT_PAL8)
> +        return FFALIGN(width*height, 4) + 1024;
>      return av_image_get_buffer_size(pix_fmt, width, height, 1);
>  }
>  
> diff --git a/libavutil/imgutils.c b/libavutil/imgutils.c
> index adf6fdd..976bd23 100644
> --- a/libavutil/imgutils.c
> +++ b/libavutil/imgutils.c
> @@ -372,6 +372,10 @@ int av_image_get_buffer_size(enum AVPixelFormat pix_fmt,
>      if (desc->flags & AV_PIX_FMT_FLAG_PSEUDOPAL)
>          return FFALIGN(width, align) * height;
>  
> +    // For paletted images we do not align the palette as in AVFrames
> +    if (desc->flags & AV_PIX_FMT_FLAG_PAL)
> +        return FFALIGN(width, align) * height + 1024;
> +
>      return av_image_fill_arrays(data, linesize, NULL, pix_fmt,
>                                  width, height, align);
>  }

It seems wrong to litter the code with specific checks for an obscure
pixel format, and change behavior, just to make these function's
behavior line up with what NUT needs?

It's also possible that this is actually what the API user wants; there
are many implications.

Also, is this consistent with av_image_fill_arrays etc.?