[FFmpeg-devel] [PATCH]Fix linesize for pix_fmt pal8

[Michael Niedermayer]

On Sat, Jun 30, 2012 at 11:41:39PM +0200, Stefano Sabatini wrote:
> On date Wednesday 2012-06-20 07:33:04 +0000, Carl Eugen Hoyos encoded:
> > Stefano Sabatini <stefasab <at> gmail.com> writes:
> > 
> > > Check attached patch, it should fix the problem when coupled 
> > > with the new imgutils functions.
> > 
> > The patch breaks decoding some of the samples from ticket #823.
> 
> New try (I still don't like very much this approach), but at least I
> realized that changing the pal8 output alignment is just wrong. And we
> could use a better heuristic for detecting the linesize (for example
> iterating the loop:
> aligned_size(A) <= buf_size && aligned_size(A+1) >= buf_size
> => choose A
> 
> and may be also useful to be able to override the alignment in case of
> need (working on a patch for that).
> 
> But apart from this I see more failures with other formats:
> ffmpeg -i tests/lena.pnm -s 257x257 -vcodec rawvideo -pix_fmt PIX_FMT -y out.avi
> 
> in particular with formats monob, rgb565, rgb24, still investigating
> (hints are welcome).
> -- 
> FFmpeg = Frenzy & Fascinating Monstrous Prodigious Elastic Ghost

>  rawdec.c |   23 +++++++++++++++--------
>  1 file changed, 15 insertions(+), 8 deletions(-)
> c02246ed0069dbe581c989237cc382e919c87a20  0004-lavc-rawdec-fix-palette-alignment-setting.patch
> From dc6fd4caa5b7ee4925845785fab67a697f24aeb0 Mon Sep 17 00:00:00 2001
> From: Stefano Sabatini <stefasab at gmail.com>
> Date: Sat, 30 Jun 2012 19:50:48 +0200
> Subject: [PATCH] lavc/rawdec: fix palette alignment setting
> 
> ---
>  libavcodec/rawdec.c |   23 +++++++++++++++--------
>  1 files changed, 15 insertions(+), 8 deletions(-)
> 
> diff --git a/libavcodec/rawdec.c b/libavcodec/rawdec.c
> index a0211b6..91135fc 100644
> --- a/libavcodec/rawdec.c
> +++ b/libavcodec/rawdec.c
> @@ -217,15 +217,22 @@ static int raw_decode(AVCodecContext *avctx,
>              frame->palette_has_changed = 1;
>          }
>      }
> -    if((avctx->pix_fmt==PIX_FMT_BGR24    ||
> -        avctx->pix_fmt==PIX_FMT_GRAY8    ||
> -        avctx->pix_fmt==PIX_FMT_RGB555LE ||
> -        avctx->pix_fmt==PIX_FMT_RGB555BE ||
> -        avctx->pix_fmt==PIX_FMT_RGB565LE ||
> +    if (avctx->pix_fmt==PIX_FMT_BGR24     ||
> +        avctx->pix_fmt==PIX_FMT_GRAY8     ||
> +        avctx->pix_fmt==PIX_FMT_RGB555LE  ||
> +        avctx->pix_fmt==PIX_FMT_RGB555BE  ||
> +        avctx->pix_fmt==PIX_FMT_RGB565LE  ||
>          avctx->pix_fmt==PIX_FMT_MONOWHITE ||
> -        avctx->pix_fmt==PIX_FMT_PAL8) &&
> -        FFALIGN(frame->linesize[0], linesize_align)*avctx->height <= buf_size)
> -        frame->linesize[0] = FFALIGN(frame->linesize[0], linesize_align);
> +        avctx->pix_fmt==PIX_FMT_PAL8) {
> +        int aligned_plane_size = FFALIGN(frame->linesize[0], linesize_align)*avctx->height;
> +        int plane_buf_size = avctx->pix_fmt == PIX_FMT_PAL8 && !context->buffer ?
> +            buf_size - AVPALETTE_SIZE : buf_size;
> +        if (aligned_plane_size < plane_buf_size) {
                                 ^^^
> +            av_log(avctx, AV_LOG_DEBUG, "aligned plane size %d <= plane buffer size %d, assuming alignment of %d\n",
                                                                 ^^^^

mismatch

otherwise if it works it should be ok

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Frequently ignored awnser#1 FFmpeg bugs should be sent to our bugtracker. User
questions about the command line tools should be sent to the ffmpeg-user ML.
And questions about how to use libav* should be sent to the libav-user ML.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20120703/81024566/attachment.asc>