[FFmpeg-devel] H.264 decoder calls fill_rectangle() with unaligned buffers

[Michael Niedermayer]

On Mon, Jul 14, 2008 at 06:18:30PM +0100, M?ns Rullg?rd wrote:
> 
> Michael Niedermayer wrote:
> > On Mon, Jul 14, 2008 at 02:58:18PM +0100, M?ns Rullg?rd wrote:
> >>
> >> Michael Niedermayer wrote:
> >> > On Mon, Jul 14, 2008 at 12:36:47AM +0100, M?ns Rullg?rd wrote:
> >> >> If STRIDE_ALIGN is set to 16, the H.264 decoder will call
> >> >> fill_rectangle() with buffer addresses failing the alignment assert()
> >> >> at the top of the function.  This obviously does no harm to this
> >> >> function, so I'm not sure which is incorrect, the assert() or the
> >> >> calling code.
> >> >
> >> > And i am not sure without knowing what that calling code is.
> >>
> >> Sorry, I forgot to mention.  It can happen with any of the
> >> fill_rectangle(h->mv_cache[list][ scan8[0] ], 4, 4, 8, val, 4)
> >> and similar calls where width*size > 8, for instance the one
> >> on line 4784 of h264.c.
> >
> > I suspect you are missing a
> > #define DECLARE_ALIGNED_8(t, v) DECLARE_ALIGNED(16, t, v)
> > in dsputil.h where STRIDE_ALIGN is set to 16
> >
> > (yes DECLARE_ALIGNED_8 is not a good name i know, it should be
> > renamed to DECLARE_ALIGNED_NATIVE or _MAX or so)
> 
> Or why not like this?
> 
> #define DECLARE_STRIDE_ALIGNED(t, v) DECLARE_ALIGNED(STRIDE_ALIGN, t, v)

fine for me as well

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Everything should be made as simple as possible, but not simpler.
-- Albert Einstein
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080714/50ecd67b/attachment.pgp>