[FFmpeg-devel] flashsv error due to negative linesize value - not an intentional cross post!

[Jason Askew]

Yep, I tried abs, it did not work.

Michael, answer me this:  the av_alloc would be just fine with an
abs(linesize)*height, right?

If so, then it's only copy_region_enc that needs work, and if that's
the case, I think I can handle that.

My currently vague but growing understanding of this is that linesize
serves dual purpose, one of which is which direction in memory to move
through these data chunks. Yes?  No?



On 5/9/07, Michael Niedermayer <michaelni at gmx.at> wrote:
> Hi
>
> On Wed, May 09, 2007 at 07:35:55AM +0200, Benjamin Larsson wrote:
> > Jason Askew wrote:
> > > Advised to come to list from the users list:
> > >
> > > I've found a bug in flashsv encoding.
> > >
> > > There are several locations in flashsvenc.c that use linesize[0] to do
> > > a av_mallocz, i.e.:
> > >
> > > av_mallocz(p->linesize[0]*s->image_height);
> > >
> > > and other such actions:
> > >
> > >    memcpy(s->previous_frame, p->data[0], s->image_height*p->linesize[0]);
> > >
> > > but, as I've read several times now, linesize can be negative (with
> > > AVI files?).  This leads to a error of
> > > "Memory Allocation failed" when running ffmpeg
> > >
> > > The failure is in function:  flashsv_encode_frame
> > >
> > > I got everything setup to compile ffmpeg with flashsv on my windows PC
> > > and I've tried using just the absolute value of linesize, but that
> > > then seems to dork up things in another function, copy_region_enc.
> > >
> > > So...  I'll gladly bang on it some more, but if a dev could give me
> > > some guidance or ideas, that would great.
> > >
> > > Thanks.
> >
> > Hi, try putting an abs around the linesize.
>
> this alone wont work as data[X] points to the leftmost pixel of the
> _last_ line instead of the first so things like
> memcpy(data[0], ..., abs(linesieze)*height)
> would copy the last line and random data
>
> [...]
> --
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
>
> The greatest way to live with honor in this world is to be what we pretend
> to be. -- Socrates
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at mplayerhq.hu
> http://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-devel
>
>