[FFmpeg-devel] flashsv error due to negative linesize value - not an intentional cross post!

[Benjamin Larsson]

Jason Askew wrote:
> Advised to come to list from the users list:
> 
> I've found a bug in flashsv encoding.
> 
> There are several locations in flashsvenc.c that use linesize[0] to do
> a av_mallocz, i.e.:
> 
> av_mallocz(p->linesize[0]*s->image_height);
> 
> and other such actions:
> 
>    memcpy(s->previous_frame, p->data[0], s->image_height*p->linesize[0]);
> 
> but, as I've read several times now, linesize can be negative (with
> AVI files?).  This leads to a error of
> "Memory Allocation failed" when running ffmpeg
> 
> The failure is in function:  flashsv_encode_frame
> 
> I got everything setup to compile ffmpeg with flashsv on my windows PC
> and I've tried using just the absolute value of linesize, but that
> then seems to dork up things in another function, copy_region_enc.
> 
> So...  I'll gladly bang on it some more, but if a dev could give me
> some guidance or ideas, that would great.
> 
> Thanks.

Hi, try putting an abs around the linesize.

MvH
Benjamin Larsson