[FFmpeg-devel] [PATCH] [ISSUE 798] VP3/Theora: prevent stack corruption

[Alex Converse]

On Fri, Jan 30, 2009 at 11:37 AM, Michael Niedermayer <michaelni at gmx.at> wrote:
>
> On Thu, Jan 29, 2009 at 08:00:45PM -0500, Alex Converse wrote:
> > Hi,
> >
> > The attached patch fixes the crash associated with issue 798. (Though the
> > real issue appears to be with the demuxer).
> >
> > Basically a custom coding method doesn't get get set for one index value, so
> > when that index occurs a garbage value is used. This garbage value
> > propagates into current_frame_type through the compatible compatible_frame
> > table. current_frame_type is then used to index a write into last_dc causing
> > stack corruption.
> >
> > The behavior for an unset custom coding method is not specified by the
> > theora specification, so I chose to copy libtheora's behavior and assign it
> > to MODE_INTER_NO_MV.
>
> patch ok

applied