[FFmpeg-devel] [PATCH] potential fix for memory underread issue in h264 predict

[Jason Garrett-Glaser]

On Wed, Jan 19, 2011 at 3:03 PM, Ronald S. Bultje <rsbultje at gmail.com> wrote:
> Hi,
>
> On Wed, Jan 19, 2011 at 4:34 PM, Ronald S. Bultje <rsbultje at gmail.com> wrote:
>> On Tue, Jan 18, 2011 at 9:32 PM, Daniel Kang <daniel.d.kang at gmail.com> wrote:
>>> A VLC user (GomGom on freenode) has brought up a bug. h264's
>>> pred8x8l_horizontal may underread a buffer in certain cases (i.e. if the
>>> top row does not exist). The check for this has been moved up so the
>>> buffer is not underread. The patch passes
>>> fate-(h264|svq3|vp8|real-rv40), the tests relevant to this patch.
>>>
>>> I believe this occurs when the flag emu_edge is set.
>>>
>>> I cannot replicate the issue, so I am not sure if it is a correct fix.
>>
>> attached is a better version of this patch. Passes make fate-h264 and
>> fixes at least some valgrind errors with -flags emu_edge on one of the
>> fate samples that's failing with that flag set. It still breaks, I'm
>> working on the other errors to fix it for good so we can enable this
>> on fate and track it automatically.
>
> Slightly improved one attached.
>
> Ronald

LGTM if it works.

Jason