[FFmpeg-devel] [PATCH] potential fix for memory underread issue in h264 predict

[Ronald S. Bultje]

Hi,

On Wed, Jan 19, 2011 at 8:25 PM, Jason Garrett-Glaser <jason at x264.com> wrote:
> On Wed, Jan 19, 2011 at 3:03 PM, Ronald S. Bultje <rsbultje at gmail.com> wrote:
>> Hi,
>>
>> On Wed, Jan 19, 2011 at 4:34 PM, Ronald S. Bultje <rsbultje at gmail.com> wrote:
>>> On Tue, Jan 18, 2011 at 9:32 PM, Daniel Kang <daniel.d.kang at gmail.com> wrote:
>>>> A VLC user (GomGom on freenode) has brought up a bug. h264's
>>>> pred8x8l_horizontal may underread a buffer in certain cases (i.e. if the
>>>> top row does not exist). The check for this has been moved up so the
>>>> buffer is not underread. The patch passes
>>>> fate-(h264|svq3|vp8|real-rv40), the tests relevant to this patch.
>>>>
>>>> I believe this occurs when the flag emu_edge is set.
>>>>
>>>> I cannot replicate the issue, so I am not sure if it is a correct fix.
>>>
>>> attached is a better version of this patch. Passes make fate-h264 and
>>> fixes at least some valgrind errors with -flags emu_edge on one of the
>>> fate samples that's failing with that flag set. It still breaks, I'm
>>> working on the other errors to fix it for good so we can enable this
>>> on fate and track it automatically.
>>
>> Slightly improved one attached.
>>
>> Ronald
>
> LGTM if it works.

Applied. Thanks for spotting, Daniel.

Ronald