[FFmpeg-devel] [PATCH] potential fix for memory underread issue in h264 predict

[Ronald S. Bultje]

Hi,

On Wed, Jan 19, 2011 at 4:34 PM, Ronald S. Bultje <rsbultje at gmail.com> wrote:
> On Tue, Jan 18, 2011 at 9:32 PM, Daniel Kang <daniel.d.kang at gmail.com> wrote:
>> A VLC user (GomGom on freenode) has brought up a bug. h264's
>> pred8x8l_horizontal may underread a buffer in certain cases (i.e. if the
>> top row does not exist). The check for this has been moved up so the
>> buffer is not underread. The patch passes
>> fate-(h264|svq3|vp8|real-rv40), the tests relevant to this patch.
>>
>> I believe this occurs when the flag emu_edge is set.
>>
>> I cannot replicate the issue, so I am not sure if it is a correct fix.
>
> attached is a better version of this patch. Passes make fate-h264 and
> fixes at least some valgrind errors with -flags emu_edge on one of the
> fate samples that's failing with that flag set. It still breaks, I'm
> working on the other errors to fix it for good so we can enable this
> on fate and track it automatically.

Slightly improved one attached.

Ronald
-------------- next part --------------
A non-text attachment was scrubbed...
Name: horizontal_simd.patch
Type: application/octet-stream
Size: 2285 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20110119/b96b1fec/attachment.obj>