[FFmpeg-devel] [PATCH] h264 deblocking result doesn't match with SIMD on and off.

[David Conrad]

On May 23, 2009, at 3:01 PM, Michael Niedermayer wrote:

> On Sat, May 23, 2009 at 01:39:50PM -0400, David Conrad wrote:
>> On May 23, 2009, at 1:25 PM, Michael Niedermayer wrote:
>>
>>> On Sun, May 24, 2009 at 12:40:18AM +0900, Haruhiko Yamagata wrote:
>>>> In h264 decoder, deblocking result doesn't match with SIMD on and  
>>>> off.
>>>>
>>>> In h264_loop_filter* (dsputil.c), there are several
>>>>
>>>>  if( FFABS( p0 - q0 ) < alpha &&
>>>>      FFABS( p1 - p0 ) < beta &&
>>>>      FFABS( q1 - q0 ) < beta )
>>>>
>>>> In SIMD optimized functions, this is done in H264_DEBLOCK_MASK
>>>> (h264dsp_mmx.c).
>>>>
>>>>  if (!(FFABS( p0 - q0 ) > (uint8_t)(alpha -1) ||
>>>>        FFABS( p1 - p0 ) > (uint8_t)(beta  -1) ||   // beta == 0 ?  
>>>> Oops :
>>>> ...
>>>>        FFABS( q1 - q0 ) > (uint8_t)(beta  -1) ))
>>>>
>>>> So the SIMD optimized functions expect alpha and beta not to be  
>>>> zero.
>>>> OK, let them assume it for performance.
>>>> The attached patch fixes this issue.
>>>>
>>>> The errors are usually invisible, but after a long chain of  
>>>> reference,
>>>> the errors accumulate and eventually get visible.
>>>
>>> could you provide a h264 file that decodes differently?
>>>
>>> all reference streams that we claim to support decode binary  
>>> identical
>>> as far as i remember
>>
>> This fixes issue 776
>
> so can you confirm that mmx&c matches the reference decoder output  
> exactly
> with that patch?

Yep, just tried with JM and the results match mmx+c with the patch (c  
is unchanged with the patch of course)