[FFmpeg-devel] [PATCH] h264 deblocking result doesn't match with SIMD on and off.

[David Conrad]

On May 23, 2009, at 1:25 PM, Michael Niedermayer wrote:

> On Sun, May 24, 2009 at 12:40:18AM +0900, Haruhiko Yamagata wrote:
>> In h264 decoder, deblocking result doesn't match with SIMD on and  
>> off.
>>
>> In h264_loop_filter* (dsputil.c), there are several
>>
>>   if( FFABS( p0 - q0 ) < alpha &&
>>       FFABS( p1 - p0 ) < beta &&
>>       FFABS( q1 - q0 ) < beta )
>>
>> In SIMD optimized functions, this is done in H264_DEBLOCK_MASK
>> (h264dsp_mmx.c).
>>
>>   if (!(FFABS( p0 - q0 ) > (uint8_t)(alpha -1) ||
>>         FFABS( p1 - p0 ) > (uint8_t)(beta  -1) ||   // beta == 0 ?  
>> Oops :
>> ...
>>         FFABS( q1 - q0 ) > (uint8_t)(beta  -1) ))
>>
>> So the SIMD optimized functions expect alpha and beta not to be zero.
>> OK, let them assume it for performance.
>> The attached patch fixes this issue.
>>
>> The errors are usually invisible, but after a long chain of  
>> reference,
>> the errors accumulate and eventually get visible.
>
> could you provide a h264 file that decodes differently?
>
> all reference streams that we claim to support decode binary identical
> as far as i remember

This fixes issue 776