[FFmpeg-devel] [PATCH] h264 deblocking result doesn't match with SIMD on and off.

[Haruhiko Yamagata]

In h264 decoder, deblocking result doesn't match with SIMD on and off.

In h264_loop_filter* (dsputil.c), there are several

    if( FFABS( p0 - q0 ) < alpha &&
        FFABS( p1 - p0 ) < beta &&
        FFABS( q1 - q0 ) < beta )

In SIMD optimized functions, this is done in H264_DEBLOCK_MASK (h264dsp_mmx.c).

    if (!(FFABS( p0 - q0 ) > (uint8_t)(alpha -1) ||
          FFABS( p1 - p0 ) > (uint8_t)(beta  -1) ||   // beta == 0 ? Oops : ...
          FFABS( q1 - q0 ) > (uint8_t)(beta  -1) ))

So the SIMD optimized functions expect alpha and beta not to be zero.
OK, let them assume it for performance.
The attached patch fixes this issue.

The errors are usually invisible, but after a long chain of reference,
the errors accumulate and eventually get visible.

Best regards,
Haruhiko Yamagata
ffdshow-tryouts

-------------- next part --------------
A non-text attachment was scrubbed...
Name: dont_deblock_if_beta_is_0.patch
Type: application/octet-stream
Size: 1305 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20090524/e26410a1/attachment.obj>