[FFmpeg-devel] [PATCH] Fix VP3 dequantization matrices when an IDCT permutation is used

[David Conrad]

On Sep 17, 2008, at 3:37 PM, Aurelien Jacobs wrote:

> David Conrad wrote:
>
>> Hi,
>>
>> It appears that the reason why the VP3 SSE/MMX IDCTs weren't bitexact
>> to the C version was because the dct coefficients weren't being
>> dequantized correctly when a permutation was needed. The coefficients
>> are permutated correctly in unpack_vlcs() but the dequantization
>> matrices are built assuming no permutation. This fix allows both the
>> SSE2 and MMX IDCTs produce bitexact output for the VP3 and Theora
>> samples I have.
>>
>> [...]
>>
>> --- a/libavcodec/i386/dsputil_mmx.c
>> +++ b/libavcodec/i386/dsputil_mmx.c
>> @@ -2528,9 +2528,7 @@ void dsputil_init_mmx(DSPContext* c,  
>> AVCodecContext *avctx)
>>                 c->idct_permutation_type= FF_LIBMPEG2_IDCT_PERM;
>> #endif
>>             }else if((ENABLE_VP3_DECODER || ENABLE_VP5_DECODER ||  
>> ENABLE_VP6_DECODER) &&
>> -                     idct_algo==FF_IDCT_VP3 &&
>> -                     avctx->codec->id!=CODEC_ID_THEORA &&
>> -                     !(avctx->flags & CODEC_FLAG_BITEXACT)){
>> +                     idct_algo==FF_IDCT_VP3){
>
> As you enable this code for theora, you also must add a
> '|| ENABLE_THEORA_DECODER' to the above list, to ensure this code
> won't be disabled when only the theora decoder is compiled in.

Ah right, fixed.

> Aurel