[FFmpeg-devel] [PATCH] Move VC1 loop filter to DSPContext

[David Conrad]

On Apr 14, 2009, at 12:04 AM, David Conrad wrote:

> On Apr 13, 2009, at 11:09 AM, Kostya Shishkov wrote:
>
>> On Mon, Apr 13, 2009 at 11:39 AM, David Conrad <lessen42 at gmail.com>  
>> wrote:
>>> On Apr 3, 2009, at 2:42 AM, Kostya wrote:
>>>
>>>> On Wed, Apr 01, 2009 at 02:44:13PM -0400, David Conrad wrote:
>>>>>
>>>>> The loop filter is identical to the VC1 spec and the one in the
>>>>> reference implementation.
>>>>> The file I have that uses the loop filter is bitexact to the  
>>>>> binary
>>>>> codec for I and P frames, but not B frames.
>>>>> Of the files I have that do not use the loop filter, three are
>>>>> bitexact and one is not bitexact for any frame.
>>>>> The two files that aren't bitexact are from HDDVD rips, so I'll  
>>>>> create
>>>>> a roundup ticket since there's probably some weird feature or
>>>>> something being used.
>>>>
>>>> That may be caused by the fact spec requires 16-bit processing
>>>> (well, 10-bit) because of transform and filtering (loop, overlap)  
>>>> and
>>>> my code uses 8-bit everywhere.
>>>>
>>>> Now I'm a bit far from my computers so I probably look at it in two
>>>> weeks. Roundup ticket will be a good reminder.
>>>
>>> I was looking through the code, and it seems that qpel isn't done  
>>> for
>>> forward MVs, nor is the rounding flag taken into account  
>>> (vc1_interp_mc()
>>> line 1911-1919 of vc1.c). I haven't read the spec to see if this  
>>> is actually
>>> correct, but I'll look into this more tomorrow unless Kostya beats  
>>> me to it.
>>
>> Yes, please look.
>> Maybe it should be 1/4 pel MC there too.
>
> It turns out that, although needed, the lack of qpel on forward MVs  
> wasn't causing the main +/- 1 differences; that was due to MMX2 hpel  
> and MMX no_rnd hpel. The MMX2 hpel was under !bitexact, so my fault  
> there; I'm looking into why the MMX no_rnd hpel isn't bitexact with  
> the C version. This still leaves some +/- 1 differences that appear  
> to be loop filter related.

The two remaining issues with the loop filter are that the frame  
quantizer should be used rather than the macroblock quantizer, and the  
order of application is wrong, so I went ahead and applied this to  
make it a bit easier on me making patches for those orthogonal issues.