[FFmpeg-devel] [PATCH] RV40 Loop Filter

[Kostya]

On Mon, Oct 27, 2008 at 01:38:35PM +0100, Michael Niedermayer wrote:
> On Mon, Oct 27, 2008 at 02:26:45PM +0200, Kostya wrote:
> [...]
> > I just treated this as a sum of four consequent coeffs multiplied
> > by 26 with some additional changes - the first one (or the last one)
> > should be (26-1)*coefficient, so I subtract one (here's the "sub" name)
> > and depending on the last coefficient position we have either 25 or
> > 26+25 multiplier.
> [...]
> > 0 - horizontal, 1 - vertical
> 
> this should be in the doxy ...
> 
> 
> >  
> > [filter horror]
> >  
> > > i will not accept this mess, sorry.
> > > If you dont (or cant) clean this up i will try eventually but that might not
> > > be soon.
> > > as is, this is too much of a mess and iam unwilling to belive that h264
> > > drafts required such mess.
> > 
> > Me neither. Another example is that they use 1/4th pel interpolation filter
> > (1*X[-2] - 5*X[-1] + 52*X[0] + 20*X[1] - 5*X[2] + 1*X[3] + 32) >> 6
> > instead of calculating halfpel value first and then averaging it,
> > which leads to some rounding errors.
> 
> (1*X[-2] - 5*X[-1] + 52*X[0] + 20*X[1] - 5*X[2] + 1*X[3] + 32) >> 6
> =
> ((1*X[-2] - 5*X[-1] + 20*X[0] + 20*X[1] - 5*X[2] + 1*X[3]) >> 5 + X[0] + 1) >> 1
> 
> though intermediates can be outside 0..255

Of course, and the lack of rounding for interpolated halfpel leads to
several differences. I've found it on filtering that values:
       20 20 20 |20| 22 23 23
RV40 gives 20 for it, H264 gives 21

> > 
> > Since I've fixed some quantization-related bugs, it looks more decent
> > even without loop filter, so I'm tempted to try less sophisticated
> > loop filter (for low-power CPUs) from the same codecs, it looks simple
> > and sane.
> 
> ill try to find some unused vomit proof bags and see if i can simplify
> this darn rv loop filter.

Please wait until I try simpler filter.
 
> [...]
> -- 
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> Awnsering whenever a program halts or runs forever is
> On a turing machine, in general impossible (turings halting problem).
> On any real computer, always possible as a real computer has a finite number
> of states N, and will either halt in less than N cycles or never halt.

reminds me of my current AAC encoder :S