[FFmpeg-soc] [PATCH] AAC clipped escape distortion

[Kostya]

On Tue, Jun 16, 2009 at 12:38:51PM -0400, Alex Converse wrote:
> We discussed this once without ever reaching a consensus. I'm going to bring
> it up again.
> 
>     if(vec[k] == 64.0f){//FIXME: slow
> This line is verifies an escape
>         if(t >= 165140.0f*IQ){ // clipped value
> This line is a shortcut to determine if we have a clipped value
>         di = t - 165140.0f;
> This line attempts to inverse quantize the coefficient and subtract it from
> the actual coefficient. Using the equation t - value^(4/3) where value =8191
>         curbits += 21;
>     }else{
>         int c = av_clip(quant(t, Q), 0, 8191);
> This line quantizes the coefficient
>         di = t - c*cbrt(c)*IQ;
> This line inverse quantizes the coefficient and subtracts it from the actual
> coefficient. Using the equation t - value^(4/3) * IQ
>         curbits += av_log2(c)*2 - 4 + 1;
>     }
> 
> The two di lines do not use the same equation.

Patch ok. I forgot that with very small quantizer  ven low values may hit
that wall.