[FFmpeg-soc] [PATCH] AAC clipped escape distortion

[Alex Converse]

We discussed this once without ever reaching a consensus. I'm going to bring
it up again.

    if(vec[k] == 64.0f){//FIXME: slow
This line is verifies an escape
        if(t >= 165140.0f*IQ){ // clipped value
This line is a shortcut to determine if we have a clipped value
        di = t - 165140.0f;
This line attempts to inverse quantize the coefficient and subtract it from
the actual coefficient. Using the equation t - value^(4/3) where value =8191
        curbits += 21;
    }else{
        int c = av_clip(quant(t, Q), 0, 8191);
This line quantizes the coefficient
        di = t - c*cbrt(c)*IQ;
This line inverse quantizes the coefficient and subtracts it from the actual
coefficient. Using the equation t - value^(4/3) * IQ
        curbits += av_log2(c)*2 - 4 + 1;
    }

The two di lines do not use the same equation.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: aacenc-clipped-escape-distortion.diff
Type: text/x-patch
Size: 2098 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-soc/attachments/20090616/d9e1a68d/attachment.bin>