[Ffmpeg-devel] [PATCH][RFC] ac3enc window generation

[Benjamin Larsson]

Hi,

Michael Niedermayer wrote:

>[...]
>
>
>well, do you think that trying round up / round down pair wise (down, down)
>(up, down) (down, up) would be non smooth too? if so then why do you think
>that the normal rounding / flooring is more smooth? they should have a
>systematic error too (using a simple line as very crude model for our
>window and rounding/flooring it to 16bit would result in nasty periodic 
>errors ...)
>  
>
Ok, I though that you meant that every pair should be minimized
individually. But after trying the diffrent cases, round to nearest was
the best match.

>and about the decission between rounding vs. flooring i dont feel like i
>understand the MDCT/windowing and its reversibility and such well enough to
>decide it
>maybe you could elaborate on where that overflow happens in case 
>x(i)^2+x(i+n)^2>1 ?
>  
>
After looking at the code, overflow would never happen, so it shouldn't
really matter.

>isnt that at the decoding side already? in that case i dont think
>there a problem if they add to larger then 1 as due to quantization and
>rounding errors in the (I)MDCT the decoder must deal with too large samples
>anyway
>  
>
True.

After all tests and some more thinking my opinion is to use the rounded
window, the error is smaller and the dynamic range is increased slightly.

I'll do some more tests and if they turn out ok I'll post a final patch.

MvH
Benjamin Larsson

-- 
"incorrect information" is an oxymoron. Information is, by definition, factual, correct.