[FFmpeg-devel] [PATCH] AAC decoder

[Robert Swain]

On 29 May 2008, at 12:30, Michael Niedermayer wrote:
> On Thu, May 29, 2008 at 12:17:30PM +0100, Robert Swain wrote:
>> 2008/5/29 Robert Swain <robert.swain at gmail.com>:
>>> 2008/5/29 Robert Swain <robert.swain at gmail.com>:
>>>> I'm considering what precision to use for the tables. Using  
>>>> printf()
>>>> %42.32f gives the first value as:
>>>>
>>>>            0.00000000000000088817841970012523,
>>>>
>>>> and the last as:
>>>>
>>>>    451452825.35479241609573364257812500000000,
>>>>
>>>> How does one decide such things? :)
>>>
>>> It seems scientific notation would be best for this. That should  
>>> have
>>> been obvious to me looking at those numbers. I've used 16 decimal
>>> places, is that OK? Looking at ivquant_tab now.
>
> The number of digits should be enough so that the float table is equal
> to what the generator produced. Iam not sure if this is true for  
> ivquant_tab.

Tested manually by generating the tables with varying decimal places  
and comparing to a float casted calculated result for each table value  
and increasing the decimal places until all were the same.

>> Patch attached.
>
> ok

Applied with tested tables.

Rob