[FFmpeg-devel] Implementing AAC fixed point decoder

[Nedeljko Babic]

Hello,

We are implementing AAC fixed point decoder based on AAC floating point decoder 
that is already implemented in FFmpeg. We have a code that is decoding correctly, 
but we have a problem with output signal.

If we use AV_SAMPLE_FMT_S16P sample format, we have correct output, but we have 
to do scaling during decoding and we have to add additional buffers and that is not 
the way that floating point AAC decoder works. Integrating it with fp decoder in 
FFmpeg will not look nice. On the other hand, if we use AV_SAMPLE_FMT_S32P sample 
format, internal working of our decoder is very similar to AAC fp dec, but there is 
some scaling outside of the decoder that diminishes signal almost completely.

Is there a way to use arbitrary scaling  with AV_SAMPLE_FMT_S32P sample fmt since 
default scaling is to large? Or maybe we should use some other sample fmt instead 
S16P and S32P that we tried?  Can we influence scaling from inside of the decoder?

Thanks,
Nedeljko