[FFmpeg-devel] [PATCH] ac3dec: fix non-optimal dithering of zero bit mantissas

[Reimar Döffinger]

On Sat, Jan 05, 2013 at 12:10:07PM +0100, Reimar Döffinger wrote:
> On Sat, Jan 05, 2013 at 12:00:32PM +0100, madshi wrote:
> > 2013/1/5 Reimar Döffinger <Reimar.Doeffinger at gmx.de>
> > > Not sure about lfg (though IMHO it should be replaced by
> > > lcg anyway to not needlessly waste CPU time) but usually
> > > the lower bits contain more "randomness". Thus you should
> > > generally be using some kind of modulo operation instead
> > > of a division.
> > > In addition, doing division/modulo by a value that is not
> > > a power of 2 will introduce some bias. It might not be
> > > relevant, but it should be analyzed to make sure it truly
> > > isn't.
> > 
> > Well, my knowledge about random numbers is kinda limited. Maybe someone
> > with more knowledge in that area can improve my patch? In any case, with my
> > patch applied the frequency response of the AC3 decoder is clearly better
> > than without it, so the patch seems to work reasonably well as it is...
> 
> Well, my suggestion would be to use % 11864550 instead of / 362,
> that's trivial and more readable.

Though my calculations would suggest
(rng % 11863283) - 5931641
to be probably more accurate (closer to sqrt(2)/2*0x800000)?
With a bias (of same magnitude) towards mantissas < 458849.
It would have a significantly higher probability of needed a retry
if you would want non-biased random numbers.