[FFmpeg-devel] [RFC] Off-by-one checksumming

[Reimar Döffinger]

Hello,
You maybe a aware that our current solution to verifying inexact decoders is to store an uncompressed reference.
I believe that is not an acceptable solution in the long term, both for reasons of space used, difficulty of updating due to not being in the git repository and generally being a pain.
So I wanted to see if we can get up with something better.
I have a suggestion that should work - with caveats. I believe it has a low but significant chance of false positives and biased off-by-one errors might cause some pain depending on variant - which might be a good thing.
Since I am lazy I am asking for comments before actually testing it :-)
It works like this:
NBINS e.g. 256
uint64_t sums[NBINS] = {0};
for each sample:
  uint32_t rng = get lcg pseudo-random number;
  // note: should use high bits instead of %
  sums[rng % NBINS] += sample;

The check would be that each of the sums is fairly close to the reference. This will probably have to be configurable.
One variant would be to let the pseudo-random number decide between addition or subtraction, which should hide biased off-by-one errors - though the question is whether we want those to be hidden, float rounding errors should not normally end with a bias.

Thanks for reading,
Reimar