[FFmpeg-devel] inverse weighitng tables for DV seems to be wrong

[Roman V. Shaposhnik]

Guys,

while working on a particular discrepancy in DV codec, I've noticed
that the static inverse weighting tables in DV codec seem to be
slightly wrong. Here's how I verified it:

    for (i=1; i<64; i++) {
        for (level = 0; level < 8192; level++) {
            int coeff  = (level * dv_weight_88[i]  + (1 << (dv_weight_bits))) >> (dv_weight_bits+1);
            int nlevel = (2*coeff * dv_iweight_88[i] + (1 << (dv_iweight_bits - 1))) >> dv_iweight_bits;
            if (level != nlevel)
                printf("%03d %d %d %d [%d]\n", FABS(level-nlevel), level, coeff, nlevel, i);
        }
    }

and here's what I've got:
   
$ ./test | cut -f1 -d\  | sort -n | uniq -c | awk '{print $0; total+=$1;} END {print total}'
 182700 001
  82533 002
  48647 003
  30470 004
  22314 005
  16054 006
   9311 007
   4336 008
   1557 009
   1198 010
   1188 011
   1142 012
    854 013
    516 014
    190 015
403010

So as you can see we get errors as high as 15. 

Now comes the question. The best way I know how to fix the situation
is to get rid of one of the tables altogether and generate it on
the fly. At least that way ffmpeg itself will be as close to preserving
material in multiple generations as possible. Now, here's the code:

    for (i=0; i<64; i++)
        dv_iweight_88[i] = ((uint64_t)8589934592/dv_weight_88[i] + 1)>>1;

and here's the result of my verification step:

$ ./test | cut -f1 -d\  | sort -n | uniq -c
 309105 001
   9421 002
318526

As you can see -- almost all of the errors are now in 1 range. 

But! Is there a way we can do better?

My division-foo is weak today, so any kind of suggestions will be appreciated.

Thanks,
Roman.