[Ffmpeg-devel] improving encoding (possibly big perceptual gains)

[Rich Felker]

with all the recent discussions over improving lavc, i've got some
ideas. mainly i'm worried that psnr is very misleading when trying to
improve motion estimation decisions since often a particular motion
vector will give better psnr but much worse actual quality due to loss
of small detail and mud/blocking.

because of this, i'd like to suggest that lavc be extended to measure
the 'uqi' (universal [image] quality index) proposed by the paper at
this site:

http://www.cns.nyu.edu/~zwang/files/research/quality_index/demo_lena.html

i think the images there speak for themselves regarding how stupid
psnr is as a judge of quality.

however, printing statistics isn't much use if you can't actually
improve encoding. so the main idea is too add a *cmp function to
measure the uqi and use this for motion estimation and optimal
quantization and qprd, etc. maybe also the ratecontrol engine itself.
this should do a much better job of telling when it's ok to use poor
quantization, zero ac coeffs, etc. and when it will look like shit
(like the last lena picture ;).

the next more ambitious idea is for future codecs, perhaps snow: to
use only the correlation part of uqi to evaluate motion vectors, then
measure the difference in mean and variance and store those in the
bitstream. this would allow complex changes such as fades and changes
in lighting to be encoded almost entirely with motion vectors, whereas
mpeg4 codecs usually use dct-coded residue for such scenes with
horrible quality as the result unless qp is 2 or less.

ideas, anyone? sorry i'm too lame to code on lavc.. actually i don't
know the codebase and i'm busy working on another project right now.
however the *cmp system seems extensible enough that adding a uqi
method to it should be easy. the other things (quantization, qprd,
rate control) might be more difficult.

rich