[FFmpeg-devel] Testing Inexact Data

[Mike Melanson]

Hi,

I am at the point with the FATE server ( http://fate.multimedia.cx/ ) 
where I want to start testing data that is not defined to be bit exact. 
Michael recommended that I should use the tiny_psnr utility. Sounds 
great, except that, well... I'm not very good at math and I don't know 
what the numbers actually mean. :)

I decoded surge-2-16-B-.mp3.mov from:

   http://samples.mplayerhq.hu/mov/qtaudio/

I decoded the file to WAV on x86_32 and PPC machines and then ran:

$ tiny_psnr mp3-ppc.wav mp3-x86.wav 2 0 0
stddev:  0.53 PSNR:101.81 bytes:2322432

When I compare one of the files against itself:
stddev:  0.00 PSNR:99.99 bytes:2322432

Issues:

1) What are good numbers vs. bad numbers in this case? And for other cases?

2) I'm assuming that anything on one side of a threshold should be a 
"PASS" and anything on the other side should be a "FAIL". I need a tool 
that can easily return this information. One approach would be to supply 
a threshold number to tiny_psnr and modify the program to return 
non-zero if that threshold fails.

-- 
	-Mike Melanson