[MPlayer-dev-eng] possible bugs in vf_decimate filter

[Trent Piepho]

On Sun, 15 Oct 2006, Rich Felker wrote:
> On Sat, Oct 14, 2006 at 07:09:32AM -0700, Trent Piepho wrote: that would be
> much slower and not much more accurate. The idea is to look for maximal change
> over _any_ 8x8 block, not just aligned-to-8-pixels 8x8 blocks.
>
> > The filter also skips the leftmost 8 columns.  Nothing is skipped on the
> > top, bottom, or right side, and this behavior is not documented.
>
> It does? This sounds like a bug, unless there was some reason that
> measuring them was difficult.
>
> > The calculation for the number of blocks that need to exceed the 'lo' value
> > is wrong too.  It's using (w/16)*(h/16)*frac, but, if the code is adjusted
> > to not do overlapping SADs, it should be (w/8)*(h/8)*frac.  If the code is
> > supposed to do SADs that overlap by four pixels, then it would be ((w-4)/4)
> > * ((h-4)/4) * frac.
>
> The normalization is probably to prevent integer overflow. Don't
> change it to /8 or /4 unless you're sure it won't overflow. Also it's
> really irrelevant exactly what the normalization constant is as long
> as:

The man page says frac as, "not more than <frac> portion (1 meaning the
whole image) differs by..." What's there now has 16 mean the whole image.
I've changed it so 1 means all of the image.  There shouldn't be any
realistic possibility of overflow, as that would need an image with
dimensions around 200,000 pixels.

This patch will fix the problem with the left column too, and continues to
check a block every four pixels.
-------------- next part --------------
Index: libmpcodecs/vf_decimate.c
===================================================================
--- libmpcodecs/vf_decimate.c	(revision 20277)
+++ libmpcodecs/vf_decimate.c	(working copy)
@@ -80,9 +80,14 @@
 {
 	int x, y;
 	int d, c=0;
-	int t = (w/16)*(h/16)*frac;
+	int t = ((w-4)/4)*((h-4)/4)*frac;
+
+	/* The goal is to find _any_ 8x8 blocks that exceeds the thresholds.  However, 
+	   there are rather a lot of possible blocks.  As a performance compromise, x 
+	   and y are incremented by 4 to check one out of 16 blocks. */
+
 	for (y = 0; y < h-7; y += 4) {
-		for (x = 8; x < w-7; x += 4) {
+		for (x = 0; x < w-7; x += 4) {
 			d = diff(old+x+y*os, new+x+y*ns, os, ns);
 			if (d > hi) return 0;
 			if (d > lo) {