[FFmpeg-cvslog] avcodec/texturedspenc: Fix indexing in color distribution determination

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Mon Jan  2 00:28:33 2017 +0100| [ade36d61de8ea5a5acb30a05a0cbcda069127143] | committer: Marton Balint

avcodec/texturedspenc: Fix indexing in color distribution determination

Fixes CID1396405

MSE and PSNR is slightly improved, and some noticable corruptions disappear as
well.

Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
Signed-off-by: Marton Balint <cus at passwd.hu>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=ade36d61de8ea5a5acb30a05a0cbcda069127143
---

 libavcodec/texturedspenc.c |  6 +++---
 tests/ref/fate/vbn-dxt1    | 16 ++++++++--------
 tests/ref/fate/vbn-dxt5    | 16 ++++++++--------
 3 files changed, 19 insertions(+), 19 deletions(-)

diff --git a/libavcodec/texturedspenc.c b/libavcodec/texturedspenc.c
index 21e341c613..7ae28ea134 100644
--- a/libavcodec/texturedspenc.c
+++ b/libavcodec/texturedspenc.c
@@ -255,11 +255,11 @@ static void optimize_colors(const uint8_t *block, ptrdiff_t stride,
 
         muv = minv = maxv = bp[0];
         for (y = 0; y < 4; y++) {
-            for (x = 4; x < 4; x += 4) {
+            for (x = 0; x < 4; x++) {
                 muv += bp[x * 4 + y * stride];
-                if (bp[x] < minv)
+                if (bp[x * 4 + y * stride] < minv)
                     minv = bp[x * 4 + y * stride];
-                else if (bp[x] > maxv)
+                else if (bp[x * 4 + y * stride] > maxv)
                     maxv = bp[x * 4 + y * stride];
             }
         }
diff --git a/tests/ref/fate/vbn-dxt1 b/tests/ref/fate/vbn-dxt1
index 0af4444d94..e33d9b63e1 100644
--- a/tests/ref/fate/vbn-dxt1
+++ b/tests/ref/fate/vbn-dxt1
@@ -1,11 +1,11 @@
 frame:0    pts:0       pts_time:0
-lavfi.psnr.mse.r=36.794250
-lavfi.psnr.psnr.r=32.473003
-lavfi.psnr.mse.g=33.350525
-lavfi.psnr.psnr.g=32.899776
-lavfi.psnr.mse.b=28.317383
-lavfi.psnr.psnr.b=33.610271
+lavfi.psnr.mse.r=36.699280
+lavfi.psnr.psnr.r=32.484230
+lavfi.psnr.mse.g=33.508362
+lavfi.psnr.psnr.g=32.879272
+lavfi.psnr.mse.b=27.947266
+lavfi.psnr.psnr.b=33.667412
 lavfi.psnr.mse.a=21186.988281
 lavfi.psnr.psnr.a=4.870111
-lavfi.psnr.mse_avg=5321.362793
-lavfi.psnr.psnr_avg=10.870575
+lavfi.psnr.mse_avg=5321.285645
+lavfi.psnr.psnr_avg=10.870638
diff --git a/tests/ref/fate/vbn-dxt5 b/tests/ref/fate/vbn-dxt5
index dbec0bd13d..ab345d60a6 100644
--- a/tests/ref/fate/vbn-dxt5
+++ b/tests/ref/fate/vbn-dxt5
@@ -1,11 +1,11 @@
 frame:0    pts:0       pts_time:0
-lavfi.psnr.mse.r=36.794250
-lavfi.psnr.psnr.r=32.473003
-lavfi.psnr.mse.g=33.350525
-lavfi.psnr.psnr.g=32.899776
-lavfi.psnr.mse.b=28.317383
-lavfi.psnr.psnr.b=33.610271
+lavfi.psnr.mse.r=36.699280
+lavfi.psnr.psnr.r=32.484230
+lavfi.psnr.mse.g=33.508362
+lavfi.psnr.psnr.g=32.879272
+lavfi.psnr.mse.b=27.947266
+lavfi.psnr.psnr.b=33.667412
 lavfi.psnr.mse.a=0.000183
 lavfi.psnr.psnr.a=85.503792
-lavfi.psnr.mse_avg=24.615585
-lavfi.psnr.psnr_avg=34.218700
+lavfi.psnr.mse_avg=24.538773
+lavfi.psnr.psnr_avg=34.232277