[FFmpeg-devel] [PATCH] avcodec/texturedspenc: Fix indexing in color distribution determination

[James Almer]

On 1/1/2017 8:28 PM, Michael Niedermayer wrote:
> Fixes CID1396405
> 
> Untested except fate which does not test this, please someone test this

The encoder seems to need libsnappy, so no FATE test can be added.

CCing Vittorio since he wrote this. He may be able to test and/or
review.

> 
> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> ---
>  libavcodec/texturedspenc.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/texturedspenc.c b/libavcodec/texturedspenc.c
> index 8b2863033b..5171c69d82 100644
> --- a/libavcodec/texturedspenc.c
> +++ b/libavcodec/texturedspenc.c
> @@ -255,11 +255,11 @@ static void optimize_colors(const uint8_t *block, ptrdiff_t stride,
>  
>          muv = minv = maxv = bp[0];
>          for (y = 0; y < 4; y++) {
> -            for (x = 4; x < 4; x += 4) {
> +            for (x = 0; x < 4; x++) {
>                  muv += bp[x * 4 + y * stride];
> -                if (bp[x] < minv)
> +                if (bp[x * 4 + y * stride] < minv)
>                      minv = bp[x * 4 + y * stride];
> -                else if (bp[x] > maxv)
> +                else if (bp[x * 4 + y * stride] > maxv)
>                      maxv = bp[x * 4 + y * stride];
>              }
>          }
>