[FFmpeg-devel] [PATCH] lavc/dvdsubenc: improve color distance function.

[Clément Bœsch]

On Tue, Jan 01, 2013 at 05:23:27PM +0100, Nicolas George wrote:
> Consider the color space as an hypercone with apex alpha=0
> and base alpha=1 instead of an hypercube.
> Make the encoder consider very transparent colors more similar
> even if the hue is very different.
> This corresponds roughly to using the alpha as a weight for the
> color difference.
> Only 4 bits of alpha are used, because this is what dvdsub uses,
> and it avoids overflows.
> 
> Fix trac ticket #2005.
> 
> Signed-off-by: Nicolas George <nicolas.george at normalesup.org>
> ---
>  libavcodec/dvdsubenc.c |   11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/dvdsubenc.c b/libavcodec/dvdsubenc.c
> index bf54aa2..cd041c3 100644
> --- a/libavcodec/dvdsubenc.c
> +++ b/libavcodec/dvdsubenc.c
> @@ -94,10 +94,14 @@ static void dvd_encode_rle(uint8_t **pq,
>  static int color_distance(uint32_t a, uint32_t b)
>  {
>      int r = 0, d, i;
> +    int alpha_a = 8, alpha_b = 8;
>  

Why not 0x0f?

> -    for (i = 0; i < 32; i += 8) {
> -        d = ((a >> i) & 0xFF) - ((b >> i) & 0xFF);
> +    for (i = 24; i >= 0; i -= 8) {
> +        d = alpha_a * (int)((a >> i) & 0xFF) -
> +            alpha_b * (int)((b >> i) & 0xFF);
>          r += d * d;
> +        alpha_a = a >> 28;
> +        alpha_b = b >> 28;
>      }
>      return r;
>  }
> @@ -130,7 +134,8 @@ static void count_colors(AVCodecContext *avctx, unsigned hits[33],
>          if (match) {
>              best_d = INT_MAX;
>              for (j = 0; j < 16; j++) {
> -                d = color_distance(color & 0xFFFFFF, dvdc->global_palette[j]);
> +                d = color_distance(0xFF000000 | color,
> +                                   0xFF000000 | dvdc->global_palette[j]);
>                  if (d < best_d) {
>                      best_d = d;
>                      best_j = j;

Otherwise LGTM, thanks

-- 
Clément B.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20130101/e55dada3/attachment.asc>