[FFmpeg-devel] [PATCH] Fix off-by-one error in dvdsubdec.c

[Oliver Fromme]

I recently noticed that the lowest pixel rows from VOBSUB
subtitles are missing.  This is particularly ugly when there
are letters with an outline border, it looks as if the lower
border of the characters is cut off.

Upon inspecting libavcodec/dvdsubdec.c I noticed that there
is a very obvious off-by-one error that is causing this
behaviour.

The patch below fixes the off-by one error.

Also, the patch improves the debugging function that creates
PPM files from the subtitles:  Instead of ignoring the alpha
channel completely, it now uses the alpha channel to overlay
the subtitle onto a green background.  The color of the
background can be changed easily in the source if required.
This was very helpful when I debugged the off-by-one issue.

Best regards
   Oliver

PS:  I'm sorry I'm not using git, but hopefully a unified diff
will be sufficient, too.  Here's a suggested commit message:

Fix an off-by-one error that causes the height of decoded
subtitles to be too small, thus cutting off the lowest row
of pixels.

Also, improve the debugging function for saving subtitles
to PPM files: Actually use the alpha channel.

--- a/libavcodec/dvdsubdec.c	2014-04-05 19:20:05.000000000 +0200
+++ b/libavcodec/dvdsubdec.c	2014-07-03 15:44:05.000000000 +0200
@@ -345,7 +345,7 @@
             w = x2 - x1 + 1;
             if (w < 0)
                 w = 0;
-            h = y2 - y1;
+            h = y2 - y1 + 1;
             if (h < 0)
                 h = 0;
             if (w > 0 && h > 0) {
@@ -462,10 +462,13 @@
 }
 
 #ifdef DEBUG
+#define ALPHA_MIX(A,BACK,FORE) (((255-(A)) * (BACK) + (A) * (FORE)) / 255)
 static void ppm_save(const char *filename, uint8_t *bitmap, int w, int h,
                      uint32_t *rgba_palette)
 {
-    int x, y, v;
+    int x, y, alpha;
+    uint32_t v;
+    int back[3] = {0, 255, 0};  /* green background */
     FILE *f;
 
     f = fopen(filename, "w");
@@ -480,9 +483,10 @@
     for(y = 0; y < h; y++) {
         for(x = 0; x < w; x++) {
             v = rgba_palette[bitmap[y * w + x]];
-            putc((v >> 16) & 0xff, f);
-            putc((v >> 8) & 0xff, f);
-            putc((v >> 0) & 0xff, f);
+            alpha = v >> 24;
+            putc(ALPHA_MIX(alpha, back[0], (v >> 16) & 0xff), f);
+            putc(ALPHA_MIX(alpha, back[1], (v >> 8) & 0xff), f);
+            putc(ALPHA_MIX(alpha, back[2], (v >> 0) & 0xff), f);
         }
     }
     fclose(f);
@@ -552,7 +556,7 @@
             sub->start_display_time,
             sub->end_display_time);
     ppm_save(ppm_name, sub->rects[0]->pict.data[0],
-             sub->rects[0]->w, sub->rects[0]->h, sub->rects[0]->pict.data[1]);
+             sub->rects[0]->w, sub->rects[0]->h, (uint32_t*) sub->rects[0]->pict.data[1]);
     }
 #endif