[Ffmpeg-devel] [PATCH] fix DV encoder VLC bit accounting

[Dan Maas]

This patch fixes VLC bit accounting in the DV encoder.  
(the "AC EOB marker is absent" message and associated image
corruption)

Details: When dv_guess_qnos() increases the quantization step for a
DCT block area, some of its AC coefficients may become zero. If this
happens, it increases the length of the corresponding run of zero
bits. VLC code lengths depend on run length, so this change can affect
the RLE-encoded size of the following block areas. Without the patch,
dv_guess_qnos() ignores this possibility, so the final RLE-encoded
size of the block may be larger than the sum of bit_size[]. This can
cause the AC bitstream to overflow the allotted number of bits,
corrupting the block.

(Roman, this patch is slightly more optimized than the version I sent
you earlier. It doesn't re-compute the VLC length unless at least one
AC coefficient was quantized to zero in a preceding area.)

Regards,
Dan
-------------- next part --------------
Index: libavcodec/dv.c
===================================================================
RCS file: /cvsroot/ffmpeg/ffmpeg/libavcodec/dv.c,v
retrieving revision 1.71
diff -u -p -u -r1.71 dv.c
--- libavcodec/dv.c	5 Feb 2006 13:35:16 -0000	1.71
+++ libavcodec/dv.c	20 Feb 2006 05:00:00 -0000
@@ -740,23 +740,40 @@ static inline void dv_guess_qnos(EncBloc
           qnos[i]--;
           size[i] = 0;
           for (j=0; j<6; j++, b++) {
-             for (a=0; a<4; a++) {
-                if (b->area_q[a] != dv_quant_shifts[qnos[i] + dv_quant_offset[b->cno]][a]) {
-                    b->bit_size[a] = 1; // 4 areas 4 bits for EOB :)
-                    b->area_q[a]++;
-                    prev= b->prev[a];
-                    for (k= b->next[prev] ; k<mb_area_start[a+1]; k= b->next[k]) {
-                       b->mb[k] >>= 1;
-                       if (b->mb[k]) {
-                           b->bit_size[a] += dv_rl2vlc_size(k - prev - 1, b->mb[k]);
-                           prev= k;
-                       } else {
-                           b->next[prev] = b->next[k];
-                       }
-                    }
-                    b->prev[a+1]= prev;
-                }
-                size[i] += b->bit_size[a];
+	      int update_vlc = 0;
+	      for (a=0; a<4; a++) {
+		 if (b->area_q[a] != dv_quant_shifts[qnos[i] + dv_quant_offset[b->cno]][a]) {
+		     /* update quantizations and bit_size */
+		     b->bit_size[a] = 1; // 4 areas 4 bits for EOB :)
+		     b->area_q[a]++;
+		     prev = b->prev[a];
+		     for (k= b->next[prev] ; k<mb_area_start[a+1]; k= b->next[k]) {
+			 b->mb[k] >>= 1;
+			 if (b->mb[k]) {
+			     b->bit_size[a] += dv_rl2vlc_size(k - prev - 1, b->mb[k]);
+			     prev = k;
+			 } else {
+			     /* coefficient became zero */
+			     b->next[prev] = b->next[k];
+			     update_vlc = 1;
+			 }
+		     }
+		     b->prev[a+1] = prev;
+		 } else if (update_vlc) {
+		     /* update bit_size but not quantizations */
+		     /* we must do this since RLE code lengths depend on run length */
+		     b->bit_size[a] = 1; // 4 areas 4 bits for EOB :)
+		     prev = b->prev[a];
+		     for (k = b->next[prev] ; k < mb_area_start[a+1]; k = b->next[k]) {
+			 if (b->mb[k]) {
+			     b->bit_size[a] += dv_rl2vlc_size(k - prev - 1, b->mb[k]);
+			     prev = k;
+			 }
+		     }
+		     b->prev[a+1] = prev;
+		 }
+
+		 size[i] += b->bit_size[a];
              }
           }
        }