[FFmpeg-devel] [PATCH]Fix odd RGB555 camstudio, v2

[Reimar Döffinger]

On Fri, Apr 20, 2012 at 09:24:14PM +0200, Reimar Döffinger wrote:
> On Fri, Apr 20, 2012 at 07:13:40PM +0000, Carl Eugen Hoyos wrote:
> > Michael Niedermayer <michaelni <at> gmx.at> writes:
> > 
> > > iam fine with any solution
> > > i just wanted to make sure this difference in handling wasnt
> > > unintended
> > 
> > So is the patch ok?
> 
> IMHO not quite: Either we do want to avoid writing in the
> area between width and stride, then you would need to change
> your patch to work like the 24 bpp case, since currently
> it writes more.
> Hm, I think that is an actual bug, if an application would
> provide a buffer with exactly linesize == 2*width then
> after your change I believe the decoder would be writing
> outside the buffer.

To be more specific, something like below (completely untested):
(and the FFALIGN should probably be moved right into the
copy_frame_default/add_frame_default and the extra argument removed)
--- a/libavcodec/cscd.c
+++ b/libavcodec/cscd.c
@@ -61,9 +61,9 @@ static void add_frame_default(AVFrame *f, const uint8_t *src, int src_stride,
 }
 
 #if !HAVE_BIGENDIAN
-#define copy_frame_16(f, s, l, h) copy_frame_default(f, s, l, l, h)
+#define copy_frame_16(f, s, l, h) copy_frame_default(f, s, FFALIGN(l, 4), l, h)
 #define copy_frame_32(f, s, l, h) copy_frame_default(f, s, l, l, h)
-#define add_frame_16(f, s, l, h) add_frame_default(f, s, l, l, h)
+#define add_frame_16(f, s, l, h) add_frame_default(f, s, FFALIGN(l, 4), l, h)
 #define add_frame_32(f, s, l, h) add_frame_default(f, s, l, l, h)
 #else
 static void copy_frame_16(AVFrame *f, const uint8_t *src,
@@ -78,6 +78,7 @@ static void copy_frame_16(AVFrame *f, const uint8_t *src,
           src += 2;
           dst += 2;
         }
+        src += 2 * (linelen & 1);
         dst -= f->linesize[0] + linelen;
     }
 }
@@ -112,6 +113,7 @@ static void add_frame_16(AVFrame *f, const uint8_t *src,
           src += 2;
           dst += 2;
         }
+        src += 2 * (linelen & 1);
         dst -= f->linesize[0] + linelen;
     }
 }
@@ -235,9 +237,7 @@ static av_cold int decode_init(AVCodecContext *avctx) {
     c->pic.data[0] = NULL;
     c->linelen = avctx->width * avctx->bits_per_coded_sample / 8;
     c->height = avctx->height;
-    stride = c->linelen;
-    if (avctx->bits_per_coded_sample == 24)
-        stride = FFALIGN(stride, 4);
+    stride = FFALIGN(c->linelen, 4);
     c->decomp_size = c->height * stride;
     c->decomp_buf = av_malloc(c->decomp_size + AV_LZO_OUTPUT_PADDING);
     if (!c->decomp_buf) {