[FFmpeg-devel] [PATCH] Do not access beyond the end of the palette in pcxenc

Daniel Verkamp daniel
Wed May 13 03:46:42 CEST 2009


Hi,

Attached is a patch to fix the monoblack case of pcxenc; it was
accessing beyond the end of the array.

An alternate fix (second patch) would involve just making
monoblack_pal 16 entries long instead of 2.

I don't have any real preference for one patch over the other, but the
code for the second fix stays slightly cleaner, at the expense of data
size vs. code size.

(As an aside, it seems to me that AVFrame.data[1] could be set with a
black and white palette for monoblack/monowhite pixfmts to avoid
duplication of this kind of special-case code in codecs.)

Thanks,
-- Daniel Verkamp
-------------- next part --------------
>From 9a9608206d3989d881218555d30e83c30a20a9f6 Mon Sep 17 00:00:00 2001
From: Daniel Verkamp <daniel at drv.nu>
Date: Tue, 12 May 2009 20:43:13 -0500
Subject: [PATCH] Do not access beyond the end of the palette

---
 libavcodec/pcxenc.c |    8 ++++++--
 1 files changed, 6 insertions(+), 2 deletions(-)

diff --git a/libavcodec/pcxenc.c b/libavcodec/pcxenc.c
index 36f7d1d..ea51774 100644
--- a/libavcodec/pcxenc.c
+++ b/libavcodec/pcxenc.c
@@ -154,8 +154,12 @@ static int pcx_encode_frame(AVCodecContext *avctx,
     bytestream_put_le16(&buf, avctx->height - 1);   // y max
     bytestream_put_le16(&buf, 0);                   // horizontal DPI
     bytestream_put_le16(&buf, 0);                   // vertical DPI
-    for (i = 0; i < 16; i++)
-        bytestream_put_be24(&buf, pal ? pal[i] : 0);// palette (<= 16 color only)
+    i = 0;
+    if (pal)
+        for (; i < FFMIN(1 << (bpp * nplanes), 16); i++)
+            bytestream_put_be24(&buf, pal[i]);      // palette (<= 16 color only)
+    for (; i < 16; i++)
+        bytestream_put_be24(&buf, 0);
     bytestream_put_byte(&buf, 0);                   // reserved
     bytestream_put_byte(&buf, nplanes);             // number of planes
     bytestream_put_le16(&buf, line_bytes);          // scanline plane size in bytes
-- 
1.6.2.5
-------------- next part --------------
>From 9ceee433779c287ac67cb19b13dc36b5a489323d Mon Sep 17 00:00:00 2001
From: Daniel Verkamp <daniel at drv.nu>
Date: Tue, 12 May 2009 20:47:52 -0500
Subject: [PATCH] Make the monoblack_pal larger to avoid access beyond end

---
 libavcodec/pcxenc.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/libavcodec/pcxenc.c b/libavcodec/pcxenc.c
index 36f7d1d..5e7c1e4 100644
--- a/libavcodec/pcxenc.c
+++ b/libavcodec/pcxenc.c
@@ -33,7 +33,7 @@ typedef struct PCXContext {
     AVFrame picture;
 } PCXContext;
 
-static const uint32_t monoblack_pal[] = { 0x000000, 0xFFFFFF };
+static const uint32_t monoblack_pal[16] = { 0x000000, 0xFFFFFF };
 
 static av_cold int pcx_encode_init(AVCodecContext *avctx)
 {
-- 
1.6.2.5



More information about the ffmpeg-devel mailing list