[FFmpeg-cvslog] avcodec/mjpegdec: bayer and rct are incompatible

[Michael Niedermayer]

ffmpeg | branch: release/5.1 | Michael Niedermayer <michael at niedermayer.cc> | Sat Aug 13 22:47:31 2022 +0200| [9eb36ab0a1135327982afb8fa6f23e60cf50906c] | committer: Michael Niedermayer

avcodec/mjpegdec: bayer and rct are incompatible

Fixes: out of array read
Fixes: 49434/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TIFF_fuzzer-5208501080686592

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit a44f5a521227adc7be2f78b411f56da1a4d98704)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=9eb36ab0a1135327982afb8fa6f23e60cf50906c
---

 libavcodec/mjpegdec.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/libavcodec/mjpegdec.c b/libavcodec/mjpegdec.c
index 32874a5a19..869aee425e 100644
--- a/libavcodec/mjpegdec.c
+++ b/libavcodec/mjpegdec.c
@@ -1092,6 +1092,10 @@ static int ljpeg_decode_rgb_scan(MJpegDecodeContext *s, int nb_components, int p
         return AVERROR_INVALIDDATA;
     if (s->v_max != 1 || s->h_max != 1 || !s->lossless)
         return AVERROR_INVALIDDATA;
+    if (s->bayer) {
+        if (s->rct || s->pegasus_rct)
+            return AVERROR_INVALIDDATA;
+    }
 
 
     s->restart_count = s->restart_interval;
@@ -1942,6 +1946,8 @@ static int mjpeg_decode_app(MJpegDecodeContext *s)
         }
 
         len -= 9;
+        if (s->bayer)
+            goto out;
         if (s->got_picture)
             if (rgb != s->rgb || pegasus_rct != s->pegasus_rct) {
                 av_log(s->avctx, AV_LOG_WARNING, "Mismatching LJIF tag\n");