[FFmpeg-cvslog] avcodec/jpeglsdec: Check Jpeg-LS LSE

[Michael Niedermayer]

ffmpeg | branch: release/6.0 | Michael Niedermayer <michael at niedermayer.cc> | Fri Dec 15 22:23:33 2023 +0100| [bda83cba9dcb3ec9416041466aa36eb420ed5770] | committer: Michael Niedermayer

avcodec/jpeglsdec: Check Jpeg-LS LSE

Fixes: signed integer overflow: 2147478526 + 33924 cannot be represented in type 'int'
Fixes: shift exponent 32 is too large for 32-bit type 'unsigned int'
Fixes: 64243/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEGLS_fuzzer-5195717848989696

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit c72a20f01a6122e1832f73801ea5f54b188abea3)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=bda83cba9dcb3ec9416041466aa36eb420ed5770
---

 libavcodec/jpeglsdec.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/libavcodec/jpeglsdec.c b/libavcodec/jpeglsdec.c
index ec163b8964..c245cf0279 100644
--- a/libavcodec/jpeglsdec.c
+++ b/libavcodec/jpeglsdec.c
@@ -382,6 +382,19 @@ int ff_jpegls_decode_picture(MJpegDecodeContext *s, int near,
     state->T3     = s->t3;
     state->reset  = s->reset;
     ff_jpegls_reset_coding_parameters(state, 0);
+
+    /* Testing parameters here, we cannot test in LSE or SOF because
+     * these interdepend and are allowed in either order
+     */
+    if (state->maxval >= (1<<state->bpp) ||
+        state->T1 > state->T2 ||
+        state->T2 > state->T3 ||
+        state->T3 > state->maxval ||
+        state->reset > FFMAX(255, state->maxval)) {
+        ret = AVERROR_INVALIDDATA;
+        goto end;
+    }
+
     ff_jpegls_init_state(state);
 
     if (s->bits <= 8)