[FFmpeg-cvslog] avcodec/pnmdec: Use 64bit for input size check

[Michael Niedermayer]

ffmpeg | branch: release/4.3 | Michael Niedermayer <michael at niedermayer.cc> | Thu Jul 18 21:12:54 2024 +0200| [da2240ccbecdf3fe8df6465ae3de7a1b565a2c68] | committer: Michael Niedermayer

avcodec/pnmdec: Use 64bit for input size check

Fixes: out of array read
Fixes: poc3

Reported-by: VulDB CNA Team
Found-by: CookedMelon
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 3faadbe2a27e74ff5bb5f7904ec27bb1f5287dc8)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=da2240ccbecdf3fe8df6465ae3de7a1b565a2c68
---

 libavcodec/pnmdec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/pnmdec.c b/libavcodec/pnmdec.c
index 9add5cfc84..8540215fb2 100644
--- a/libavcodec/pnmdec.c
+++ b/libavcodec/pnmdec.c
@@ -256,7 +256,7 @@ static int pnm_decode_frame(AVCodecContext *avctx, void *data,
         }
         break;
     case AV_PIX_FMT_GBRPF32:
-        if (avctx->width * avctx->height * 12 > s->bytestream_end - s->bytestream)
+        if (avctx->width * avctx->height * 12LL > s->bytestream_end - s->bytestream)
             return AVERROR_INVALIDDATA;
         scale = 1.f / s->scale;
         if (s->endian) {