[FFmpeg-cvslog] avcodec/tiff: Fix default white level

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Thu Jul 16 22:42:14 2020 +0200| [d54c24acde88a214489d5ef410982eedac7ffc29] | committer: Michael Niedermayer

avcodec/tiff: Fix default white level

According to the spec bits per sample should be used

Fix invalid shift with bpp=32
Fixes: shift exponent 32 is too large for 32-bit type 'unsigned int'
Fixes: 23507/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TIFF_fuzzer-4815432665268224

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=d54c24acde88a214489d5ef410982eedac7ffc29
---

 libavcodec/tiff.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/libavcodec/tiff.c b/libavcodec/tiff.c
index 7d0a78cb3c..359d613bb3 100644
--- a/libavcodec/tiff.c
+++ b/libavcodec/tiff.c
@@ -1895,8 +1895,14 @@ again:
     if (is_dng) {
         int bps;
 
+        if (s->bpp % s->bppcount)
+            return AVERROR_INVALIDDATA;
+        bps = s->bpp / s->bppcount;
+        if (bps < 8 || bps > 32)
+            return AVERROR_INVALIDDATA;
+
         if (s->white_level == 0)
-            s->white_level = (1 << s->bpp) - 1; /* Default value as per the spec */
+            s->white_level = (1LL << bps) - 1; /* Default value as per the spec */
 
         if (s->white_level <= s->black_level) {
             av_log(avctx, AV_LOG_ERROR, "BlackLevel (%"PRId32") must be less than WhiteLevel (%"PRId32")\n",
@@ -1904,11 +1910,6 @@ again:
             return AVERROR_INVALIDDATA;
         }
 
-        if (s->bpp % s->bppcount)
-            return AVERROR_INVALIDDATA;
-        bps = s->bpp / s->bppcount;
-        if (bps < 8 || bps > 32)
-            return AVERROR_INVALIDDATA;
         if (s->planar)
             return AVERROR_PATCHWELCOME;
     }