[FFmpeg-cvslog] avcodec/dirac_vlc: Fix undefined shift

[Michael Niedermayer]

ffmpeg | branch: release/3.2 | Michael Niedermayer <michael at niedermayer.cc> | Wed Jul 19 01:43:24 2017 +0200| [c368f07565362423ad3387ab90d07c400aa50b39] | committer: Michael Niedermayer

avcodec/dirac_vlc: Fix undefined shift

Fixes: runtime error: shift exponent 64 is too large for 64-bit type 'residual' (aka 'unsigned long')
Fixes: 2674/clusterfuzz-testcase-minimized-4999700518273024

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 69e7daf6ce2a5893936ba18572c58180b29d67f9)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=c368f07565362423ad3387ab90d07c400aa50b39
---

 libavcodec/dirac_vlc.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/libavcodec/dirac_vlc.c b/libavcodec/dirac_vlc.c
index 336d22a182..773f720858 100644
--- a/libavcodec/dirac_vlc.c
+++ b/libavcodec/dirac_vlc.c
@@ -216,9 +216,14 @@ static void generate_offset_lut(DiracGolombLUT *lut, int off)
         INIT_RESIDUE(res);
         SET_RESIDUE(res, idx, LUT_BITS);
 
-        l->preamble      = CONVERT_TO_RESIDUE(res >> (RSIZE_BITS - off), off);
         l->preamble_bits = off;
-        l->sign = ((l->preamble >> (RSIZE_BITS - l->preamble_bits)) & 1) ? -1 : +1;
+        if (off) {
+            l->preamble  = CONVERT_TO_RESIDUE(res >> (RSIZE_BITS - off), off);
+            l->sign      = ((l->preamble >> (RSIZE_BITS - l->preamble_bits)) & 1) ? -1 : +1;
+        } else {
+            l->preamble  = 0;
+            l->sign = 1;
+        }
 
         search_for_golomb(l, res << off, LUT_BITS - off);
     }