[FFmpeg-cvslog] wavpack: Fix 32-bit clipping

[Derek Buitenhuis]

ffmpeg | branch: master | Derek Buitenhuis <derek.buitenhuis at gmail.com> | Fri Dec 16 13:31:29 2011 -0500| [bb9747c8eee134f2bf6058d368f8cbc799f4b7d3] | committer: Anton Khirnov

wavpack: Fix 32-bit clipping

In the case that (frame_flags & 0x03) == 3, hybrid_maxclip
may have had a signed integer overflow.

Signed-off-by: Derek Buitenhuis <derek.buitenhuis at gmail.com>
Signed-off-by: Anton Khirnov <anton at khirnov.net>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=bb9747c8eee134f2bf6058d368f8cbc799f4b7d3
---

 libavcodec/wavpack.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/wavpack.c b/libavcodec/wavpack.c
index 5358967..3cf5986 100644
--- a/libavcodec/wavpack.c
+++ b/libavcodec/wavpack.c
@@ -408,7 +408,7 @@ static inline int wv_get_value_integer(WavpackFrameContext *s, uint32_t *crc, in
     bit = (((S + bit) << s->shift) - bit) << s->post_shift;
 
     if(s->hybrid)
-        bit = av_clip(bit, -s->hybrid_maxclip, s->hybrid_maxclip - 1);
+        bit = av_clip(bit, -s->hybrid_maxclip - 1, s->hybrid_maxclip);
 
     return bit;
 }
@@ -798,7 +798,7 @@ static int wavpack_decode_block(AVCodecContext *avctx, int block_no,
     s->joint = s->frame_flags & WV_JOINT_STEREO;
     s->hybrid = s->frame_flags & WV_HYBRID_MODE;
     s->hybrid_bitrate = s->frame_flags & WV_HYBRID_BITRATE;
-    s->hybrid_maxclip = 1 << ((((s->frame_flags & 0x03) + 1) << 3) - 1);
+    s->hybrid_maxclip = (1LL << ((((s->frame_flags & 0x03) + 1) << 3) - 1)) - 1;
     s->post_shift = 8 * (bpp-1-(s->frame_flags&0x03)) + ((s->frame_flags >> 13) & 0x1f);
     s->CRC = AV_RL32(buf); buf += 4;
     if(wc->mkv_mode)