[FFmpeg-cvslog] avcodec/adxenc: Avoid undefined left shift of negative numbers

[Andreas Rheinhardt]

ffmpeg | branch: release/3.2 | Andreas Rheinhardt <andreas.rheinhardt at gmail.com> | Mon Jan 20 20:20:42 2020 +0100| [9d7bc32e1c9533f15fa23885ae1a04a5efe5d8d5] | committer: Andreas Rheinhardt

avcodec/adxenc: Avoid undefined left shift of negative numbers

Replace "((a << shift) + b) >> shift" by "a + (b >> shift)". This avoids
a left shift which also happens to trigger undefined behaviour in case "a"
is negative. This affected the FATE-tests acodec-adpcm-adx and
acodec-adpcm-adx-trellis; it also fixes ticket #8008.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at gmail.com>
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 59a9d65e0d790821f88527a82569f56eb2f8a9be)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at gmail.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=9d7bc32e1c9533f15fa23885ae1a04a5efe5d8d5
---

 libavcodec/adxenc.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libavcodec/adxenc.c b/libavcodec/adxenc.c
index f1ba5911b3..77f6bf0487 100644
--- a/libavcodec/adxenc.c
+++ b/libavcodec/adxenc.c
@@ -48,7 +48,7 @@ static void adx_encode(ADXContext *c, uint8_t *adx, const int16_t *wav,
     s2 = prev->s2;
     for (i = 0, j = 0; j < 32; i += channels, j++) {
         s0 = wav[i];
-        d = ((s0 << COEFF_BITS) - c->coeff[0] * s1 - c->coeff[1] * s2) >> COEFF_BITS;
+        d = s0 + ((-c->coeff[0] * s1 - c->coeff[1] * s2) >> COEFF_BITS);
         if (max < d)
             max = d;
         if (min > d)
@@ -79,13 +79,13 @@ static void adx_encode(ADXContext *c, uint8_t *adx, const int16_t *wav,
     s1 = prev->s1;
     s2 = prev->s2;
     for (i = 0, j = 0; j < 32; i += channels, j++) {
-        d = ((wav[i] << COEFF_BITS) - c->coeff[0] * s1 - c->coeff[1] * s2) >> COEFF_BITS;
+        d = wav[i] + ((-c->coeff[0] * s1 - c->coeff[1] * s2) >> COEFF_BITS);
 
         d = av_clip_intp2(ROUNDED_DIV(d, scale), 3);
 
         put_sbits(&pb, 4, d);
 
-        s0 = ((d << COEFF_BITS) * scale + c->coeff[0] * s1 + c->coeff[1] * s2) >> COEFF_BITS;
+        s0 = d * scale + ((c->coeff[0] * s1 + c->coeff[1] * s2) >> COEFF_BITS);
         s2 = s1;
         s1 = s0;
     }