[FFmpeg-cvslog] avcodec/h263: Fix global-buffer-overflow with noout flag2 set

[Andreas Rheinhardt]

ffmpeg | branch: release/4.4 | Andreas Rheinhardt <andreas.rheinhardt at outlook.com> | Sun Nov 21 01:57:41 2021 +0100| [f82aaea3ff35c171ffe5b894026eacbc3c0574de] | committer: Andreas Rheinhardt

avcodec/h263: Fix global-buffer-overflow with noout flag2 set

h263_get_motion_length() forgot to take an absolute value;
as a consequence, a negative index was used to access an array.
This leads to potential crashes, but mostly it just accesses what
is to the left of ff_mvtab (unless one uses ASAN), thereby defeating
the purpose of the AV_CODEC_FLAG2_NO_OUTPUT because the sizes of
the returned packets differ from the sizes the encoder would actually
have produced.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at outlook.com>
(cherry picked from commit 9207dc3b0db368bb9cf5eb295cbc1129c2975e31)

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=f82aaea3ff35c171ffe5b894026eacbc3c0574de
---

 libavcodec/h263.h | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/libavcodec/h263.h b/libavcodec/h263.h
index 491f2e0aac..f5355e7ced 100644
--- a/libavcodec/h263.h
+++ b/libavcodec/h263.h
@@ -100,15 +100,16 @@ void ff_h263_encode_motion(PutBitContext *pb, int val, int f_code);
 
 
 static inline int h263_get_motion_length(int val, int f_code){
-    int l, bit_size, code;
+    int bit_size, code, sign;
 
     if (val == 0) {
         return ff_mvtab[0][1];
     } else {
         bit_size = f_code - 1;
         /* modulo encoding */
-        l= INT_BIT - 6 - bit_size;
-        val = (val<<l)>>l;
+        val  = sign_extend(val, 6 + bit_size);
+        sign = val >> 31;
+        val  = (val ^ sign) - sign; /* val = FFABS(val) */
         val--;
         code = (val >> bit_size) + 1;