[FFmpeg-cvslog] avutil/mathematics: make av_gcd more robust

Ganesh Ajjanagadde git at videolan.org
Fri Oct 30 00:14:45 CET 2015


ffmpeg | branch: master | Ganesh Ajjanagadde <gajjanagadde at gmail.com> | Wed Oct 28 22:02:25 2015 -0400| [b7fb7c4542af63fea433a5417e4efe2d8c4422f6] | committer: Ganesh Ajjanagadde

avutil/mathematics: make av_gcd more robust

This ensures that no undefined behavior is invoked, while retaining
identical return values in all cases and at no loss of performance
(identical asm on clang and gcc).
Essentially, this patch exchanges undefined behavior with implementation
defined behavior, a strict improvement.

Rationale:
1. The ideal solution is to have the return type a uint64_t. This
unfortunately requires an API change.
2. The only pathological behavior happens if both arguments are
INT64_MIN, to the best of my knowledge. In such a case, the
implementation defined behavior is invoked in the sense that UINT64_MAX
is interpreted as INT64_MIN, which any reasonable implementation will
do. In any case, any usage where both arguments are INT64_MIN is a
fuzzer anyway.
3. Alternatives of checking, etc require branching and lose performance
for no concrete gain - no client cares about av_gcd's actual value when
both args are INT64_MIN. Even if it did, on sane platforms (e.g all the
ones FFmpeg cares about), it produces a correct gcd, namely INT64_MIN.

Reviewed-by: Michael Niedermayer <michael at niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde at gmail.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=b7fb7c4542af63fea433a5417e4efe2d8c4422f6
---

 libavutil/mathematics.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavutil/mathematics.c b/libavutil/mathematics.c
index 16e4eba..fde460c 100644
--- a/libavutil/mathematics.c
+++ b/libavutil/mathematics.c
@@ -52,7 +52,7 @@ int64_t av_gcd(int64_t a, int64_t b) {
         v -= u;
         v >>= ff_ctzll(v);
     }
-    return u << k;
+    return (uint64_t)u << k;
 }
 
 int64_t av_rescale_rnd(int64_t a, int64_t b, int64_t c, enum AVRounding rnd)



More information about the ffmpeg-cvslog mailing list