[FFmpeg-devel] [PATCH] avformat/cache: Avoid int-overflow in cache compare function

[Bryan Huh]

cache protocol indexes its cache using AVTreeNodes which require a cmp
function for inserting and searching new cache-entries. This cmp
function expects a 32-bit int return value (negative, zero, or positive)
but the cache cmp function returns an int64_t which can overflow the
int, giving negative numbers for when it should be positive, vice versa.
This manifests itself only for very large files (e.g. 4GB+)
---
 libavformat/cache.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/libavformat/cache.c b/libavformat/cache.c
index 31f63e6..5631586 100644
--- a/libavformat/cache.c
+++ b/libavformat/cache.c
@@ -67,7 +67,8 @@ typedef struct Context {
 
 static int cmp(const void *key, const void *node)
 {
-    return (*(const int64_t *) key) - ((const CacheEntry *) node)->logical_pos;
+    int64_t diff = (*(const int64_t *) key) - ((const CacheEntry *) node)->logical_pos;
+    return diff > 0 ? 1 : diff < 0 ? -1 : 0;
 }
 
 static int cache_open(URLContext *h, const char *arg, int flags, AVDictionary **options)
-- 
1.7.1