[FFmpeg-devel] [PATCH] adpcm: Reset the ssd back to zero more often

[Martin Storsjo]

If using very large trellis sizes (e.g. -trellis 15), the frontier
is so large that the difference between the best and the worst
trellis node in the frontier is large enough to cause wraparound.

Resetting at (1<<20) is enough to avoid the issue at -trellis 16
with the sample I'm testing, therefore doing the resets at (1<<18)
to have some extra safety margin.

This doesn't incur any noticeable slowdown.
---
I noticed this while testing the optimizations that Michael suggested
(using a hash to find out which samples to skip), which made the
encoder fast enough to actually be able to try out the maximum
trellis levels.

 libavcodec/adpcm.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/libavcodec/adpcm.c b/libavcodec/adpcm.c
index 56eb602..f6e3cb7 100644
--- a/libavcodec/adpcm.c
+++ b/libavcodec/adpcm.c
@@ -444,7 +444,7 @@ static void adpcm_compress_trellis(AVCodecContext *avctx, const short *samples,
         nodes_next = u;
 
         // prevent overflow
-        if(nodes[0]->ssd > (1<<28)) {
+        if(nodes[0]->ssd > (1<<18)) {
             for(j=1; j<frontier && nodes[j]; j++)
                 nodes[j]->ssd -= nodes[0]->ssd;
             nodes[0]->ssd = 0;
-- 
1.7.3.1