[FFmpeg-devel] [PATCH 2/4] zmbvenc: ensure mx, my, xored are always set together in zmbv_me()

[matthew.w.fearnley at gmail.com]

From: Matthew Fearnley <matthew.w.fearnley at gmail.com>

Store the value of *xored computed within block_cmp() in a local variable,
and only update the *xored parameter at the same time as *mx,*my are set.
This ensures that the value of *xored is accurate for the value of *mx,*my
whenever the function ends.

Note that the local variable is not needed in the intial block_cmp for (0,0)
because *mx,*my and *xored are all set there.

The previous logic worked by exiting early if ever !*xored, but put implicit
requirements on block_cmp() to:
- always return 0 if !xored
- never return a negative value
---
 libavcodec/zmbvenc.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/libavcodec/zmbvenc.c b/libavcodec/zmbvenc.c
index 2f041dae32..0e8ee5ce31 100644
--- a/libavcodec/zmbvenc.c
+++ b/libavcodec/zmbvenc.c
@@ -100,6 +100,7 @@ static int zmbv_me(ZmbvEncContext *c, uint8_t *src, int sstride, uint8_t *prev,
                    int pstride, int x, int y, int *mx, int *my, int *xored)
 {
     int dx, dy, tx, ty, tv, bv, bw, bh;
+    int txored;
 
     *mx = *my = 0;
     bw = FFMIN(ZMBV_BLOCK, c->avctx->width - x);
@@ -111,11 +112,12 @@ static int zmbv_me(ZmbvEncContext *c, uint8_t *src, int sstride, uint8_t *prev,
             if(tx == x && ty == y) continue; // we already tested this block
             dx = tx - x;
             dy = ty - y;
-            tv = block_cmp(c, src, sstride, prev + dx + dy * pstride, pstride, bw, bh, xored);
+            tv = block_cmp(c, src, sstride, prev + dx + dy * pstride, pstride, bw, bh, &txored);
             if(tv < bv){
                  bv = tv;
                  *mx = dx;
                  *my = dy;
+                 *xored = txored;
                  if(!bv) return 0;
              }
          }
-- 
2.17.1