[FFmpeg-devel] [Patch] vf_edgedetect: properly implement double_threshold()

[Valery Kot]

vf_edgedetect video filter implements Canny algorithm
(https://en.wikipedia.org/wiki/Canny_edge_detector)

Important part of this algo is the double threshold step: pixels above
"high" threshold being kept, pixels below "low" threshold dropped,
pixels in between kept if they are attached to "high" pixels.

This is implemented in the double_threshold() function. However,
condition to start checking attached pixels, as it is now and as it
was in FFmpeg since 2012, only allows checking on the boundary, not
inside the video. It is a very lucky coincidence that those boundary
pixels are always 0, otherwise following lines would be reading
outside of the buffer.

As it is now, double_threshold() only implements "high" thresholding.
As a result, edges are either noisy or fragmented, depending on "high"
threshold selection; "low" threshold is simply ignored.

Attached one char patch fixes this.

Please review.

Valery
-------------- next part --------------
From b78f5960736de52d1c8e41bd598a465092c1de60 Mon Sep 17 00:00:00 2001
From: vkot <valery.kot at kinetiq.tv>
Date: Fri, 19 Jun 2020 16:57:13 +0200
Subject: [PATCH] vf_edgedetect: properly implement double_threshold()

---
 libavfilter/vf_edgedetect.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavfilter/vf_edgedetect.c b/libavfilter/vf_edgedetect.c
index a5614ea63b..df8afbd532 100644
--- a/libavfilter/vf_edgedetect.c
+++ b/libavfilter/vf_edgedetect.c
@@ -294,7 +294,7 @@ static void double_threshold(int low, int high, int w, int h,
                 continue;
             }
 
-            if ((!i || i == w - 1 || !j || j == h - 1) &&
+            if (!(!i || i == w - 1 || !j || j == h - 1) &&
                 src[i] > low &&
                 (src[-src_linesize + i-1] > high ||
                  src[-src_linesize + i  ] > high ||
-- 
2.26.2.windows.1