q

se
i@
u
e

zh
@s
Fundamentals of Computer

qi
i
zh
Vision
qi

i
se

zh
i@

qi
Lecture 6：Edge Detection
u
se

zh
i@

qi

u
2023 spring

se
zh

i@
eu

zh
@s

qi
hi
 q

se
Contents

i@
u
• the origin and characteristics of edges

zh
@s
• detect the edge through the 1st derivative of the

qi
i
image zh
• detect the edge through the 2nd derivative of the
qi

i
se

zh
image

i@

qi
• effect of noise to edge detection and its solutions
u
se

zh
• Canny edge detection
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 q

se
Edge detection

i@
e u
• Goal: Identify sudden changes of intensity

zh
@s
(discontinuities) in an image

qi
i
zh
• Intuitively, most semantic and shape information from the
image can be encoded in the edges.
qi

i
• More compact than pixels

se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 Origin of edges

se
i@
u
Edges are caused by a variety of factors:

zh
@s

qi
i
zh
qi

i
surface normal discontinuity

se

zh
i@
depth discontinuity

qi
u
se

zh surface color disconinuity

i@

qi

u
se
illumination discontinuity
zh

i@
eu

zh
@s

qi
hi
 Characterizing edges

se
i@
u
• An edge is a place where a rapid change in the image

zh
@s
intensity is detected. first derivative

qi
i
image
zh intensity function
(along horizontal scanline)
qi

i
se

zh
i@

qi
u
se

zh edges correspond to
extrema or zero crossing of
i@

qi

u
derivative

se
zh

i@
eu

zh
@s

qi
hi

Second derivative
 Characterizing edges

se
i@
u
•Edge descriptors

zh
@s
– Edge position or center: the image position at

qi
which the edge is located.

i
zh
– Edge normal: unit vector in the direction of
qi

i
se
maximum intensity change.

zh
– Edge direction: unit vector along edge

i@

qi
u

(perpendicular to edge normal).

se

zh
– Edge strength or magnitude: local image contrast
i@

qi

u
along the normal.

se
zh

i@
eu

zh
@s

qi
hi
 q

se
Edge Detection

i@
u
We want edge detectors to return

zh
@s
• the position / location

qi
i
• the strength
zh
• and the orientation of the edge.
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
Edge Detection Using Derivatives

i@
e u

zh
@s
Often, points that lie on an edge are detected by:

qi
i
(1) Detecting the local maxima or minima of the first
zh
derivative.
qi

i
For example：gradient based edge detection

se

zh
i@

qi
u

(2) Detecting the zero-crossings of the second derivative.

se

zh
For example：Laplacian based edge detection or canny
i@

edge detection
qi

u
se
zh

i@
eu

zh
@s

qi
hi
 Gradient based edge detection

se
i@
u
• The gradient of an image gives us the direction and

zh
@s
magnitude of maximum intensity change at every pixel

qi
i
location in an image.
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
Q：How is this direction of gradient related to the
@s

qi
direction of the edge?
hi
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
fy

zh
i@ q
se
qi
zh u
i@ qi
fx

se zh
u i@
qi se
zh
i
 q
Derivatives with convolution

se
i@
u
For 2D function f(x,y), the partial derivative is:

zh
@s

qi
f ( x, y ) f ( x   , y )  f ( x, y )

i
zh  lim
x  0 
qi

i
se

zh
For discrete data, we can approximate using finite differences:

i@

qi
u

f ( x, y ) f ( x  1, y )  f ( x, y )
se

 zh
1
i@

x
qi

u
se
zh

The calculation of gradient at each pixel is normally done

i@
through a convolution.
eu

zh
Q：What would be the associated convolution filter for such a
@s

gradient calculation?
qi
hi
 q

se
i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

Q：Which one is the x directional derivative of the

zh
@s

image, upper or lower?

qi
hi
 q

se
Some common choices for gradient operator:

i@
e u

zh
@s

qi
i
f ( x, y) zh
qi

i
x

se

zh
i@

qi
u
se

f ( x, y)
zh
i@

qi

u
y

se
zh

i@
eu

zh
@s

qi
hi
 q

se
Steps of Gradient Based Edge Detection

i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
(i.e., sqrt is costly!)

i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 zh
i@
se

(c) |dy|
(b) |dx|
u

(d) |dx|+|dy|
hi qi
@s

(a)Smoothed image
zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 q

se
Edge Detection Using Derivatives

i@
e u

zh
Often, points that lie on an edge are detected

@s

qi
by:

i
zh
(1) Detecting the local maxima or minima of the first
derivative. For example：gradient based edge detection
qi

i
se

zh
(2) Detecting the zero-crossings of the second

i@

qi
u

derivative. For example：Laplacian based edge

se

detection zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
Laplacian filter

i@
u
0 0 0

zh
2
 f

@s
 f ( x  1)  f ( x  1)  2 f ( x ) 1 -2 1

qi
2
 x 0 0 0

i
zh +
qi

i
0 1 0

se

zh
2
 f
 f ( y  1)  f ( y  1)  2 f ( y ) 0 -2 0

i@

qi
2
 y
u

0 1 0
se

zh
i@

qi

u
se
2 2
 f  f
zh

2
 f  2  2 0 1 0

i@
eu

 x  y 1 -4 1
zh
@s

qi
0 1 0
hi
 q

se
Laplacian filter

i@
e u

zh
In order to detect edges along more directions， we

@s

qi
usual apply the Laplacian filter as

i
zh
qi

i
se

zh
1 1 1 1 4 1

i@

qi
u
se

1 -8 1 zh 4 -20 4
i@

qi

u
1 1 1 1 4 1

se
zh

i@
eu

zh
@s

qi
hi
 Steps of Laplacian Edge Detection

se
i@
e u

zh
@s

qi
2  fˆL  fˆ ( x , y ) * L ( x , y )

i
zh
(3) the zero-crossing poinst in fˆL are the candidate
qi

i
se
edge points.

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
2nd derivative method just gives the position of edge,
eu

while no magnitude, nor direction. zh

@s

qi
hi
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 q

se
Effects of noise

i@
e u

zh
@s

qi
• Second column:

i
zh
first derivative image
qi

i
se

zh
• Third column: second
derivative image

i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
Solution: smooth first

i@
eu

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi


Q：According to ( h  f ) , can you identify where the edge is?
x
 q

se
Maxima and minima of first derivative

i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi

Q： What’s the advantage of this solution over the last one?

 q

se
Maxima and minima of first derivative

i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi

Q： What’s the advantage of this solution over the last one?

 q

se
Solution: smooth first

i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
2
eu

(h  f )
x 2
zh
@s

qi
hi

Q： can you identify where the edge is?

 q

se
Zero crossing of the second derivative

i@
u
Consider  2
 h
2

(h  f )  ( ) f

zh
x 2
x 2

@s

qi
i
zh
qi

i
se

zh
i@

qi
u

Laplacian of Gaussian
se

zh operator
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi

Q：Where is the edge? What is the effect of a LoG kernel？

 q

se
Zero crossing of the second derivative

i@
u
 2
 h 2

zh
(h  f )  ( ) f

@s
x 2
x 2

qi
i
zh
qi

i
se

zh
i@

qi
Laplacian of Gaussian
u
se

operator
zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi

Q： What’s the advantage of this solution over the last one?

 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
Practice

se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 q

se
i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

ØThe apparent structures differ depending on Gaussian’s

scale parameter σ. zh
@s

qi
•Larger values: larger scale edges detected
hi

•Smaller values: finer features detected

 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 q

se
i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
• Smoothed derivative
zh

removes noise, but blurs

i@
eu

edge. Also finds edges at

different “scales”.
zh
@s

qi
hi
 q

se
i@
u
x 2 y 2
1

e

Gaussian

zh
g(x, y)  e 2 2

@s
2 2

qi
i
 zh x 
x2  y2

g ( x, y )  g ( x, y )  e
qi

u
2 2

i
x 2

se
x 4

zh
 First

i@

qi
derivative of
u

 y x2  y2
se

g ( x, y )  g ( x, y ) 
y
x 2
e zh 4

2 2
Gaussian
i@

qi

u
se
zh

 2
 2

i@
 g ( x, y ) 
2
g ( x, y )  g ( x, y )
eu

x 2
y 2

zh
@s

 x  y  2 
2 2 2

x2  y2
Laplacian of
 e
qi
2 2

 Gaussian
hi

  
4
 q

se
i@
e u

zh
-1 0 1 1

@s

qi
G x= -2 0 2 = 2 -1 0 1

i
zh -1 0 1 1
qi

i
se

zh
i@

qi
u

Sobel filter Gaussian

se

differentiation
zh smoothing
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
Compare Gaussian， Laplacian， and LoG

zh
i
 q

se
Compare Gaussian， Laplacian， and LoG

i@
u
Q:（a) Smoothing (b) 2nd derivative (c) low pass filter (d) high pass filter

zh
(e) band stop filter (f) Edge detection

@s

qi
 0.003 0.0133 0.0219 0.0133 0.003 

i
0.0133 0.0596 0.0133

0.0219
zh
0.0596
0.0983
0.0983
0.1621 0.0983 0.0219

qi

u
0 1 0

i
0.0133 0.0596 0.0983 0.0596 0.0133

se
 

zh
 0.003 0.0133 0.0219 0.0133 0.003  1 -4 1

i@
“Gaussian filter” 0 1 0

qi
u
se

zh “Laplacian filter”
i@

qi

u
se
zh

i@
eu

Laplacian of Gaussian
zh
@s

qi
hi

Q: what about sobel filter？

 Bouns: Kernal Properties

se
i@
u
• Smoothing

zh
@s
– Values positive

qi
i
– Sum to 1  constant regions same as input
zh
– Amount of smoothing proportional to mask size
qi

i
se

zh
– Remove “high-frequency” components; “low-pass”
filter

i@

qi
u
se

• Derivatives zh
i@

qi

u
– Opposite signs used to get high response in regions

se
zh

of high contrast

i@
eu

– Sum to 0  no response in constant regions

zh
@s

– High absolute value at points of high contrast

qi
– high pass filter
hi
 Canny Edge Operator

se
i@
e u

zh
@s
• The Canny edge detection operator was invented in

qi
i
1986 by John Canny at MIT.
zh
•Most widely used edge detector in computer vision. And
qi

i
it is still the state-of-art optimal edge detector with low

se

zh
computational cost.

i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
i@
e u

zh
This is not a good edge

@s
detection. Because,

qi
i
zh ü a single edge has
qi

i
se
multiple response in

zh
the result.

i@

qi
u

ü some edges are

se

zh misssing.
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
Criteria for Optimal Edge Detection

i@
u
(1) Good detection

zh
• Minimize the probability of false positives (i.e.,

@s
detecting spurious edges).

qi
i
• Minimize the probability of false negatives (i.e.,
zh
missing real edges).
qi

i
(2) Good localization

se

zh
• Detected edges must be as close as possible to

i@

qi
the true edges.
u
se

(3) Single response zh

i@

qi

u
• Minimize the number of local maxima around the true

se
edge.
zh

(i.e. detector must return single point for each true

i@
edge point)
eu

zh
@s

qi
hi
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 Five Steps of Canny Edge Detection

se
i@
e u

zh
1. Noise reduction: smooth the image to

@s

qi
eliminate the noise.

i
zh
2. Gradient calculation: calculate the
qi

u
derivative of the image, then highlight the

i
se

zh
region with larger gradient values.

i@

qi
3. Nonmaximum suppression is an edge
u
se

zh
thinning technique that it gets rid of
i@

qi

u
spurious responses to edge detection.

se
zh

4. Double threshold;

i@
5. Linking by Hysteresis that link the
eu

zh
remaining reliable pixels into continuous
@s

qi
edges.
hi
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
The Canny edge detector

original image
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 q

se
calculate the derivatives of the image

i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

magnitude of the gradient

qi
hi
 q

se
Keep points with larger gradient magnitude

i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

thresholding
qi
hi
 q

se
i@
e u

zh
@s

qi
How to turn

i
zh these thick
regions of
qi

i
the gradient

se

zh
into curves?

i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

thresholding
qi
hi
 q

se
Non-maximum Suppression

i@
eu

zh
@s

qi
i
zh
qi

i
se

zh
i@
q

qi
u
se

zh
i@

qi

u
se
zh

i@
Check if pixel is local maximum along gradient
eu

direction, select single max across width of the edge

zh
@s

qi
hi
 q

se
Non-maximum Suppression

i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@
q

qi
u
se

zh
i@

qi

u
se
zh

i@
Check if pixel is local maximum along gradient
eu

direction, select single max across width of the edge

zh
@s

qi
hi
 q

se
Non-maximum Suppression

i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
Check if pixel is local maximum along gradient direction,
eu

select single max across width of the edge

zh
@s

• in the above case of a 3-by-3 heighborhood, check the neighboring

qi
hi

pixels p and r along the edge direction

 q

se
i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

After nonmaximum suppression, you'll get something

zh
called 'thin edges'. These thin edges that can be broken
@s

at various points are usually caused by noises.

qi
hi
 Double Thresholding

se
i@
u
Double thresholding.

zh
Above the high threshold T1 : strong edge

@s

qi
Below the low threshold T2: non edge

i
between the high and low thresholds: weak edge
zh
qi

i
Preserve those strong edges, and link the weak edge

se

zh
only when they are connected to the strong edges.

i@

qi
Throw away all the others.
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 The image after the double thresholding. The non zero

se
points represent either strong edge points with brighter

i@
intensities or weak edge points in grey.

e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q
Linking by Hysteresis

se
i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
1.Initialize: - Slice gradient magnitude along the gradient direction -

i@

qi
u

Mark the point along the slide where the magnitude is max
se

zh
2. Propagate chain from current point:
i@

qi

u
• Predict next points using the normal to the gradient at that

se
zh

point

i@
• Find which point is a local max magnitude in gradient
eu

direction
zh
@s

• Retain in magnitude > T

qi
hi
 q

se
Final Canny Edges

i@
u
the original image the final canny edges

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi
 q

se
Parameters of Canny Edge Detection

i@
e u

zh
@s
Except T1 and T2

qi
i
zh
Size of Gaussian filter：
–large σ：detects large scale edges
qi

i
se
–small σ：detects fine features

zh
i@

qi
u

Double thresholding
se

zh
–the high threshold gets higher：missing important
i@

qi

u
information

se
- the low threshold gets lower：introducing more
zh

i@
irrelevant information
eu

zh
@s

qi
hi
 q
Effect of σ (Gaussian kernel spread/size)

se
i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

•The choice of depends on desired behavior

i@
eu

–large σ：detects large scale edges

zh
@s

–small σ：detects fine features

qi
hi
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 q
Canny edge detection vs modern methods

se
i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi

[1] Holistically-Nested Edge Detection, by Saining Xie and Zhuowen Tu, 2015, CVPR. citation 1385
 Prisma - DeepArt

se
i@
e u

zh
@s

qi
i
zh
qi

i
se

zh
i@

qi
u
se

zh
i@

qi

u
se
zh

i@
eu

zh
@s

qi
hi

A Neural Algorithm of Artistic Style， Leon A. Gatys, Alexander S. Ecker, Matthias Bethge， 2015
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i
 zh
i@
se
u
hi qi
@s zh
eu i@s
qi e u
zh
i@ q
se
qi
zh u
i@ qi
se zh
u i@
qi se
zh
i