TheAILearner

Mastering Arti�cial Intelligence

Harris Corner Detection

In the previous blog, we discussed what are features and how corners are considered as a good
feature as compared to edges and �at surfaces. In this blog, let’s discuss one of the famous and most
commonly used corner detection methods known as Harris Corner Detection. This was one of the
early attempts to �nd the corners by Chris Harris & Mike Stephens in their paper A Combined
Corner and Edge Detector in 1988. Now it is called the Harris Corner Detector. So, let’s �rst under-
stand the basic idea behind this algorithm, and then we will dive into mathematics. Let’s get started.

As discussed in the previous blog, corners are regions in the image with large variations in intensity in
all directions. For instance, take a look at the below image. If you shift the window by a small amount,
then corners will produce a signi�cant change in all directions while edges will output no change if we
move the window along the edge direction. And the �at region will output no change in all directions
on window movement.

So, the authors took this simple idea of �nding the di�erence in intensity for a displacement of (u,v) in
all directions into a mathematical form. This is expressed as
Here,

• the window function is either a rectangular window or a Gaussian window which gives weights
to pixels underneath.
• E(u,v) is the di�erence in intensities between the original and the moved window.

As can be clearly seen, for nearly constant patches the error function will be close to 0 while for dis-
tinctive patches this will be larger. Hence, our aim is to �nd patches where this error function is large.
In other words, we need to maximize this error function for corner detection. That means we have to
maximize the second term. We can do this by applying Taylor Expansion and using some mathemati-
cal steps as shown below

So, the �nal equation becomes

Then comes the main part. As we have already discussed that corners are the regions in the image
with large variations in intensity in all directions. Or we can say it in terms of the above matrix M as “A
corner is characterized by a large variation of M in all directions of the vector [u,v]”. So, if you remem-
ber that eigenvalues tell us about the variance thus by simply analyzing the eigenvalues of the matrix
M we can infer the results.

But the authors note that the exact computation of the eigenvalues is computationally expensive,
since it requires the computation of a square root, and instead suggests the following score func-
tion which determines if a window contains a corner or not. This is shown below

Therefore, the algorithm does not have to actually compute the eigenvalue decomposition of the ma-
trix M and instead it is su�cient to evaluate the determinant and trace of matrix M to �nd the corners.

Now, depending upon the magnitude of the eigenvalues and the score (R), we can decide whether a
region is a corner, an edge, or �at.

• When |R| is small, which happens when λ1 and λ2 are small, the region is �at.
 • When R<0, which happens when λ1>>λ2 or vice versa, the region is edge.
• If λ1>>λ2, then vertical edge
• otherwise horizontal edge
• When R is large, which happens when λ1 and λ2 are large and λ1∼λ2, the region is a corner

This can also be represented by the below image

So, this algorithm will give us a score corresponding to each pixel. Then we need to do thresholding in
order to �nd the corners.

Because we consider only the eigenvalues of the matrix (M), we are considering quantities that are in-
variant also to rotation, which is important because objects that we are tracking might rotate as well
as move. So, this makes this algorithm rotation invariant.

So, this concludes the Harris Corner Detector. I hope you understood this. Now, let’s see how to do
this using OpenCV-Python.

OpenCV

OpenCV provides a builtin function cv2.cornerHarris() that runs the Harris corner detector on the im-
age. Below is the syntax for this.

1 cv2.cornerHarris(src, blockSize, ksize, k)

2
3 # src - Input single-channel 8-bit or floating-point image.
4 # blockSize - Neighborhood size used when computing the matrix M.
5 # ksize - Aperture parameter for the Sobel operator.
6 # k - Harris detector free parameter in the score equation.

For each pixel (x,y) it calculates a 2×2 gradient covariance matrix M(x,y) over
a blockSize×blockSize neighborhood. Then using this matrix M, this calculates the score for each pixel.
Below is the code for this

1 import cv2
2
3 # Read the image and convert to greyscale
4 img = cv2.imread('D:/downloads/contracing.png')
5 gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
6
7 # Find the corners using the Harris Corner Detector
8 dst = cv2.cornerHarris(gray,2,3,0.04)
9
10 #result is dilated for enhancing the corners, not important
11 dst = cv2.dilate(dst,None)
12
13 # Threshold for an optimal value, it may vary depending on the image.
14 thresh = 0.01*dst.max()
15 img[dst>thresh]=[0,255,0]
16
17 # Display the image
18 cv2.imshow('dst',img)
19 cv2.waitKey(0)

Below is the result of this.

So, this is how you can implement the Harris Corner Detector using OpenCV-Python. I hope you un-
derstood this. In the next blog, we will discuss the Shi-Tomasi algorithm that improves this Harris
Corner Detector even further. Hope you enjoy reading.

If you have any doubts/suggestions please feel free to ask and I will do my best to help or improve my-
self. Goodbye until next time.

This entry was posted in Image Processing and tagged cv2.cornerHarris(), feature detection, feature
detection opencv, Harris Corner Detector Opencv, image processing, opencv python on 25 Sep 2021
[https://theailearner.com/2021/09/25/harris-corner-detection/] by kang & atul.