Deep Learning Class Project

Sankha Das (2019A7PS0029P)

December 2021

1 Introduction
This class project is based on conducting experiments on feature extraction and
image matching on a set of face images captured by different subjects. The
data is collected as a joint effort of all the students enrolled in the course. Each
student captured 20 images each of foreheads a total of 6 different persons,
spread over two sessions and taken from two perspectives (far and near image).
From this collected data, images were finally provided for 28 different subjects,
both from far and near perspective and from two different sessions. The total
number of images in the dataset were 336 with 168 images from session 1 and
168 images from session 2.
We are required to perform feature extraction using popular feature extrac-
tors such as SIFT, SURF, ORB, etc. It is then required to perform matching
on pairs of different images, one image being from session 1 and the other being
from session 2. The matching scores obtained would then be used to plot gen-
uine/imposter histogram and find values of false acceptance rate, false rejection
rate, correct recognition rate and other evaluation parameters.

2 n ∗ n Feature Matching
For the n ∗ n feature matching, all images corresponding to session 1 were
sequentially compared with each image of session 2. As a result a total of 28224
comparisons were made from 168 images of session 1 and 168 images of session
2.
None of the images of session1 were compared with each other and similarly
for session 2. However, images of the same person from different sessions were
compared and checked for comparison scores.
For the purpose of the feature extraction, SIFT and SURF feature extractors
were used. As the images were being compared for similarity, a matching score
was being generated simultaneously. The comparisons between images of the
same entity were populating scores for genuine matchings and those between
images of different entities were populating scores for imposter matchings.

1
3 Genuine/Imposter Histogram
The genuine and imposter matching scores obtained above were used to plot
the genuine and imposter histograms. The histograms show the frequency of
each matching score obtained, respectively for the genuine matching scores and
imposter matching scores.
The red histogram is the imposter histogram while the blue histogram is the
genuine matching histogram.

Figure 1: Genuine/Imposter Histogram (SIFT)

Figure 2: Genuine/Imposter Histogram (SURF)

2
 Figure 3: Genuine/Imposter Histogram (ORB)

4 Matching Score and Method

The matching scores are obtained by the following algorithm:

• Feature keypoints and descriptors are extracted from both images.

• The descriptors are then used to perform the matching computation using
a KNN matching model.

• The model produces a number of matched keypoints from the descriptors

passed to it.
• The number of matched keypoints over the number of total keypoints is
calculated as the matching score for the matching of the two images.
• If either of two images are such that no keypoints can be extracted from
them, then the matching is aborted on them.

The genuine matching scores and imposter matching scores obtained from
the above steps are used to calculate the number of false positives, false nega-
tives, true positives and true negatives with respect to some threshold values.
The number of false positives and false negatives are used to calculate the False
Acceptance Rate (FAR) and False Rejection Rate (FRR) respectively, for each
threshold value.
N umber of F alse P ositives
F AR =
T otal N umber of M atchings
N umber of F alse N egatives
F RR =
T otal N umber of M atchings

3
 The values of FAR and FRR obtained for each threshold value are stored in
an array and this array is used to plot the values of the FAR and FRR against
threshold values.

Figure 4: FAR and FRR values for different threshold values (SIFT)

Figure 5: FAR and FRR values for different threshold values (SURF)

As can be seen from the above graph, the value of FAR and FRR become
equal just after the threshold value of 0.42 for SIFT, 0.5 for SURF and 0.40 for
ORB feature extractors. This value of equality of FAR and FRR is called Equal
Error Rate (ERR) which can also be found from the Receiver Operating Curve
(ROC).

4
 Figure 6: FAR and FRR values for different threshold values (ORB)

5 Receiver Operating Curve

The ROC plot is obtained by plotting the values of FRR against FAR for each
threshold value obtained above. The intersection of the curve with the line
y = x gives the point of Equal Error Rate as explained above. The ERR value
from both ROC and FAR/FRR curve comes out to be around 0.033 for SIFT,
0.035 for SURF and 0.039 for ORB feature extractor.

Figure 7: ROC Curve (SIFT)

5
Figure 8: ROC Curve (SURF)

Figure 9: ROC Curve (ORB)

6
6 Correct Recognition Rate
Correct Recognition Rate (CRR) is defined as the number of actual matches
that are obtained at rank one recognition. Mathematically,
N umber of M atches Correctly Recognized
CRR =
T otal N umber of M atchings

The values of CRR obtained for each threshold value is plotted below.

Figure 10: CRR values for different threshold values (SIFT)

Figure 11: CRR values for different threshold values (SURF)

7
 Figure 12: CRR values for different threshold values (ORB)

7 Accuracy
The accuracy of the matching as obtained by the formula below was found to be
99.976% for SIFT, 99.972% for SURF and 99.971% for ORB feature extractors.

F RRi + F ARi
Accuracy = max(100 − ) , i = 1, 2, ..., no. of threshold values
2

8 Conclusions
The following table summarizes the results obtained for the matching performed
using the different feature extractors.

Feature Extractor EER EER Threshold Accuracy

SIFT 0.033 0.42 99.976%
SURF 0.035 0.50 99.972%
ORB 0.039 0.40 99.971%

In general, it was observed that with a higher value of EER threshold value,
the accuracy of the matching was lower. The CRR for all three feature extractors
followed a uniform pattern of increase in value with the threshold value.
The code used to obtain these results can be found on the Google Colab note-
book at https://colab.research.google.com/drive/1EqVGq1Sgtaucy7lnE7T9qxTOEekT7FP5?
usp=sharing.