The k-nearest neighbors algorithm, also

known as KNN, is a non-parametric,

supervised learning classifier, which uses
proximity to make classifications or
predictions about the grouping of an individual
data point. While it can be used for either
regression or classification problems, it is
typically used as a classification algorithm,
working off the assumption that similar points
can be found near one another.

2
KNN works by finding the distances between
a query and all the examples in the data,
selecting the specified number examples (K)
closest to the query, then votes for the most
frequent label (in the case of classification) or
averages the labels (in the case of
regression).

3
“A baby learns to crawl,
walk and then run. We
are in the crawling stage
when it comes to applying
machine learning.”
~Dave Waters

4
KNN can be used for regression
as well as classification problems.

5
KNN is an algorithm that classifies a sample
based on the classes of the samples that
come closest to it.

6
 This is a three-class classification problem where the aim is
to determine the class of the sample represented by a circle.
7
In the KNN-regression problem, the only
difference is that the distance between
training points and sample points is evaluated
and the point with the lowest average
distance is declared as the nearest neighbor. It
predicts the result on the basis of the average
of the total sum

8
The sample that we need to
predict is the y of the sample
denoted by a cross and it’s
given a certain value of x.
For K = 2, we search for the
two points whose x
components come closest to
the x value of the cross, take
their y-values and find their
arithmetic mean. The result is
y = (3.2+4.2)/2 = 3.6. So this is
the predicted y for the cross

9
K value indicates the count of the nearest
neighbors. We have to compute distances
between test points and trained labels points.
Updating distance metrics with every iteration
is computationally expensive, and that’s why
KNN is a lazy learning algorithm.

10
For k=1 , we search for the first nearest neighbor.
in this case ,the sample gets assigned to the plus

11
For k=3, one of the neighbors is a plus, while two are
squares. The sample is then classified as a square

12
For K=2 , one of the neighbors is a square while the
other is a plus . The classification depends on the
implementation of the algorithm

13
There is no particular way to determine the
best value for "K", so we need to try some
values to find the best out of them. The most
preferred value for K is 5

14
- KNN is pretty intuitive - No Training Step
and simple

- it has One Hyper - Very easy to implement

Parameter: for multi-class problem

- It constantly evolves - has no assumptions

15
- Slow algorithm - Outlier-sensitive

- Curse of Dimensionality - No optimal number of

neighbors
- Needs homogeneous
features -Imbalanced data causes
problems

16
The objective is to identify each of a large
number of black-and-white rectangular
pixel displays as one of the 26 capital
letters in the English alphabet. The
character images were based on 20
different fonts, and each letter within these
20 fonts was randomly distorted to produce
a file of 20,000 unique stimuli. Each
stimulus was converted into 16 primitive
numerical attributes (statistical moments
and edge counts) which were then scaled to
fit into a range of integer values from 0
through 15.
18
19
- Number of instances: 20,000 (20% is for
testing )

- Number of attributes: 17 (letter category -

and 16 numeric features)

- Attribute characteristics: Integer

- Associated Tasks: Classification

- Missing Attribute Values: None

20
21
The Y axis is the True Class while the X axis is
the Predicted class, the correct predictions are
located on the diagonal axis.

For example:

On the “A” column, 158 predictions were correct

and 2 were off. We conclude that precision is
158/160 = 0.9875

On the C” column, 161 predictions were correct

and none were off. We conclude that precision is 1

22
Salma Lamine Gebril Amor Amani Salah