DSC 433/533 Class 13 Nearest Neighbors example

Task: classify a credit card holder as a likely responder to an offer (or not) and also predict his/her average monthly balance
(in $) based on:
Gender (M or F);
age (in years);
salary (in $ thousands).
Training data for 4 credit card holders:

Anne
Bob
Carrie
Doug

Gender
F
M
F
M

Age
25
22
23
26

Salary
50
55
55
40

Response
No
No
Yes
Yes

Balance
300
600
400
500

Eric: 24 year-old male earning $45k

Distances
Anne
Bob
Carrie
Doug
Maximum

Gender
1
0
1
0

Age
1
2
1
2
2

Std. Age
0.5
1.0
0.5
1.0

Salary
5
10
10
5
10

Std. Salary
0.5
1.0
1.0
0.5

Manhattan
2.0 (2=)
2.0 (2=)
2.5 (4)
1.5 (1)

Euclidean
1.22 (2)
1.41 (3)
1.50 (4)
1.12 (1)

Gender distances are 0 for a match, 1 for a mismatch.

Standardized distances for quantitative variables are the absolute difference divided by the maximum absolute difference.
Manhattan distance is G + A + S.
Euclidean distance is (G2 + A2 + S2).
Neighbors (k)
1
2
3

Manhattan classification
1 Y, so classify Y
1 Y, 2 N, so classify N
1 Y, 2 N, so classify N

Manhattan prediction
$500
(500+300+600)/3=$467
(500+300+600)/3=$467

Euclidean classification
1 Y, so classify Y
1 Y, 1 N, so classify ?
1 Y, 2 N, so classify N

Euclidean prediction
$500
(500+300)/2=$400
(500+300+600)/3=$467

Fiona: 22 year-old female earning $50k

Distances
Anne
Bob
Carrie
Doug
Maximum
Neighbors (k)
1
2
3

Gender
0
1
0
1

Age
3
0
1
4
4

Manhattan classification
1 Y, 1 N, so classify ?
1 Y, 1 N, so classify ?
1 Y, 2 N, so classify N

Std. Age
0.75
0.00
0.25
1.00

Salary
0
5
5
10
10

Manhattan prediction
(400+300)/2=$350
(400+300)/2=$350
(400+300+600)/3=$433

Std. Salary
0.0
0.5
0.5
1.0

Manhattan
0.75 (1=)
1.50 (3)
0.75 (1=)
3.00 (4)

Euclidean classification
1 Y, so classify Y
1 Y, 1 N, so classify ?
1 Y, 2 N, so classify N

Euclidean
0.75 (2)
1.12 (3)
0.56 (1)
1.73 (4)

Euclidean prediction
$400
(400+300)/2=$350
(400+300+600)/3=$433