Machine Learning

Individual Assignment 1
Algorithm and Mathematics of Machine Learning
Submission Deadline: April 20, 2023
Note: Only handwritten answer is accepted

Introduction

This assignment was designed to allow you to assess how much you know about the algorithm and
mathematics of clustering, classification, and dimension reduction. Hence, you are asked to provide the
details (also steps) for each of the following questions.

Task 1: Clustering.

1) Use the k-means clustering algorithm and Euclidean distance to cluster the following eight
examples into 3 clusters:
Points X1 X2
P1 10 2
P2 5 2
P3 4 8
P4 8 5
P5 5 7
P6 4 6
P7 2 1
P8 9 4
And also, assume that Points P1, P4, and P7 are initially selected as cluster centers. The stopping
criteria are when there is no movement.
a) Show all the necessary steps until convergence?
b) Show the grouping after convergence?
c) Calculate silhouette score
d) Discuss if there is any issue that you observed while applying the K-means algorithm to the
above data.

2) Apply agglomerative clustering to group the data described in the above table (Question 1) and
show the dendrograms after clustering?

1
Task 3: Classification

3) This task will classify the data using the Decision Tree classifier and KNN.

a) Given the training data, construction Decision Tree classifier. Plot the flow chart of the rule at
the end.

Training data
Points x1 x2 y
P1 1 0 0
P2 1 1 0
P3 0 0 1
P4 0 0 1
P5 1 1 0
P6 1 1 0
P7 1 0 0

b) Using the rule you extracted, predict the class label of P8 (0, 1).

P8 0 1 ?

c) Apply the KNN algorithm on it to the above training data.

2
Task 4: General Questions

4) Given the following confusion matrix, calculate:

a. Accuracy
b. Precision
c. Recall
d. F1-Score
e. Micro average precision
f. Weighted average recall

3
5. Assume we have the dataset below, which has four features and 5 training examples.
Apply Principal component analysis (PCA) and transform it to two dimensional space.

Show all the necessary steps:

➔ Standardize the dataset.
➔Calculate the covariance matrix for the features in the dataset.
➔Calculate the eigenvalues and eigenvectors for the covariance matrix.
➔Sort eigenvalues and their corresponding eigenvectors.
➔Pick k eigenvalues and form a matrix of eigenvectors.
➔Transform the original matrix.