Assignment 3

Introduction to Machine Learning

Prof. B. Ravindran
1. For linear classification we use:

(a) A linear function to separate the classes.

(b) A linear function to model the data.
(c) A linear loss.
(d) Non-linear function to fit the data.

Sol. (a)
Refer to lecture.

2. Logit transformation for P r(X = 1) for given data is

S = [0, 1, 1, 0, 1, 0, 1]
3
(a) 4
4
(b) 3
4
(c) 7
3
(d) 7

Sol. (b)
P r(X = 1) = 47
p(x) 4×7 4
Logit = 1−p(x) = 3×7 = 3

3. The output of binary class logistic regression lies in this range.

(a) [−∞, ∞]
(b) [−1, 1]
(c) [0, 1]
(d) [−∞, 0]

Sol. (c)
Refer to lecture.

4. If
1 − p(x)
log( ) = β0 + βx
1 + p(x)
What is p(x)?
1+eβ0 +βx
(a) p(x) = eβ0 +βx
1+eβ0 +βx
(b) p(x) = 1−eβ0 +βx

1
 eβ0 +βx
(c) p(x) = 1+eβ0 +βx
1−eβ0 +βx
(d) p(x) = 1+eβ0 +βx

Sol. (d)

1 − p(x)
log( ) = β0 + βx
1 + p(x)
1 − p(x)
= eβ0 +βx
1 + p(x)
1 − p(x) = eβ0 +βx + p(x).eβ0 +βx
1 − eβ0 +βx = p(x)(1 + eβ0 +βx )
1 − eβ0 +βx
p(x) =
1 + eβ0 +βx

5. Logistic regression is robust to outliers. Why?

(a) The squashing of output values between [0, 1] dampens the affect of outliers.
(b) Linear models are robust to outliers.
(c) The parameters in logistic regression tend to take small values due to the nature of the
problem setting and hence outliers get translated to the same range as other samples.
(d) The given statement is false.

Sol. (a)
Refer to lecture.

6. Aim of LDA is (multiple options may apply)

(a) Minimize intra-class variability.
(b) Maximize intra-class variability.
(c) Minimize the distance between the mean of classes
(d) Maximize the distance between the mean of classes
Sol. (a), (d)
Refer to lecture.

7. We have two classes in our dataset with mean 0 and 1, and variance 2 and 3.
(a) LDA may be able to classify them perfectly.
(b) LDA will definitely be able to classify them perfectly.
(c) LDA will definitely NOT be able to classify them perfectly.
(d) None of the above.

2
 Sol. (c)
The two classes overlap and hence cannot be classified perfectly by LDA.
8. We have two classes in our dataset with mean 0 and 5, and variance 1 and 2.
(a) LDA may be able to classify them perfectly.
(b) LDA will definitely be able to classify them perfectly.
(c) LDA will definitely NOT be able to classify them perfectly.
(d) None of the above.
Sol. (a)
Depends on the actual data points.

9. For the two classes ’+’ and ’-’ shown below.

While performing LDA on it, which line is the most appropriate for projecting data points?
(a) Red
(b) Orange
(c) Blue
(d) Green
Sol. (c)
The blue line is parallel to the line joining the mean of the clusters and will therefore maximize
the distance between the means.

3
10. LDA assumes that the class data is distributed as:

(a) Poisson
(b) Uniform
(c) Gaussian
(d) LDA makes no such assumption.

Sol. (c)
Refer to lecture