TEE Solved questions

Instructions
• Please enter your answers for each question in the space provided.

• Use Matlab notation to specify matrices and vectors.

– Separate successive elements

  of a vector by comma and space. For example, if the answer to a
1
question is the vector  2 , type your answer as [1,2,3]’
3
here ’ indicates transpose.
– For a matrix, separate elements within
 a row by commas and successive rows by semicolons. For
1 2 3
example, to represent the matrix type your answer as [1, 2, 3; 4, 5, 6]
4 5 6
• Unless otherwise specified, round off all numeric values to two decimal places

• Specify complex numbers by first specifying the real part and using i to indicate the imaginary part.
For example, 2.58 + 4.56 i. Round off numeric values to two decimal places.

1 15 marks each
1. Consider the function specified in (1) for the first three parts of this question.

(x1 − x2 )2 − (x3 − x4 )2 − 8 (1)

(a) Specify two stationary points of the function in (1), if they exist. If the function has only one
stationary point, indicate it clearly. Specify each stationary point with the corresponding values
of the variables enclosed within parentheses. For example, suppose a function of two variables
x1 and x2 has two stationary points at (x1 = 5.98, x2 = −.57) and (x1 = 8, x2 = 1.86) you may
type your answer as
(x1 = 5.98, x2 = -.57), (x1 = 8, x2 = 1.86)
If no stationary points exist, your answer should be
No stationary points exist for the given function.

Answer : Any set of 4 values where x1 = x2 and x3 = x4 for example x1 = 1, x2 = 1, x3 =

2, x4 = 2

1
 Solution/explanation : Apply the first order conditions for minimization of a function. We get
∂f
∂x1
= 0 =⇒ 2(x1 − x2 ) = 0
∂f
∂x2
= 0 =⇒ −2(x1 − x2 ) = 0
∂f (2)
∂x3
= 0 =⇒ −2(x3 − x4 ) = 0
∂f
∂x4
= 0 =⇒ 2(x3 − x4 ) = 0
Note that these are not independent equations i.e. there are free variables. Hence there are
multiple solutions.
(b) Identify two local minima for the function in (1) if they exist. If the function has only one local
minimum, indicate it clearly. Specify each local minimum with the corresponding values of the
variables enclosed within parentheses. For example, suppose a function of three variables x,y
and z has a local minimum at (x = 4.4, y = −.5, z = 1.6) type your answer as
(x= 4.4, y = -.5, z = 1.6)
If no local minima exist, your answer should be No local minima exist for the given function.
Answer : No local minima exist for the given function

Detailed explanation : Suppose a particular set of values,say x1 = x∗1 , x2 = x∗2 , x3 = x∗3 , x4 =

x∗4 is a local minimum. In fact by choosing values for x3 and x4 so that x3 − x4 is large, we can
make the value of the overall function as low as possible (high negative value).

To formally check if any of the stationary points are local minima, one would have to apply
second order conditions.
(c) Suppose we wish to minimize the function shown in (1), but with the following additional
constraints given in (3).
Constraint C1 x1 + 2x4 = 20
(3)
Constraint C2 2x2 − x3 = 10
Specify one solution to the above constrained optimization (if at least one solution exists).
• If there is at least one solution, specify where it occurs i.e. the values of the variables and
what is the function value for these variable values
for example (x1 = 1, x2 = 2, x3 = 10, x4 = 20), function value = 20
• If there is no solution, type your answer as :
No solution exists for the constrained optimization problem
Answer : A solution exists at (x1 = 8, x2 = 8, x3 = 6, x4 = 6). Value of function at above
solution = -8.
Apply the method of Lagrange multipliers to solve the above constrained optimization problem.

Consider two line segments whose end points are indicated below for the next two sub parts.
• Line segment 1 : end points (x=1,y=1) and (x=8,y=15)
• Line segment 2 : end points (x=2,y=4) and (x=10,y=20)
(d) Are the line segments 1 and 2 intersecting, coincident or parallel?
Specify your answer as The lines are intersecting/coincident/parallel (depending on whether
they are intersecting/coincident/parallel).

2
 Answer : Two lines are parallel if their slopes are the same.
Slope of line segment 1 = 15−1
8−1
20−4
= 2 and slope of line segment 2 = 10−2 =2
Since the slopes are the same, the line segments are parallel.
Also specify the lengths of the line segments. Type your answer as :
Line 1 : length = < l1 > units, Line 2 : length = < l2 > units
where < l1 > and < l2 > are to be replaced by the actual numeric values.
Answer The length of a line segment
p is the Euclidean distance
√ between its end points.
2 2
The length of line segment 1 = (15 − 1) + (8 − 1) = 245 = 15.65 (rounded to 2 decimal
places)
p √
Similarly, the length of line segment 2 = (20 − 4)2 + (10 − 2)2 = 320 = 17.89 (rounded to
2 decimal places)

(e) Suppose the following

  transforms are applied on the points of the two line segments above.
x
Suppose p = and p0 is the corresponding location, the transformation matrix M is such
y
that p0 = M p
The transforms applied are :
 
0.5 0
• First transform Scaling as specified by the matrix
0 2
 
−1 0
• Second transform Reflection as specified by the matrix
0 −1
After application of the transforms, are the line segments joining the corresponding end
points, intersecting, coincident or parallel?

Specify your answer as The lines are intersecting/coincident/parallel (depending on whether

they are intersecting/coincident/parallel).
Also specify the lengths of the line segments after the above sequence of transformations.
Type your answer as :
Line 1 after transformation : length = < l1 > units, Line 2 after transformation : length = < l2 >
units
where < l1 > and < l2 > are to be replaced by the actual numeric values.
Answer : Recall that a composition of linear transforms is also a linear transform. The trans-
formations can be applied one at a time or a combined transform specified by a single matrix
Mcombined = M2 M1 , where M1 and M2 are the matrices specifying the first and second trans-
form respectively.
The points that the end points of line segments 1 and 2 are mapped to, after applying the trans-
formations are :
• Line segment 1 :
(1, 1) → (−0.5, −2)
(8, 15) → (−4, −30)

• Line segment 2 :
(2, 4) → (−1, −8)

3
 (10, 20) → (−5, −40)

It can be verified that the lines are parallel even after the transformations are applied.

The lengths of the line segmentspafter applying the transforms are :

The length of line segment 1 = (−30 − (−2))2 + (−4 − (−0.5))2 = 28.22 (rounded to 2 dec-
imal places) p
Similarly, the length of line segment 2 = (−40 − (−8))2 + (−5 − (−1))2 = 22.36 (rounded
to 2 decimal places)

2. Consider the following 5 data samples from the well known iris dataset. Suppose we consider only
two attributes, namely sepal length and sepal width as shown on table 1.

Sample 1 Sample 2 Sample 3 Sample 4 Sample 5

Sepal length 5.1 4.9 4.7 4.6 5
Sepal width 3.5 3 3.2 3.1 3.6
Table 1: A small subset of the iris data set

(a) A common pre-processing step is to subtract the relevant mean vector from each sample. What
is the sample mean vector computed from this set of 5 samples? Answer : Mean vector =
[4.86, 3.28]0
Specifically, mean sepal length = 5.1+4.9+4.7+4.6+5
5
= 4.86 and
3.5+3+3.2+3.1+3.6
mean sepal width = 5
= 3.28
(b) What is the data after mean subtraction? i.e. subtract the relevant mean vector from each of the
five samples and write the result as a matrix where column i displays the mean subtracted values
for sample i.
Answer : Subtract the corresponding mean values from each entry in table 1 i.e. from each
entry in row 1 subtract mean sepal length and from each entry in row 2 subtract the mean sepal
width. This yields

Sample 1 Sample 2 Sample 3 Sample 4 Sample 5

Sepal length 0.24 0.04 -0.16 -0.26 0.140000
Sepal width 0.22 -0.28 -0.08 -0.18 0.32
Table 2: Values obtained after subtracting sample means

(c) Let Ams refer to the matrix obtained in the previous step after mean subtraction. Suppose the
Singular value decomposition (SVD) of the matrix Ams yields Ams = U ΣV T . Let U and Σ be
as given in (4)
   
−0.5864 −0.8100 0.6113 0 0 0 0
U= Σ= (4)
−0.8100 0.5864 0 0.2575 0 0 0

Compute the percentage variability of the data accounted for by each component.
For example if there are 4 principal components, each accounting for 25% of the variance, your

4
 answer should be typed as :
PC1 : 25%, PC2 : 25%,PC3 : 25%, PC4 : 25%

Answer : PC1 : 84.93% PC2 : 15.07%

Explanation : There are only two principal components since the number of attributes = 2.
Recall that the squares of the singular values of the mean subtracted data matrix are the eigen-
values of the covariance matrix. Also recall that the eigenvalues of the covariance matrix are the
variances along the directions specified by the principal components.
(d) Using the first principal component only, reconstruct sample 1 and sample 2 i.e. the vectors in
column 1 and column 2 of table 1. Your answer should be typed as
Sample 1 reconstructed : (Sepal length = < sl1 >, Sepal width = < sw1 >
Sample 2 reconstructed : (Sepal length = < sl2 >, Sepal width = < sw2 >
where < sl1 >, < sw1 >, < sl2 > and < sw2 > are to be replaced by the actual numeric
values.
Answer :
Sample 1 reconstructed : (Sepal length = 3.42, Sepal width = 4.72
Sample 2 reconstructed : (Sepal length = 3.11, Sepal width = 4.29

(e) Compute reconstruction error for Sample 1 and Sample 2 i.e. the first two columns in table 1.
Use the standard Euclidean L2 norm of the difference vector as the reconstruction error. The
difference vector is the difference between the original data vector and reconstructed data vector.
Answer :
Sample 1 reconstruction error = 2.0789
Sample 2 reconstruction error = 2.21

2 Single part short answer questions

   
0.1 0.1
 −0.1   0.12 
1. What is the angle between the two vectors 
 0.4  and  0.4 ? Express your answer in degrees
  

1 1
rounded to two decimal places such that the value lies between 0 and 180.
Answer : cos−1 (0.9795) = 0.2071 radians = 11.61 degrees

2. If the dot product (also referred to as inner product) of two unit vectors is zero, the angle between the
two vectors in degrees =
Answer : 90 degrees
 
4 8 −2.5 0
 0 −2 5 8 
3. The determinant of the matrix 
 0
 is
0 7.5 8 
−2 0 0 2
Answer : The determinant of above matrix = -520.

5
 It can be computed by a co factor expansion along any row or column, it is convenient to choose either
row 4 or column 1 since these have fewer non-zero entries.

4. Consider a set of 3 vectors S = {(2, 1, 4), (1, 2, 5), (v1 , v2 , v3 )}. Here each vector is enclosed within
(). For example, (2,1,4) specifies a vector where first component is 2, second is 1 and third component
is 4. Specify at least one set of values for v1 , v2 , v3 for which S is an orthogonal set. Briefly justify
your answer.
Answer : No possible values for v1 , v2 , v3 such that the above set of vectors is an orthogonal set.
Note that in an orthogonal set of vectors, any two vectors must be orthogonal to each other. In the
above set, the vectors (2, 1, 4) and (1, 2, 5) are not orthogonal. Hence, even if it is possible to find
values for v1 , v2 , v3 such that the third vector is orthogonal to the other two, the given set of vectors is
not an orthogonal set.
√ √
5. Consider a set of 3 vectors V = {(−0.5, 0.5, a), (0.5, b, 0.5), (1/ 2, 0, −1/ 2)}. Here each vector is
enclosed within (). For example, (2,0,-1) specifies a vector where first component is 2, second is 0 and
third component is -1. State with brief justification, for what values of a and b is V an orthonormal
set? If there no such possible values for a and b, type your answer as
No possible value for a and b that makes the given set of vectors, an orthonormal set. and add a
brief justification.
Answer : No possible value for a and b that makes the given set of vectors, an orthonormal set.
Explanation : Recall that in an orthonormal set of vectors, each vector is of unit magnitude and any
two vectors are orthogonal (this can be verified by checking that the dot product of the two vectors is
0).

• For the dot product of first and second vector to be 0, we have -0.5(0.5) + 0.5(b) + 0.5(a) = 0
• The dot product of second and third vector = 0 i.e. for any value of b, these vectors are orthogonal
to each other.
√ √
• The dot product of third and first vector = 1/ 2 × −0.5 + 0 + −1/ 2 × a. For this to be 0, we
must have a = −0.5. However, with a = −0.5 the first vector is not of unit magnitude.

6. Identify all stationary points of the function x2 − 2xy + y 2 + 2x + 4y.

Specify each stationary point with the corresponding values of the variables enclosed within paren-
theses. For example, suppose a function of two variables x1 and x2 has two stationary points at
(x1 = 7.98, x2 = −.7) and (x1 = 8, x2 = 1.86) you may type your answer as
(x1 = 7.98, x2 = -0.7), (x1 = 8, x2 = 1.86)
If no stationary points exist, your answer should be
No stationary points exist for the given function along with a suitable justification.
Answer : No stationary points exist for the given function.
A stationary point is where the gradient is 0 i.e. the partial derivatives with respect to each variable
when evaluated at the specific values are 0.

Let f (x, y) = x2 − 2xy + y 2 + 2x + 4y. Taking partial derivatives yields

∂f (x,y)
• ∂x
= 2x − 2y + 2

6
 ∂f (x,y)
• ∂y
= −2x + 2y + 4
For the partial derivatives to be 0, the system of equations to be solved is :
2x − 2y = −2
−2x + 2y = −4
This system of equations is inconsistent and hence there is no possible values for x and y that makes
all partial derivatives to be 0.
2
7. Evaluate the numeric value of the gradient of the function e−(x−y) at x = 2, y = 4.
2
Answer : Let f (x, y) = e−(x−y) . The gradient is :
 −(x−y)2 
e × −2(x − y)
∇x,y f (x, y) = 2
e−(x−y) × 2(x − y)
Substituting x = 2, y = 4 in the gradient expression, the numerical value of the gradient is :
 −(2−4)2   
e × −2(2 − 4) 0.073263
2 =
e−(2−4) × 2(2 − 4) −0.073263
 
2 1
8. Compute the eigenvalues of the matrix . Write the eigenvalues separated by a comma fol-
8 6
lowed by space, round off all numeric values to two decimal places.
Answer : 0.54, 7.46
9. Compute and specify
 any eigenvector corresponding to the eigenvalue of largest magnitude of the
2 1
matrix
8 6
Answer : The largest eigenvalue = 7.46. An eigenvector corresponding to this eigenvalue =
 
−0.18
−0.9837
Note that any scalar multiple of an eigenvector is also an eigenvector.
10. Consider a sequence of geometric transformations
• Step 1 : x0 = x + 2 and y 0 = y − 4.
• Step 2 : x0 = 0.5x and y 0 = 2y
Let p and p0 represent the homogeneous coordinates before and after transformation respectively.
Assume that p and p0 are represented as column vectors using homogeneous coordinates, where the
first component is the x-coordinate and the second component is the y-coordinate and are of same
shape and size. Specify a single matrix M that achieves the combined effect of the above sequence of
transformations such that given any p we can compute p0 as
p0 = M p

If no such matrix is possible, type your answer as

No such matrix is possible.
Answer : The matrices (in homogeneous coordinates) for the individual steps are :

7
  
1 0 2
• Step 1 : M1 =  0 1 −4 
0 0 1
 
0.5 0 0
• Step 2 : M2 =  0 2 0 
0 0 1
The combined effect
  of the two transformations can be expressed by a single matrix M = M2 M1 =
0.5 0 1
 0 2 −8 
0 0 1