ECSE 4850/6850, Spring 2022

Introduction to Deep Learning

Assignment #1,
Due date: 2 pm, January 27th , 2022
1. Given a system of linear equations below for a vector x=[ x1 x2 x3 ]T
9x1 + 4x2 + 2x3=1
4x1 + 2x2 = 7
2x1 + 4x3= -6

Solve each question below

1) Manually construct the A matrix and the b vector

2) Perform SVD on A =UDVT , you can use a software to perform SVD computation.
3) Compute the eigenvalues and eigenvector for A from it’s SVD decompositions, U, D, and V .
4) Using A’s SVD decomposition, compute the determinant of A , the rank of A , and the null
space of A.
5) Generate an A’ that is rank 2 approximation to A, i.e., reconstruct A’ using first two columns
of U and V.
6) Solve x in terms A and b by computing the pseudoinverse of A. Is there a unique solution for x?

T T
2. Given f ( X )=log {( AX +b ) (CX +d )+λX X } , where X is a N x 1 vector, A and C are MxN
matrix, b and d are a M x 1 vector, and  is a scalar, A, C, b, and d are not function of X. Derive
∂f ( X )
the equation for computing ∂ X

3. Let UNx1, VKx1, XMx1 be vectors and ANxk be a matrix

T
∂σ (U AV )
1) Compute ∂X , where U and V are a function of X but A is not
T
∂σ (U AV )
2) Compute ∂X , where U and V are NOT a function of X but A is
where (z) is the sigmoid function and it is defined as
1
σ ( z )=
1+e−z
W [k ] X +b[k ]
e
σ M (k )= K
∂σ M (k )
∑ e
W [ k ' ] X +b[ k ' ]

∂ X , where W is KxN
4. Given the softmax function k ' =1 , compute
matrix, X is a Nx1 vector, b a Kx1 vector, W[k] the kth row of W and b[k] is the k’s element of b.