Geometric view of linear algebra

( we want to find a set of vectors, whose linear combination generates all the column vectors of a data
matrix)
Basis Vector: are a set of vectors that are linearly independent and spans(any vector of m-dimensional
space can be written as a linear combination) the m-dimensional space.

2D Space: R2 => (x,y)

Ex: v1 = [1 0]T v2 = [0 1]T
[2 1] = 2[1 0] + 1. [ 0 1] = 2v1 + 1v2

V1, v2 are basis vectors, linearly independent.

Basis vectors for a m-dimensional space is not unique.

The number of vectors in the basis set is same=> for spanning R2 , 2 basis vectors required
for spanning R3 , 3 basis vectors required

To span a subspace ( a subset of R2 ), less number of vector may suffice

Data Matrix : [ [c1] [c2] [c3] ….], where [ci] is a column vector.

Rank of matrix : r : => basis vector ( r linearly independent columns) ( in this case) => this vectors spans
the remaining columns.
Q: does the linearly independent vector span the entire space?

Gain in term of storage of the data matrix:

Data : 400 samples (columns), 4 attributes (rows) : 400 * 4 = 1600 elements to be stored
( in a rectangular matrix of size m x n, rank = rank( minimum of row , column))
Let row rank of this matrix is 2:
It is sufficient to store 2 * 4 = 8 values + 2 ( which vector I am choosing 1 or 2) * 400 = 808 values to
store 400 samples.

 Reduces the size of storage for the data matrix.

--
Distance of vectors, hyperplanes, halfspaces, eigenvalues and eigenvectors

Represent a line and plane:

R2
Line : ax1 + bx2 +c = 0
n1x1 + n2x2 + b = 0 n
nTX + b = 0, where n = [n1 n2]T; X = [x1 x2], n is normal vector. and X are vectors.

X2 X1(x11,x12)
Point X1 and X2 are on line.
nT X1 + b = 0;
nT X2 + b = 0;

X1
 subtract second from one nT(X2 – X1) = 0 => orthogonal vectors ; (X2 – X1) gives direction of line. N is
perpendicular to a line.

This can be generalized for any m dimension.

3D space : R3

n = [ n1 n2 n3]T , n is perpendicular to a plane(2D)

Projection :
( reduce the m dimensional data to a smaller dimensional space for approximation)
Target : best approximation to represent data in lower dimension. From 3d space to 2 D space (
example)
Consider R2 space. Need 2 basis vectors v1 and v2
X( x1 x2 x3)
Projection: X’ of a vector X onto a lower dimension (2D) is define as:
X’ = c1V1 + c2V2 - eq 1
V2
X’
X = c1v1 + c2v2 + n ; n is a normal vector such that
nTv1 = v1Tn = 0
V1 nTv2 = v2Tn = 0 ; n vector is orthogonal to v1 as well as v2

v1Tn = 0
v1T(X - c1v1 + c2v2) = 0
v1TX = c1v1T v1
find value of c1
find value of c2 using v2Tn
X’ = substitute c1 and c2 in eq 1

Generalization : X’ = sum(j=1 to k) cjvj projection of X onto a space spanned by k linearly independent

basis vectors.