Linear Algebra

A gentle introduction

Linear Algebra has become as basic and as applicable

as calculus, and fortunately it is easier.
--Gilbert Strang, MIT

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

1 : “shiv rpi”
 What is a Vector ?
 Think of a vector as a directed line
segment in N-dimensions! (has “length”
and “direction”)
a 
  
 Basic idea: convert geometry in higher v  b 
dimensions into algebra!
 Once you define a “nice” basis along
each dimension: x-, y-, z-axis …
 c 
 Vector becomes a 1 x N matrix!
 v = [a b c]T y

 Geometry starts to become linear v

algebra on vectors like v!
x
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
2 : “shiv rpi”
 Vector Addition: A+B
A+B

A
A+B = C
(use the head-to-tail method
B to combine vectors)
C
B

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

3 : “shiv rpi”
 Scalar Product: av

av  a ( x1 , x2 )  (ax1 , ax2 )

av
v

Change only the length (“scaling”), but keep direction fixed.

Sneak peek: matrix operation (Av) can change length,

direction and also dimensionality!
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
4 : “shiv rpi”
 Vectors: Dot Product
d 
A  B  AT B   a b c   e   ad  be  cf
Think of the dot product as
a matrix multiplication
 f 

2
A  AT A  aa  bb  cc The magnitude is the dot
product of a vector with itself

The dot product is also related to the

A  B  A B cos( )
angle between the two vectors

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

5 : “shiv rpi”
 Inner (dot) Product: v.w or wTv

v 
w v.w  ( x1 , x2 ).( y1 , y2 )  x1 y1  x2 . y2

The inner product is a SCALAR!

v.w  ( x1 , x2 ).( y1 , y2 ) || v ||  || w || cos 

v.w  0  v  w
If vectors v, w are “columns”, then dot product is wTv
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
6 : “shiv rpi”
 Bases & Orthonormal Bases
 Basis (or axes): frame of reference

vs

Basis: a space is totally defined by a set of vectors – any point is a linear

combination of the basis

Ortho-Normal: orthogonal + normal

x  1 0 0 x y  0
T

y   0 1 0 xz  0
T
[Sneak peek:
Orthogonal: dot product is zero
Normal: magnitude is one ] z   0 0 1 T
yz  0
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
7 : “shiv rpi”
 What is a Matrix?
 A matrix is a set of elements, organized into rows and
columns
rows

a b 
columns
c d 
 

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

8 : “shiv rpi”
 Basic Matrix Operations
 Addition, Subtraction, Multiplication: creating new matrices (or functions)

a b   e f  a  e b  f 
c d    g 
h  c  g d  h 

   Just add elements

a b   e f  a  e b  f 
c d    g 
h  c  g d  h 
 Just subtract elements
  
a b   e f  ae  bg af  bh  Multiply each row
c d   g  
  h  ce  dg cf  dh  by each column

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

9 : “shiv rpi”
 Matrix Times Matrix

L  MN

l11 l12 l13   m11 m12 m13   n11 n12 n13 

l l l23   m21 m22 m23   n21 n22 n23 
 21 22

l31 l32 l33   m31 m32 m33   n31 n32 n33 

l12  m11n12  m12 n22  m13n32

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

10 : “shiv rpi”
 Multiplication
 Is AB = BA? Maybe, but maybe not!

a b   e f  ae  bg ... e f  a b  ea  fc ...

c d   g   
  h   ... ... g

  
h   c d   ... ...

 Matrix multiplication AB: apply transformation B first, and

then again transform using A!
 Heads up: multiplication is NOT commutative!

 Note: If A and B both represent either pure “rotation” or

“scaling” they can be interchanged (i.e. AB = BA)
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
11 : “shiv rpi”
 Matrix operating on vectors
 Matrix is like a function that transforms the vectors on a plane
 Matrix operating on a general point => transforms x- and y-components
 System of linear equations: matrix is just the bunch of coeffs !

a b   x   x'
    
 x’ = ax + by
 y’ = cx + dy

c d   y  y'

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

12 : “shiv rpi”
 Direction Vector Dot Matrix

 ax bx cx d x   vx 
a by cy d y  v y 
v  M  v   y 
 az bz cz d z   vz 
   
0 0 0 1  1

vx  vx ax  v y bx  vz cx
vy  vx a y  v y by  vz c y v  v x a  v y b  v z c
vz  vx az  v y bz  vz cz
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
13 : “shiv rpi”
 Matrices: Scaling, Rotation, Identity
 Pure scaling, no rotation => “diagonal matrix” (note: x-, y-axes could be scaled differently!)
 Pure rotation, no stretching => “orthogonal matrix” O
 Identity (“do nothing”) matrix = unit scaling, no rotation!

r1 0
0 r2
[0,1]T [0,r2]T
scaling
[1,0]T [r1,0]T

cos -sin
sin cos
[-sin, cos]T
[0,1]T
[cos, sin]T
rotation

[1,0]T
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
14 : “shiv rpi”
 Scaling
P’

r 0 a.k.a: dilation (r >1),

0 r
contraction (r <1)
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
15 : “shiv rpi”
 Rotation

P’

cos -sin
sin cos

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

16 : “shiv rpi”
 2D Translation
P’

t
P

P’
P '  ( x  t x , y  t y )  Pt t
ty
P
y

Rensselaer Polytechnic Institute x tx Shivkumar Kalyanaraman

17 : “shiv rpi”
 Inverse of a Matrix
 Identity matrix:
AI = A
 Inverse exists only for square

1 0 0
matrices that are non-singular
 Maps N-d space to another


N-d space bijectively
Some matrices have an

I  0 1 0 
inverse, such that:


AA-1 = I
Inversion is tricky: 0 0 1
(ABC)-1 = C-1B-1A-1
Derived from non-
commutativity property
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
18 : “shiv rpi”
 Determinant of a Matrix
 Used for inversion
 If det(A) = 0, then A has no inverse

a b 
A  det( A)  ad  bc
 c d 

1 1  d  b
A 
ad  bc  c a 

http://www.euclideanspace.com/maths/algebra/matrix/functi
ons/inverse/threeD/index.htm

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

19 : “shiv rpi”
 Projection: Using Inner Products (I)

p = a (aTx)
||a|| = aTa = 1
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
20 : “shiv rpi”
 Homogeneous Coordinates
 Represent coordinates as (x,y,h)
 Actual coordinates drawn will be (x/h,y/h)

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

21 : “shiv rpi”
 Homogeneous Coordinates

 The transformation matrices become 3x3 matrices,

and we have a translation matrix!

x’ 1 0 tx x
y’ = 0 1 ty y
1 0 0 1 1

New point Transformation Original point

Exercise: Try composite translation. Shivkumar Kalyanaraman

Rensselaer Polytechnic Institute

22 : “shiv rpi”
 Homogeneous Transformations
v  M  v
vx   a1 d1  v x 
b1 c1
v    a d 2  v y 
b2 c2
 y   2 
 vz   a3 b3 c3 d 3   v z 
     
1 0 0 0 1  1
vx  a1v x  b1v y  c1v z  d1
vy  a2 v x  b2 v y  c2 v z  d 2
vz  a3v x  b3v y  c3v z  d 3
1  0v x  0v y  0v z  1
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
23 : “shiv rpi”
 Order of Transformations
 Note that matrix on the right is the first applied
 Mathematically, the following are equivalent

p’ = ABCp = A(B(Cp))
 Note many references use column matrices to
represent points. In terms of column matrices
p’T = pTCTBTAT

T R M
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
24 : “shiv2rpi”
 Rotation About a Fixed Point other than
the Origin
Move fixed point to origin
Rotate
Move fixed point back
M = T(pf) R() T(-pf)

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

25 : “shiv2rpi”
 Vectors: Cross Product
 The cross product of vectors A and B is a vector C which is
perpendicular to A and B
 The magnitude of C is proportional to the sin of the angle between
A and B
 The direction of C follows the right hand rule if we are working
in a right-handed coordinate system

A×B A  B  A B sin( )
B

A
Rensselaer Polytechnic Institute Shivkumar Kalyanaraman
26 : “shiv rpi”
 MAGNITUDE OF THE CROSS
PRODUCT

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

27 : “shiv rpi”
 DIRECTION OF THE CROSS
PRODUCT
 The right hand rule determines the direction of the
cross product

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

28 : “shiv rpi”
 For more details
 Prof. Gilbert Strang’s course videos:
 http://ocw.mit.edu/OcwWeb/Mathematics/18-06Spring-2005/V
ideoLectures/index.htm

 Esp. the lectures on eigenvalues/eigenvectors, singular value

decomposition & applications of both. (second half of course)

 Online Linear Algebra Tutorials:

 http://tutorial.math.lamar.edu/AllBrowsers/2318/2318.asp

Rensselaer Polytechnic Institute Shivkumar Kalyanaraman

29 : “shiv rpi”