Vectors

• An ordered list of n numbers

Basics of Linear Algebra • Dimensionality: n

 x1 
Tim Cootes   Transpose:
 x2 
x=  xT = (x1 x2 ... xn )
:
 
x 
 n

Geometric Interpretation Manipulating Vectors

• Vector is difference between points in an n- • Basic arithmetic operations act on each
dimensional space element independently:
• For instance, in 2D:  x
   a1   b1   sa1   a1 + b1 
2 
 y        
− 2
 
2   a2   b2   sa 2   a2 + b2 
 1  a=  b =  sa =   a+b =
 
: : : : 
       
a  b   sa  a + b 
3   n  n  n  n n
−1
 

Vectors must have same number of elements

Manipulating Vectors Magnitude of a Vector

• Length of vector, written |a|
 a1 + b1  • By Pythagorus:
b  
 a 2 + b2 
a+ b =  n
a :  a = ∑ ai2
2
 
a+b  a +b 
 n n
i =1

• In 2D:
a
a2 a = a12 + a22
a1

1
 Inner (“dot”) Product Properties of the Inner product
a
• Dot product of two vectors is the sum of the a.b = a b cosθ θ
products of matching elements: b
n
Where θ is the angle between th e vectors
a.b = ∑ aibi = a1b1 + a 2b2 + ...+ an bn Since -1 ≤ cos? ≤ 1
i =1
− a b ≤ a.b ≤ a b
• Vectors must have the same number of
elements If a.b = a b thena and b are parallel (? = 0)
If a.b = 0 thena and b are orthogonal (? = 90o )

Properties of the Inner product Matrices

• If u is a unit vector (length, |u|=1), then a.u • An ordered grid of numbers
gives the length of the component of a • mxn
along the direction of u  a11 a12 ... a1 n 
– m rows  
– n columns  a 21 a22 ... a 2n 
a.u = a u cosθ = a cosθ A=
: : : 
 
a a 
 m 1 am 2 ... amn 

θ • Note: n-dimensional vector is an n x 1 matrix

u
1 a.u

Matrix Transpose Special Matrices

 d1 0 .. 0 
 
• Transpose: Flip about leading diagonal • Diagonal matrix:  0 d2 .. 0 
: : O : 
 
• If A is n x m, its transpose, AT, is m x n 0 0 0 d 
 n

 a11 a12 ... a1n   a11 a 21 ... am1 

    • Symmetric matrix (must be square):
 a 21 a 22 ... a 2n   a 21 a22 ... am 2 
A= A =
T
: : :  : : :   a1 1 a1 2 ... a1 n 
 
    a a2 2 ... a2 n 
a a m 2 ... amn  a an 2 ... a mn  aij = a ji A =  12
 m1  1n : : : 
 a 
 1n a 2n ... an n 
Line of
symmetry

2
 Matrix Arithmetic Matrix Multiplication
 sa11 sa12 ... sa1 n 
 
Ab = c
 sa 21 sa22 ... sa2 n 
sA = 
: 
 a11 a12 ... a1n  b1   c1   a11b1 + a12b2 + ... + a1n bn 
: :       
 
∑ i=1 a 2ibi
n
 sa
 m1 sam 2 ... samn 
  a 21 a 22 ... a2 n  b2   c2   
= =
 : : :  :   :   : 

 a11 + b11 a12 + b12 ... a1n + b1n      
a ... a mn  bn   cm   ∑i =1 mi i 
n
   m1 m2a a b 
 a21 + b21 a 22 + b22 ... a2 n + b2 n 
Size : ( m x n ) x ( n x 1) = ( m x 1)
A +B =  
: : :
 
a +b + ... amn + bmn 
j-th element of c given by sum of products
 m1 m 1 m 2 bm 2
a of j-th row of A with elements of b
c j = ∑i =1 a jibi
n

A and B must have same size

Matrix Identity Geometric Interpretation

• Multiplying by identity leaves vector • If matrix A is square (n x n), then it can be
unchanged thought of as applying an affine geometric
• Identity, I, is square with zero everywhere transformation to n-D vectors
except the leading diagonal: • ie it rotates, scales and shears them
1 0 ... 0  • Ab = c
 
0 1 ... 0  – c is a transformed version of b
I=
: : :
 
0 0 ... 1 

2D Examples 2D Examples
 s 0  a b  1  a  a b   0  c 
sI =   applies a scalingof s     =       =  
0 s  c d  0  b  c d   1  d 
 0 − 1  x   − y 
1 0   y  =  x 
 0 −1      
  applies a rotationof 90 o a b 
x →   x
b 
 d 
1 0  x  0  c d  
 1 a 
y    c 
 
 cosθ − sin θ  −y x
 sin θ cosθ 
 applies a rotation of θ  1
  0
 
x 
 
 s 0 y   s 0  x  sx 
  =  
  applies a shear along x 0
 1  y  y 
 0 1

3
 2D Examples Geometric Representation
a 0   0
x →   x  
 0 0 b   b • Every 2 x 2 matrix can be decomposed into a combination
 1
  of rotations, scaling and shears
• Every n x n matrix can be decomposed into a combination
 1 a  of n-dimensional scalings, rotations and shears.
   0 
 0  

 cosθ − sin θ 
x →  x
 0  sin θ cosθ 
 1
 
θ
 1
 0
 

Determinant of a Matrix Determinants

a b 
• The determinant, |A|, is a measure of the For 2D matricies, A =   = ad − bc
c d 
relative volume change applied by the
transformation represented by A  d1 0 .. 0 
 
– It is the volume of the unit square/cube/hyper-  0 d 2 .. 0 
For diagonal matricies, D =  = d 1d 2 ...d n
cube after transforming by A : : O : 
Area A  
b  0 0 0 d 
 0
Area 1 a b 
x →   x  d 
   n
 c d
 1
  a 
 c 
 
 1
 
 0

Matrix Products Matrix Products

AB = C
ABC = (AB)C = A(BC)
. . . . .
 . . . .  . . . . .
  . . . . .  
cij = ∑k =1 a ikbkj
= . . . . . n Sizes: (n x m)(m x p)(p x q) = (n x q)
 . . . .
 . . . . . . . . . 
  .  . . . . .
 . . . .  

For square matrices, AB = A B

ij-th element of C is dot product of i-th row of A with j-th column of B

Sizes: (n x m)(m x p) = (n x p)

N.cols of A matches N.rows of B

4
 Unit (“Orthonormal”) Matrices 2D Unit Matrices
• If U is a square matrix such that • 2D unit matrices have the form
U = 1 and U U = I
T
 cosθ − sinθ 
U =  
then U is a “Unit” or “Orthonormal” matrix  sin θ cosθ 

• Such matrices apply pure rotations (they are pure rotations)

Uv
– No scaling or shearing
U applies a rotation of θ
UT appliesa rotationof -? θ v

Eigenvector Decomposition Transformations

• Any square, symmetric matrix M can be • Thus the mapping x → Mx
decomposed into a combination of pure – Rotates the vector
rotations and simple scalings: – Scales the vector by d i along dimension i
– Rotates the result back again
Diagonal matrix

M = UDU T
Pure rotation Inverse of rotation
Pure scaling along each axis

Determinants of symmetric
Eigenvectors
matrices
• The columns of the rotation matrix are the “Eigen- • If M is symmetric,
vectors” of M
• The elements of the diagonal matrix are the M = UDU T = U D UT = 1. D . 1 = d1d 2...d n
“ eigenvalues” of M

Each eigenvector, ui , has a corresponding eigenvalue, d i

Note: There are numerical algorithms for computing

the eigenvectors/values of matrices