Math 102 Course Notes

by Sean Graves

for a course developed by Enver Osmanagic,

George Peschke, and Beth Powell

Elementary Linear Algebra for Engineers

Winter 2021

0
 Table of Contents

Chapter#1 – The Vector Space ℝ n

1.1 – Points, Arrows, and Vectors in ℝ n ……..……………………………………………………... 3
1.2 – Vector Addition and Scalar Multiplication …………………………………………………….. 5
1.3 – The Length/Norm/Magnitude of a Vector ……………………………………………….…….. 7
1.4 – Linear Motion …..……………………..……………………………………………………….. 8
1.5 – Linear Combinations and Span………..…………………….…….………………….…………10

Chapter#2 – The Vector Space ℂ

2.1 – Complex Numbers and their Algebraic Properties………………………………………....…... 12
2.2 – Other Forms of a Complex Number………………………………………….………….….….. 16
2.3 – Roots of a Complex Number…..……………………………………………………………….. 18
2.4 – Complex Polynomials …………………………….….……..….…..….….……….…….….…. 20

Chapter#3 – Dot Product and Orthogonal Projections

3.1 – Dot Product ...………………………………………….……………..……….…………..……. 22
3.2 – Orthogonal Projections ...…………………………………………….……………...…………. 24

Chapter#4 – Linear Equations and Hyperplanes

4.1 – Linear Equations ………………………………………………….….….…………….….……. 26

Chapter#5 – Systems of Linear Equations

5.1 – Introduction to Systems of Linear Equations ………………….……….……….……….…….. 30
5.2 – Gauss Jordan Elimination …………………………….…….….….….….….…..…….…..….... 33

Chapter#6 – Matrix Operations

6.1 – Matrices and Matrix Operations ………………………………….…….……….……….....….. 38
6.2 – Algebraic Properties of Matrices………………………………………………….……………. 42
6.3 – A Method of Finding A−1 ..……………………………………………………………………..46
6.4 – More on Linear Systems and Invertible Matrices ………………..………...…….……………..48
6.5 – Diagonal, Triangular, and Symmetric Matrices ……………………….….....….…………..….. 49

Chapter#7 – Determinants
7.1 – Determinants by Cofactor Expansion ……………………………………………….………… 51
7.2 – Evaluating Determinants by Row Reduction …………………………….…………….……… 53
7.3 – Properties of Determinants ………………………………………………..…….……………... 54

Chapter#8 – Cross Product

8.1 – Cross Product ……………………………………………………….………………….……… 55
8.2 – Determinants and Oriented Areas/Volumes …………………………………………………… 58

Chapter#9 – Vector Spaces

9.1 – Real Vector Spaces …..………………………………………………………….….………….. 59
9.2 – Subspaces …………………………………………………..………………………….…...….. 61
9.3 – Row Space, Column Space, and Null Space………………………………...………….……… 64

1
Chapter#10 – Basis and Dimension
10.1 – Linear Independence ……………………………………………….…………………...……. 66
10.2 – Coordinates and Basis …………………………………………….………………….………. 68
10.3 – Dimension …..…………………………………………………………….………………….. 71
10.4 – Rank, Nullity, and the Fundamental Matrix Spaces …………………………….……………. 74

Chapter#11 – Change of Basis

11.1 – Change of Basis ...…………………………………………………………………….………. 76
11.2 – Orthogonal and Orthonormal Bases ……………………………………………………..…… 78
11.3 – Gram-Schmidt Process ……………………………………………………………………..… 79
11.4 – Dimension Formula for Orthogonal Complements……………………………………..…….. 80

Chapter#12 – Linear Transformations

12.1 – Linear Transformations ..…………………………………………………………….……..… 81
12.2 – Linear Transformations in ℝ2 and ℝ3 …………..……………………………………..…… 84
12.3 – Properties of Matrix Transformations ……...…..…………………………………………..… 88
12.4 – Invertible Linear Transformations ……...…..…………………………..…………………..… 91
12.5 – Orthogonal Matrices ……...…..………………………………………..……………….…..… 92
12.6 – Custom Bases ……...…..…………………………………………………………………...… 94

Chapter#13 – Eigenvalues and Eigenvectors

13.1 – Eigenvalues and Eigenvectors ...………………………………….………….……….………. 95
13.2 – Diagonalization ………………………………………………….…..…………………..…… 98
13.3 – Orthogonal Diagonalization ………………………………….………………………..…..… 101
13.4 – Complex Vector Spaces ……………………………………….………………………..…..… 102

2
 n
Chapter#1 – The Vector Space ℝ
§1.1 – Points, Arrows, and Vectors in ℝ n

ℝ is the set of real numbers, this set can be represented by a number line.

2
ℝ is the set of all ordered pairs of real numbers. This set can be represented by the Cartesian plane,
where each point on the plane is given a unique ordered pair (x , y ) .

3
ℝ is the set of all ordered triples of real numbers. This set can be represented by the 3-dimensional
space that we live in, where each point in this space is given a unique ordered triple ( x , y , z) .

We can extend this idea to even higher dimensions. In general, ℝn is the set of all ordered n-tuples of
real numbers. Each point in ℝn is given a unique ordered n-tuple ( x 1 , x 2 , x 3 , ⋯ , x n ) where each of
x 1 , x 2 , x 3 , ⋯ , x n ∈ℝ .

Example

In set notation, we write ℝn = {(x 1 , x 2 , x 3 , ⋯ , x n) | x1 , x 2 , x 3 , ⋯ , x n ∈ ℝ }

An arrow from point P to point Q in ℝn is an arrow whose tail is at P and whose tip is at Q, it is
denoted ⃗ PQ .

3
Given points P( p1 , p2 ,⋯, pn ) and Q(q 1 , q 2 ,⋯, qn ) in ℝn , we can represent arrow ⃗
PQ by
the n-tuple (q 1−p 1 , q2− p2 , ⋯ , qn− p n , ) .

It follows that arrows that have the same length and same direction will be represented by the same n-
tuple; this n-tuple that represents all arrows that have the same length and same direction is what we
call a vector, denoted ⃗v . That is, vectors do not have a position in ℝn . Vectors have magnitude
(represented by their length) and direction.

If a vector is drawn so its tail is at the origin, then we say the vector is in standard position. It is in this
sense that vectors and points in ℝn are very similar. They use the same n-tuple notation, however
vectors and points have different geometric interpretations.

The vector (0 , 0 ,⋯, 0) in ℝn is called the zero-vector and is denoted 0⃗ .

4
§1.2 – Vector Addition and Scalar Multiplication
n n
ℝ is called a vector space. That is, the vectors in ℝ can be added together, and they can be
multiplied by a real number coefficient called a scalar.

⃗ = (w1 , w 2 ,⋯, wn ) are vectors in ℝn , then

If ⃗v =(v 1 , v 2 ,⋯, v n ) and w
⃗ =(v 1 +w 1 , v 2+ w2 , ⋯, v n+ w n) .
⃗v + w

Example:

To get a visualization of how vector addition/subtraction works, we can use arrow drawings:

If ⃗v =(v 1 , v 2 ,⋯, v n ) is a vector in ℝn and k is a real numbered scalar, then k ⃗v =(k v 1 , k v 2 , ⋯, k v n ) .

Examples:

Remark: Vectors ⃗v and w

⃗ are said to be parallel if there exists a scalar k ∈ℝ such that ⃗v = k w
⃗.

5
Properties of Vector Addition and Scalar Multiplication

⃗ in ℝn and for all scalars c and k in ℝ the following hold:

u , ⃗v , and w
For vectors ⃗

u + ⃗v =⃗v + ⃗u
1. ⃗
2. (⃗
u + ⃗v ) + w u +(⃗
⃗ =⃗ v +w
⃗)

3. ⃗v + ⃗
0 =⃗v

4. 0 ⃗v = ⃗0
5. 1⃗v = ⃗v
6. −1⃗v =−⃗
v

7. ⃗v + (−⃗
v )= ⃗
0

8. c (⃗
u + ⃗v )= c ⃗
u + c ⃗v
9. (c + k )⃗v = c ⃗v + k ⃗v
10. (ck )⃗
v = c (k ⃗v )

6
§1.3 – The Length/Norm/Magnitude of a Vector

To calculate the length of a vector in ℝn you use an extension of the Pythagorean Theorem. The length
(or norm, or magnitude) of vector ⃗v =(v 1 , v 2 ,⋯, v n ) is given by:

‖ ⃗v ‖ = √ v 21 + v 22 + ⋯+v 2n

Example:

Properties of Norm

v ‖ ≥0
1. ‖ ⃗

v ‖ =0 if and only if ⃗v = ⃗0
2. ‖ ⃗
3. ‖ k ⃗
v ‖ =| k|⋅‖⃗v‖
v +w
4. ‖ ⃗ ⃗ ‖≤‖ ⃗v ‖+ ‖ w
⃗ ‖ (Triangle Inequality)

A vector with length 1 is called a unit vector.

1
Given a vector ⃗v , a unit vector in the same direction as ⃗v is ⋅⃗v .
‖⃗v‖

Example:

Remark: This process is called normalizing vector ⃗v .

7
§1.4 – Linear Motion

The equation of the line, y = m x + b that we learned in junior-high and high-school is only for the 2-
dimensional space ℝ2 . To describe lines in more general spaces ( ℝn ), we use the following:

The line through the point described by position vector ⃗p in the direction ⃗ 0 is given by the
d≠⃗
⃗
(vector) equation ⃗x = ⃗p + t d , t ∈ℝ .

Definition
In physics, the motion of an object whose velocity vector does not change direction is described as
linear motion. The linear motion of an object through point ⃗p with velocity vector ⃗d ≠⃗0 is given by
⃗ n
⃗x = ⃗p + t d , t ∈ℝ .

Example
Find the vector function that describes the linear motion of a particle that passes through P(1,2 ,−3)
at t = 0 and Q(7 ,−2,1) at t = 1 .

Example
Find the vector function that describes the linear motion of a particle that passes through P(1,2 ,−3)
at t = 0 and Q(7 ,−2,1) at t = 5 .

8
Example
Find the vector function that describes the linear motion of a particle that passes through P(1,2 ,−3)
at t =−2 and Q(7 ,−2,1) at t = 5 .

Notice that the lines with equations ⃗x =(1,2) + t (2,4) , t ∈ℝ and ⃗x =(0,0)+ t (1,2) , t ∈ ℝ are
exactly the same. However, these two equations do not describe the same linear motion.

9
§1.5 – Linear Combinations and Span

Let S = { v⃗1 , v⃗2 , ⋯ , v⃗m } be a set of vectors in ℝn . We say the vector w ⃗ is a linear combination of the
vectors in S if there exists scalars k 1 , k 2 , ⋯ , k m ∈ ℝ such that w⃗ = k 1 v⃗1 + k 2 v⃗2 + ⋯+ k m v⃗m .

Example
Let S = { (1,1,1,0) , (1,2,1,1) , (1,0,1,0) } . Is (2 ,−1,2 ,−1) a linear combination of the vectors in S?

Example
Let S = { (1,1,1,0) , (1,2,1,1) , (1,0,1,0) } . Is (1,1,0,1) a linear combination of the vectors in S?

Example
In ℝ2 , describe and sketch the set S = { k 1 (1,2)+ k 2 (3,1) | 0 ≤ k 1 , k 2 ≤1 } .

10
Definition: The set of all possible linear combinations of vectors is the set S = { v⃗1 , v⃗2 ,⋯, v⃗n } is
called the span of v⃗1 , v⃗2 ,⋯, v⃗n . It is denoted span { v⃗1 , v⃗2 ,⋯, v⃗n } or span ( S) .

Examples:

11
 Chapter#2 – The Vector Space ℂ
§2.1 – Complex Numbers and their Algebraic Properties

The set of complex numbers is an extension of the set of real numbers. A complex number is a number
of the form z = a + b i where a and b are real numbers and i has the special property that i 2 =−1 , or
in other words i = √ −1 .

The real part of z is a, and the imaginary part of z is b. We notate this as Re( z )= a , and Im(z ) = b .

Example:

We can add, subtract, multiply, and divide complex numbers.

Examples:

1 +2 i 2− i
1. Simplify + .
3 −4i 5i

2. Simplify (1 −i)4 .

12
It can be helpful to visualize complex numbers by plotting them on the complex plane, where the
horizontal axis indicates the real part and the vertical axis indicates the imaginary part of the complex
number.

Example:

We can now extend the idea of absolute-value to complex numbers. Recall for the real number x, | x|
is the distance that x falls from the origin. It follows from the Pythagorean Theorem that for the
2 2
complex number z = a + b i , | z| = √ a + b .

Example: Sketch the set of all points z on the complex plane that satisfy the equation | z−1 + 3 i| = 2 .

Remarks: 1. The distance between the complex numbers z 1 and z 2 is given by | z 1 − z 2| .

2. The absolute value of a complex number z is usually called the modulus of z.

13
Example: Find the distance between z 1 =−2 + 4 i and z 2 = 1 − 3 i .

The complex conjugate of a complex number z = a + b i is defined as the complex number z = a − bi

and is denoted by z̄ .

Example:

Properties of Conjugation and Modulus

If z 1 , z 2 , and z are complex numbers, then

1. z1 ± z2 = z1 ± z2

2. z1 z2 = z1 z2

z1 z1
3. ()z2
=
z2

4. | z 1 z2|=| z 1 || z 2|

z 1 |z 1 |
5. || =
z 2 | z2|

6. ¯z̄ = z

7. | z| =| z̄ |

14
Example: What is the multiplicative inverse of the non-zero complex number z = a + b i ?

Example: Prove that z z̄ =| z|2

Remark: By viewing complex numbers as vectors in the complex plane we obtain the triangle
inequality:
| z 1 + z2|≤| z 1| + | z 2|

15
§2.2 – Other Forms of a Complex Number

Instead of using Cartesian coordinates to represent complex numbers on the complex place, we can use
polar coordinates. Recall:

It follows that the polar form of the complex number z = a + b i is z =r ( cos θ + i sin θ ) where
b
r =| z| = √ a2 + b 2 and tan θ = . Here θ is called the argument of z, denoted arg ( z) .
a

Example: Write the complex number z =−1 − i in polar form.

The well known Euler’s Formula states that e iθ = cos θ + i sin θ , thus we can write the polar form of the
complex number z = a + b i in its exponential form:

iθ b
z =r e where r =| z| and tan θ = .
a

Example: Express the complex numbers z =2 i , z =−3 , and z =1 + √ 3 i in exponential form.

16
There are many benefits of using the exponential form of a complex number. In particular evaluating
products and powers becomes very quick.

If z 1 = r e i θ and z 2 = s e i ψ , then z 1 z2 = rs ei (θ + ψ) .

Moreover, z 1k = r k e k i θ .

6
Example: Evaluate ( √ 3 + i)

−10
Example: Evaluate (1 + √ 3 i)

Complex Multiplication as a Transformation

Another benefit of using the exponential form of a complex number is that multiplication of a complex
number z by another complex number w can be viewed as a transformation. In particular, if you
multiply z by w = r e i θ , then the product will be z stretched by a factor of r and rotated counter
clockwise by θ radians.

17
§2.3 – Roots of a Complex Number

Suppose we wanted to find all solutions to the equation z 3 + 1 = 0 . There are actually three solutions
3
(we will see more about this in the next section). The solutions are the numbers that satisfy √ z =−1 .
Of course z =−1 is one solution, but what about the other two?

Fortunately, the three roots are spaced equally (in terms of rotation) around the complex plane. So, if
2π
you rotate the first root z =−1 counter clockwise radians about the origin you will find the
3
2π
second root, and if you rotate another radians you will find the third solution.
3

2π
i
The exponential form of z =−1 is z = ei π . Using the “rotator” w = e 3
, we can obtain the other two
2π 5π 5π 2π 7π
i i i i i
iπ 3 3 3 3 3
roots. They are thus e e =e and e e =e .

18
Example: Find all fourth roots of z =−8 − 8 √ 3 i .

Remark: The term roots of unity means roots of 1. Thus, for example, finding all fifth roots of
unity means finding all fifth roots of 1 (of course there are five of them).

19
§2.4 – Complex Polynomials

A complex polynomial is an expression of the form p(z)= a n z n + ⋯ + a1 z + a0 , where the coefficients

ai ∈ℂ . If an ≠ 0 then p has degree n, denoted deg ( p)= n .

Example:

The Fundamental Theorem of Algebra says that every complex polynomial with positive degree has a
root in ℂ . That is, for every complex polynomial p(z) , there exists a complex number w such that
p(w) = 0 .

It follows that every degree n polynomial can be factored as p(z)= a n (z − w1)(z − w 2) ⋯( z − wn ) . In

other words, every degree n polynomial has exactly n roots (some may be repeated). Moreover, since
every real number is also a complex number we have that every degree n polynomial with real
coefficients has exactly n roots (some of the roots may be repeated and/or complex).

Example: Find all roots of the polynomial p(z)= z 3 − 8 z 2 + 25 z − 26 .

20
It can be helpful to know that if p( z) has real coefficients and z is a root, then z̄ is also a root.

Example: Fully factor p(z) = 2 z3 − 25 z 2 + 108 z − 145 knowing that z =5 − 2 i is a root.

Remark: As an extension of the vector space ℝn , the vector space ℂn is the set of all ordered n-
tuples whose components are complex numbers. In ℂn , vector addition and scalar
multiplication are carried out in a similar way as in ℝn .

If ⃗z =(z 1 , z 2 , ⋯ , z n ) is a vector in ℂn , then Re(⃗z )=(Re(z 1 ) , Re(z 2) , ⋯ , Re( z n))

and Im(⃗z )=(Im (z 1) , Im(z 2 ) , ⋯ , Im(z n )) .

Example:

21
 Chapter#3 – Dot Product and Orthogonal Projections
§3.1 – Dot Product

The dot product is a way to “multiply” two vectors to obtain a scalar (real number). There are many
uses for the dot product, one in particular is to determine the angle between two vectors (when placed
in standard position).

Definition: u =(u1 ,u 2 ,⋯,u n) and ⃗v =(v 1 , v 2 ,⋯, v n ) in ℝn , the dot product of

Given two vectors ⃗
u and ⃗v is:
⃗
u⋅⃗v =u 1 v 1 + u 2 v 2 + ⋯+ un v n
⃗

Example:

Remark: Note that the dot product of two vectors is not a vector, it is a scalar.

Fact: Two vectors ⃗u and ⃗v are orthogonal (perpendicular) precisely when ⃗

u⋅⃗v = 0 . This can
also be denoted u
⃗ ⊥ v
⃗ .

Properties of Dot Product

⃗ in ℝn and scalar k in ℝ ,
For vectors u⃗ , ⃗v , w

u⋅⃗v = ⃗
1. ⃗ v ⋅⃗
u

2. ⃗
u⋅(⃗
v +w u⋅⃗v + ⃗
⃗ )= ⃗ u⋅ w
⃗

0⋅⃗v = 0
3. ⃗

4. ⃗v⋅⃗v ≥ 0 , and ⃗v⋅⃗v = 0 implies ⃗v =0 .

v ‖2
5. ⃗v⋅⃗v =‖ ⃗

6. k (⃗u⋅⃗
v )=(k ⃗u )⋅⃗v = ⃗
u⋅(k ⃗
v)

22
Pythagoras’s Theorem
2 2 2
u and ⃗v are orthogonal, then ‖ ⃗
If ⃗ u+⃗ u ‖ +‖ ⃗v ‖ .
v ‖ =‖ ⃗

Proof:

u and ⃗v are non-zero vectors in ℝn , then the angle θ between them satisfies
Fact: If ⃗

u⃗⋅⃗v
cos θ =
‖ ⃗u ‖‖ ⃗v ‖

Example: Find the angle between (2 , −1 , 1) and (1 , 1 , 2) .

23
§3.2 – Orthogonal Projections

Theorem: u ,⃗v ∈ ℝ n are non-zero vectors, then ⃗

If ⃗ v can be expressed in the form ⃗
v = w⃗1 + w⃗2
where w⃗1 is a scalar multiple of ⃗u and w⃗2 is orthogonal to ⃗u.

In this case, w⃗1 is called the vector component of ⃗

v along ⃗
u and w⃗2 the
vector component of ⃗ v orthogonal to ⃗u.

Definition: The vector w⃗1 in the above proof is called the projection of ⃗
v onto ⃗
u.

v⃗⋅⃗
u
proj ⃗u ⃗
v= u
⃗
u⋅⃗
⃗ u

Example: v = (1 ,2 ,3 ,4) as the sum w⃗1 + w⃗2 where w⃗1 is parallel to

Express the vector ⃗
(2,2 ,1 ,1) and w⃗2 is orthogonal to (2,2 ,1 ,1) .

24
The distance from a point to a line means the shortest distance. So given point P and line L,
PS ⊥ L .
dist ( P , L) is defined as dist ( P , S) , where S is the point on L closest to P. In such a case, ⃗

2 3
Example:
7 [] [ ]
Find the distance that point P(1, 0, 3) lies from the line ⃗x = 1 + t −1 .
4

25
 Chapter#4 – Linear Equations and Hyperplanes
§4.1 – Linear Equations

An equation of the form a1 x1 + a2 x 2 + ⋯+ an xn = b where b and all the ai ’s are real numbers, and the
x i ’s are variables is called a linear equation with n variables. If b = 0 , then the equation is said to be
homogeneous.

Example:

Remark: In ℝ2 , the equation ax + by = c is a linear equation. You may recognize this as the
equation of a line in ℝ2 .

Recall that in higher dimensions, lines are described by the vector equation
⃗x = ⃗p + t ⃗
d , t ∈ℝ .

In ℝ3 the linear equation ax + by + cz = d is the equation of a plane.

If ⃗p , v⃗1 and v⃗2 are vectors in ℝ3 and v⃗1 and v⃗2 are not parallel, then the plane through ⃗p and
parallel to both v⃗1 and v⃗2 is:
⃗x = ⃗p + t v⃗1 + s v⃗2 , s ,t ∈ ℝ

The above is called the vector equation of the plane.

Definition: A vector that is orthogonal to a plane is called a normal vector for that plane.

26
It follows that every vector parallel to the plane is orthogonal to ⃗n . So if P and Q are points on the
plane, then ⃗ PQ⋅⃗n = 0 . So an equation of the plane through the point ⃗p and orthogonal to the vector
n is ⃗
⃗ n⋅(⃗x − ⃗p )= 0 . This is called the normal equation of the plane.

n = (a, b ,c) , ⃗p = (x 0 , y 0 , z 0) , ⃗x = (x , y , z ) we obtain (a , b , c)⋅(x −x0 , y− y 0 , z−z 0) = 0 .

If ⃗

n = (a, b ,c) is:

That is, The equation of the plane through point P( x 0 , y 0 , z 0) with normal vector ⃗

a( x − x 0 ) + b( y − y 0 )+ c( z − z 0)= 0

Example: Find the equation of the plane through (−1 ,4 , 7) parallel to the plane 2 x − 5 y + z = 1 .

Example: Find the equation of the line through (2,7 ,1) perpendicular to the plane 2 x − 5 y + z = 1 .

27
Example: Find the distance that P(2, 4 ,−1) lies from the plane x − 2 y + 4 z = 1 .

The geometric object consisting of all points that satisfy a linear equation is called a hyperplane. For
example lines in ℝ2 and planes in ℝ3 are hyperplanes.

Example: Find the point on the hyperplane in ℝ4 with equation x 1 − 2 x2 − x 3 + 2 x 4 =3 which

lies closest to P(1 , 0 , 1 , 2) .

28
Example: Find the distance between the two parallel planes x − y + z = 4 and
−2 x + 2 y − 2 z = 7 in ℝ3 .

29
 Chapter#5 – Systems of Linear Equations
§5.1 – Introduction to Systems of Linear Equations

Recall: A linear equation is an equation of the form a1 x 1 + a2 x 2 +⋯+ a n x n = b where the ai ' s
are real numbers called coefficients, the x i ' s are variables and b is a real number.

Remark: A homogeneous linear equation is of the form a1 x1 + a2 x 2 +⋯+ a n x n = 0 .

Definition: A finite set of linear equations is called a system of linear equations.

Example:

In general, a system of linear equations can be written as:

Definition: To solve a system of linear equations means to find all values for the variables that will
simultaneously satisfy all equations in the system.

30
 x+ y +z=1
Example:
{ 2 x + 2 y − 2 z= 2
3 x −3 y − 3 z = 3 }

Theorem: Every system of linear equations has zero, one, or infinitely many solutions.

Example:
{34 xx −6 y = 18
− 8 y = 16 }

Remark: If a system of equations has solutions (one or infinitely many) it is called consistent.

In grade school we learn to solve a system of linear equations using “substitution” and/or
“elimination”. In this course we learn a better method called Gauss-Jordan Elimination.

31
The first step is to record the important information in an augmented matrix. The augmented matrix for
a11 x 1 + a12 x 2 +⋯ + a1 n x n = b 1

{ }
the system a21 x 1 + a22 x 2 + ⋯+ a2 n x n =b 2 is:
⋮
a m1 x 1 + am2 x 2 + ⋯+ amn x n = b m

The last column in the augmented matrix is called the augmented column.

3 x − 4 y + 7 z =8
{
Example: The augmented matrix for the system 6 x − 2 y = 4 is
x + 8 y − z =2 }
To simplify an augmented matrix, we use three elementary row operations that do not change the
solution set of the system of linear equations:

1.

2.

3.

The goal is to introduce as many zeros as possible and then solve the new system of linear equations
that the simplified augmented matrix represents. Since the two systems have the same solution set, we
are thus obtaining the solutions to the original system.

2 x + 4 y − 3 z =1
Example: Solve the system of linear equations
{x + y + 2z =9 .
}
3 x + 6 y −5 z = 0

32
§5.2 – Gauss Jordan Elimination

In the last section we used Gauss-Jordan Elimination to solve a system of linear equations. The first
step was to record the augmented matrix for the system. The next step was to use the three elementary
row operations to simplify the matrix into a matrix with the following properties:

1.

2.

3.

4.

A matrix with the four above properties is said to be in reduced row echelon form (RREF). A
matrix with properties 1, 2, and 3, is said to be in row echelon form (REF).

33
Examples:

1. RREF

2. REF

To put a matrix into RREF using elementary row operations, first put the matrix into REF by creating
0's under all leading positions (working left to right). Then put the matrix into RREF by creating 0's
above all leading entries (working left to right). Leading positions can be made into leading 1's at any
point.

1 3 0 1

Example:
[ 1 4 2 0
0 −2 −2 −1
2 −4 1 1
1 −2 −1 1
]

34
Remarks: 1. Given an arbitrary matrix, it may have several REF's, but the RREF is unique.

2. By looking at the REF of an augmented matrix, we can decide whether the system has
(i) no solutions, (ii) infinitely many solutions, (iii) one unique solution:

(i)

(ii)

(iii)

35
Examples:

1 2 3 4 5 1 2 3 4 5

[
1. 0 0 1 4 2
0 0 0 0 1 ] [
2. 0 0 1 4 2
0 0 0 1 0 ]
1 2 3 4 5 1 2 3 4 5
3.
0
0
0
[ 1
0
0
2 6 4
1 −1 3
0 1 0
] 4.
[0
0
0
1
0
0
2 6 4
1 −1 3
0 0 0
]
To find the solution(s) to a consistent system:

1.

2.

3.

4.

36
 6 x1 + x2 + 4 x 4 =−3
Example:
{
Solve the system of linear equations: −9 x 1 + 2 x 2 +3 x 3 − 8 x 4 =1
7 x1 − 4 x 3 + 5 x 4 =2 }

37
 Chapter#6 – Matrix Operations
§6.1 – Matrices and Matrix Operations:

A matrix is a rectangular array of numbers. The numbers in the array are called entries. If the matrix M
has m rows and n columns, then it is an m×n matrix. A matrix with the same number of rows and
columns is a square matrix. The entry in the ith row and jth column is denoted aij . An m×1 matrix is
called a column vector and a 1×n matrix is called a row vector. Two matrices are equal if they are the
same size and their corresponding entries are equal.

Definition: If A is an m×n matrix, then the transpose of A denoted A T is the n×m matrix
obtained by interchanging the rows and columns of A.

Example: A=
[−62 1 7
4 0 ]

Definition: If A is an n×n square matrix, then the trace of A denoted tr( A) is the sum of the
entries on the main diagonal of A.

2 1 6
Example:
[
A= 0 9 2
π −1 −1 ]
Adding Matrices: If A and B are matrices of the same size, then A + B (and A − B ) are obtained
by adding (and subtracting) corresponding entries.

Example: [−12 1 4
−6 7
+
6 4 −1
][
0 2 8 ]
Scalar Multiples: A scalar is a real number (for now). To multiply a matrix by a scalar is to
multiply each entry in the matrix by the scalar.

1 3 8
Example:
4 [ ]
−1 4
2 0

38
Notation: We use the notation A = [ a ij ] to represent the matrix whose ijth-entry is aij , and the
notation ( A)ij to represent the ijth-entry entry of the matrix A.

It follows that:

1.

2.

3.

4.

Multiplying Matrices: If A is an m×r and B is an r×n matrix, then AB is the m×n matrix
whose entries are obtained as follows:

To obtain the ijth entry of AB, take row i from A and column j from B.
Multiply the corresponding entries from the row and column, and then
add up the resulting products.

2 1 6
Example:
[ 2 1
−6 4
7
][
0 9 2
√ 3 π −1 −1 ]
Remarks: 1. ( AB)ij =

2. If B = [ c⃗1 c⃗2 ⋯ c⃗n ] where the c⃗j ' s are the column vectors of B, then
AB =

r⃗1

⋮
r⃗m
[]
3. If A = r⃗2 where the r⃗j ' s are the row vectors of A, then AB =

39
Remark: The latter two points are helpful when you are interested in a particular row/column of
AB. The first says, “The jth column of AB is A(jth column of B)”, the second says, “The
ith row of AB is (ith row of A)B.

Definition: If A 1 , A 2 ,⋯, A r are matrices of the same size, and c 1 , c 2 , ⋯,c r are scalars, then an
expression of the form c 1 A 1 + c 2 A 2 +⋯+ c r A r is called a linear combination of
A 1 , A 2 ,⋯, A r with coefficients c 1 , c 2 , ⋯,c r .

x = ⃗b is very important in linear algebra, where A is an m×n matrix, ⃗

The matrix equation A ⃗ x is an n×1
⃗
column vector and b is an m×1 column vector. Notice that

x = ⃗b ⇒
A⃗

x = ⃗b implies that ⃗
Thus, A ⃗ b can be expressed as a linear combination of the columns of A.

2
Example:
[ 1 2 3
4 −5 6 ][
−3 =
1
−1
29 ][ ] . Thus,

40
Given A and ⃗b in the matrix equation A ⃗ x = ⃗b , we solve for ⃗
x by solving the system of linear
⃗
equations with augmented matrix [ A b ] .

2 1 7 0
Example:
[
1 1 5 0] []
Find all x⃗ that satisfy the homogeneous equation −1 3 4 ⃗x = 0 .

41
§6.2 – Algebraic Properties of Matrices:

Theorem: Let A, B, C be matrices and a, b be scalars.

(a) A + B=B + A

(b) A + ( B + C )=( A + B ) + C

(c) A (BC )=( AB )C

(d) A (B + C )= AB + AC

(e) (B + C) A= BA + CA

(f) A (B − C)= AB − AC

(g) ( B −C ) A= BA − CA

(h) a(B + C)= aB + aC

(i) a(B − C)= aB − aC

(j) (a + b)C= aC + bC

(k) (a − b)C= aC − bC

(l) a(bC)=( ab)C

(m) a( BC)= B(aC )

Remark:

42
Properties of Transpose: Properties of Trace:
Let A and B be square matrices of the same size.
T T
(a) (A ) = A
(a) tr ( A + B)= tr ( A )+ tr (B)
T T T
(b) ( A + B) = A + B
(b) tr (cA )= c tr ( A)
T T T
(c) ( A − B) = A − B
(c) tr ( AT )= tr ( A)
T T
(d) (kA ) = k A
(d) tr ( A B) = tr(BA)
T T T
(e) ( AB) = B A

Definition: A matrix where every entry is 0 is called a zero matrix, denoted 0.

Eg.

Theorem: Let A be a matrix and c be a scalar.

(a) A + 0 = A =0 + A

(b) A − 0 = A and 0 A − A =− A

(c) A − A =0

(d) A + (− A)= 0 Remark: AB = 0

(e) 0 A = 0

(f) If cA = 0 , then c = 0 or A = 0

43
Definition: A square matrix with 1's on the main diagonal and 0's everywhere else is called an
identity matrix, denoted I. If the size of the matrix is important, then we notate it I n .

Eg:

Remark: For any m×n matrix A, I m A = A and A I n = A .

Definition: Let A be an n×n matrix. If there exists an n×n matrix B such that A B = I and
B A = I , then A is said to be invertible and B is called the inverse of A.

1 2 3 −40 16 9
Example:
[ ] [
If A = 2 5 3 , then A−1 = 13 −5 −3 .
1 0 8 5 −2 −1 ]
Notice that,

Theorem: The 2×2 matrix A = a b is invertible if and only if ad−bc ≠ 0 , and in this case
[ ]
c d
1 d −b .
A−1 = [
ad −bc −c a ]
Example: A= 1 2
[ ]
3 4

Theorem: If A and B are invertible matrices of the same size, then AB is invertible and
( AB)−1 = B −1 A −1 .

Proof:

44
Properties of Inverse: Let A be invertible and k≠0 .

(a)

(b)

(c)

Theorem: If A is an invertible matrix, then A T is invertible and ( A T )−1 = ( A−1 )T .

Proof:

45
§6.3 – A Method of Finding A-1

Definition: A matrix E is called an elementary matrix if it can be obtained from an identity matrix
by performing a single elementary row operation.

Examples:

If the elementary matrix E is obtained by performing a certain elementary row operation on I m and if
A is an m×n matrix, then EA is the matrix obtained when this same row operation is performed on A.

Example:

Theorem: Let A be an n×n matrix, then the following are equivalent:

(a) A is invertible.

x = ⃗0 has only the trivial solution.

(b) A ⃗

(c) The RREF of A is I n .

Proof:

46
Remark: Notice A−1 = E k E k−1 E k−2 ⋯ E 2 E 1 I . That is, the same elementary row operations that
reduce A to I will transform I to A−1 . This gives an algorithm for finding A−1 .

We reduce A to the identity matrix via elementary row operations and

simultaneously apply these operations to I to produce A−1 . We do this by
augmenting the identity matrix to the right of A to produce [ A I ] and reduce
the left side to I. These operations will transform I to A−1 .

2 6 6
Example:
[ ]
Find the inverse of A = 2 7 6 .
2 7 7

47
§6.4 – More on Linear Systems and Invertible Matrices

Theorem: If A is an invertible n×n matrix, then for each n×1 column vector ⃗
b , the matrix
equation A ⃗ x = ⃗b has exactly one solution, namely ⃗
x = A −1 ⃗
b.

Proof:

Theorem: If A and B are square matrices satisfying AB = I , then B = A−1 and A =B−1 .

Proof:

48
§6.5 – Diagonal, Triangular and Symmetric Matrices

Definition: A square matrix where every entry off of the main diagonal is 0 is called a diagonal
matrix.

Eg.

d1 0 ⋯ 0

[
In general, a diagonal matrix is of the form D = 0 d 2
⋮ ⋮
0 0
⋯ 0 .
⋱ ⋮
⋯ dn ]
Remark: Powers of a diagonal matrix are easy to compute:

Definition: A square matrix where all entries above the main diagonal are 0 is called lower
triangular, and a square matrix where all entries below the main diagonal are 0 is called
upper triangular. A matrix that is either lower or upper triangular is called triangular.

Eg:

Definition: A square matrix A is said to be symmetric if A = A T .

Eg:

49
Theorem: Let A and B be symmetric matrices of the same size, and let k be any scalar, then:

(a) A T is symmetric.

(b) A + B and A − B are symmetric.

(c) k A is symmetric.

Proof of (b):

Remark: In general it is not true that the product of symmetric matrices is symmetric.

Example: Prove that if A is an invertible symmetric matrix, then A−1 is also symmetric.

Example: Let A be any matrix. Prove that both A A T and A T A are symmetric.

50
 Chapter#7 – Determinants
§7.1 – Determinants by Cofactor Expansion

Given a square matrix A, we can calculate a number called the determinant of A which can be helpful
when finding A −1 if it exists.

a11 a12 ⋯ a1 n a11 a 12 ⋯ a1 n

a11 a12 ⋯ a 1n
[
The determinant of A = a21 a22
⋮ ⋮
an 1 an 2
⋱ ⋮
⋯ a nn
] [
⋯ a2 n is denoted det( A ) , or det a21 a 22
⋮ ⋮
an 1 an 2
⋱ ⋮
⋯ a nn
]
⋯ a2 n or simply

|a21 a22
⋮ ⋮
a n1 an 2
⋯ a 2n .
⋱ ⋮
⋯ ann
|
a b
For the 2×2 matrix A = [ ]
c d
, det( A )= . Thus, A −1 = .

2 1
Example: det [ ]
3 4
=

For larger matrices, we can use cofactor expansion to calculate det( A ) .

Definition: The cofactor of the ij -entry of A is:

1 2 3
Example:
[ ]
The cofactor of the (2,1)-entry of 4 5 6 is:
7 8 9

Theorem: To calculate the determinant of an n× n matrix (where n≥3 ) we choose any row or
column, multiply each entry by its cofactor and then sum the resulting products.

51
 1 2 3
Example:
[ ]
det 4 5 6 =
7 8 9

3 3 0 2
Example: det
[−1
3
6
4
0
0
0
8
1
]
1
0
4
=

Remark: The determinant of a triangular matrix is the product of the entries on the main diagonal.

Example:

52
§7.2 – Evaluating Determinants by Row Reduction

Theorem: Let A be an n×n matrix.

(a) If B is the matrix obtained by multiplying a row of A by the scalar k,

then det(B )= k det( A ) .

(b) If B is the matrix obtained by switching two rows,

then det(B )=−det ( A ) .

(c) If B is the matrix obtained by adding a scalar multiple of a row of A to another

row of A, then det(B )= det( A ) .

3 5 −2 6
Example: det
[1
2
3
2 −1 1
4 1 5
7 5 3
=
]

Corollary: (a) If E is the elementary matrix obtained by multiplying a row of I by the scalar k,
then det( E)= k .

(b) If E is the elementary matrix obtained by switching two rows of I,

then det( E)=−1 .

(c) If E is the elementary matrix obtained by adding a scalar multiple of a row of I to

another row, then det( E)= 1 .

Remark: This corollary shows that if E is an elementary matrix, then det( EA )= det( E) det( A) .

53
§7.3 – Properties of Determinants

Theorem: Properties of Determinants

Let A and B be n× n matrices and k be a scalar.

1) det( A) = det( A T )

2) A square matrix A is invertible if and only if det( A)≠ 0 .

3) det ( AB) =det (A )det (B)

4) det (kA) = k n det( A)

−1 1
5) det ( A ) =
det ( A)

0 0 2 0 6
Example: Solve for all ⃗x that satisfy A ⃗x = ⃗
b where A =
[
2 4 0
0 −1 3
1 1 3
6
0
3
] []
and ⃗
b=
2
8
9
.

Example: Suppose that A and B are both 3 × 3 matrices such that det( A) = 2 and
10 = det (4 A B T A−1 B A) . Find det(B) .

54
 Chapter#8 – Cross Product
§8.1 – Cross Product

Definition: u = (u 1 ,u2 , u3 ) and ⃗

If ⃗ v = (v 1 , v 2 ,v 3 ) are in ℝ3 , then the cross product ⃗
u × ⃗v is the
vector given by:
⃗i ⃗j ⃗k
⃗
[
u × ⃗v = det u1 u2
v1 v2
u3
v3 ]
where ⃗i = (1 ,0 ,0) , ⃗j = (0, 1,0) , ⃗
k =(0 , 0 ,1) (the standard unit vectors)

Example: Let ⃗
u =(2 ,1 , 7) and ⃗v =(3 , 2, 1)

u ,⃗v ∈ ℝ3 , then ⃗
Remark: If ⃗ u × ⃗v )= 0 and ⃗v⋅(⃗
u⋅(⃗ u × ⃗v )= 0 . That is, ⃗
u × ⃗v is perpendicular
to both ⃗ u and ⃗ v.

Properties of Cross Product:

u ×⃗v =−⃗
(a) ⃗ u × ⃗v

(b) ⃗
u ×(⃗v + w u × ⃗v ) + (⃗u × w
⃗ )= (⃗ ⃗)

(c) (⃗
u+ ⃗
v )× w u×w
⃗ =(⃗ v ×w
⃗ )+ (⃗ ⃗)

(d) k (⃗u × ⃗v ) =( k ⃗
u )× ⃗
v =⃗
u ×(k ⃗v )

0 ×⃗v = ⃗
(e) ⃗ 0

u ×⃗u = ⃗
(f) ⃗ 0

55
Example: Find the equation of the plane that passes through the points (3,−1 ,4 ) , (7, 0, 3) and
(2,1 ,−6) .

Theorem: Lagrange's Identity

2 2 2 2
u ,⃗v ∈ ℝ3 , then ‖⃗
If ⃗ u × ⃗v ‖ =‖⃗
u ‖ ‖⃗v ‖ −(⃗u⋅⃗
v)

Proof: u = (u 1 ,u2 , u3 ) and ⃗

Let ⃗ v = (v 1 , v 2 ,v 3) and work out both sides.

Theorem: ‖⃗u × ⃗
v ‖=‖⃗
u ‖‖⃗v ‖sin θ

Proof:

Theorem: The area of a parallelogram determined by vectors ⃗

v and w
⃗ is ‖⃗v × w
⃗‖ .

Proof:

56
Definition: The scalar triple product of vectors ⃗ ⃗ ∈ ℝ3 is ⃗
u ,⃗v , w u⋅(⃗ ⃗ .
v × w)

u1 u2 u3
Theorem: u⋅(⃗
⃗ v × w)
[
⃗ = det v1 v 2 v 3 .
w1 w 2 w 3 ]
Proof:

Theorem: The volume of the parallelepiped in ℝ3 determined by ⃗ ⃗ is:

u ,⃗v , w

u 1 u 2 u3

|[ w1 w 2 w 3 ]|
V = det v 1 v 2 v 3 =|⃗
u⋅(⃗
v×w
⃗ )|

Proof:

57
§8.2 – Determinants and Oriented Areas/Volumes

58
 Chapter#9 – Vector Spaces
§9.1 – Real Vector Spaces

Definition: A vector space V is a set of objects (called vectors) along with two operations: vector
addition and scalar multiplication, satisfying the following ten axioms:

u , ⃗v ∈ V then ⃗
1. If ⃗ u + ⃗v ∈ V .

u , ⃗v ∈ V , ⃗
2. For all ⃗ u + ⃗v =⃗v + ⃗u .

u , ⃗v , w
3. For all ⃗ ⃗ ∈V , (⃗
u + ⃗v ) + w u +(⃗
⃗ =⃗ ⃗) .
v +w

0 ∈V such that ⃗
4. There exists a vector ⃗ 0 + ⃗v =⃗v for all ⃗v ∈V .

5. For each ⃗v ∈V there exists a vector −⃗v ∈ V so that ⃗v + (−⃗

v )= ⃗
0.

6. For any ⃗v ∈ V and k ∈ℝ , k ⃗v ∈ V .

7. k (⃗u + ⃗v ) = k ⃗ u , ⃗v ∈ V and k ∈ℝ .
v for all ⃗
u+k⃗

8. (k + m)⃗ v for all ⃗v ∈ V and k , m ∈ ℝ .

v = k ⃗v + m ⃗

v for all ⃗v ∈V and k , m ∈ ℝ .

9. k (m⃗v ) =( km) ⃗

10. 1⃗v = ⃗v for all ⃗v ∈ V .

Example: The zero vector space

Example: ℝn

59
Example: M mn (ℝ)

Example: F (−∞ ,∞)

60
§9.2 – Subspaces

Definition: A subset W of a vector space V is called a subspace of V if W is itself a vector space

under the same addition and scalar multiplication on V.

Remark: To show W is a subspace, we don't actually have to check all ten axioms. It is enough to
check:

(i)

(ii)

(iii)

Example:

Example: The subspaces of ℝ2 and ℝ3 .

Example:

Example: C (−∞ ,∞)

Example: C [a ,b ]

Example: Pn

61
Example: Let A be a m × n matrix. The set of all solutions to the homogeneous system A ⃗x = ⃗0 is a
subspace of ℝn

Proof:

Definition: The set of all solutions to the homogeneous system A ⃗ x =⃗0 is called the solution space
of the homogeneous system. It is also called the nullspace of A.

Remark: The set of all solutions to a non-homogeneous system A ⃗x = ⃗

b is in general not a
⃗ ⃗
subspace. In fact, 0 is not within this set unless b = 0 .
⃗

Theorem: Given a set of vectors v⃗1 , v⃗2 ,⋯, v⃗n ∈ V , the set of all linear combinations of
v⃗1 , v⃗2 ,⋯, v⃗n is a subspace of V.

Proof:

62
Recall: The set of all possible linear combinations of v⃗1 , v⃗2 ,⋯, v⃗n is called the span of
v⃗1 , v⃗2 ,⋯, v⃗n . It is denoted span { v⃗1 , v⃗2 ,⋯, v⃗n } .

Remarks: By the previous theorem, span { v⃗1 , v⃗2 ,⋯, v⃗n } is a subspace of V.

If span { v⃗1 , v⃗2 ,⋯, v⃗n } =V , then we say { v⃗1 , v⃗2 ,⋯, v⃗n } spans V.

Example:

Example:

63
§9.3 – Row Space, Column Space, Null Space

a11 a12 ⋯ a1 n
Definition: For an m× n matrix A = a21 a22
⋮ ⋮
[
a m 1 am 2
⋯ a2 n ,
⋱ ⋮
⋯ a mn
]
the vectors r⃗1 =( a 11 , a12 ,⋯,a 1n )
r⃗2 =( a 21 , a22 ,⋯, a2n )
⋮
r⃗m = ( am 1 ,am 2 ,⋯,a mn ) in ℝn are called the row vectors of A.

a11 a12 a 1n

vectors of A.
[] [] []
The vectors c⃗1 = a21 , c⃗2 =
⋮
am1
a22 , ⋯ , c⃗ =
⋮
am 2
n
a 2n in ℝm are called the column
⋮
amn

Recall: Let v⃗1 , v⃗2 ,⋯, v⃗n ∈ V where V is a vector space. Then span { v⃗1 , v⃗2 ,⋯, v⃗n } is a subspace
of V.

Definition: If A is an m× n matrix, then the subspace of ℝn spanned by the row vectors of A is

called the row space of A. The subspace of ℝm spanned by the column vectors of A is
called the column space of A.

0 is a subspace of ℝn , called the

x =⃗
The solution space of the homogeneous system A ⃗
null space of A.

x1
Recall that the system A ⃗ x =⃗

x 1 c⃗1 + x 2 c⃗2 +⋯+ x n c⃗n = ⃗

b , which means:
[]
b where A = [ c⃗1 c⃗2 ⋯ c⃗n ] and ⃗x = x 2 becomes
⋮
xn

x =⃗
The linear system A ⃗ b has solutions precisely when ⃗
b is in the column space of A.

64
Definition: If W is a subspace of ℝn , then the set of all vectors in ℝn that are orthogonal to each
vector in W is called the orthogonal complement of W and is denoted W ⊥ .

Theorem: Let W be a subspace of ℝn .

(i) W ⊥ is also a subspace of ℝn

(ii) W ∩ W ⊥ = { ⃗
0}

(iii) (W ⊥ )⊥ = W

Example: In ℝ3

Theorem: Given an m×n matrix A,

(a) the null space of A and the row space of A are orthogonal complements in ℝn .

(b) The column space of A and the null space of A T are orthogonal complements in ℝm .

65
 Chapter#10 – Basis and Dimension
§10.1 – Linear Independence

Definition: If S = {v⃗1 , v⃗2 ,⋯, v⃗n } is a set of two or more vectors from a vector space, then S is said to
be a linearly independent set if no vector in S can be expressed as a linear combination
of the others.

Remark: A set consisting of a single vector { ⃗ v ≠⃗

v } is linearly independent if and only if ⃗ 0.

Theorem: A non-empty set S = { v⃗1 , v⃗2 ,⋯, v⃗n } in a vector space V is linearly independent if and
only if the only coefficients satisfying k 1 v⃗1 + k 2 v⃗2 +⋯+ k n v⃗n = ⃗
0 is k i = 0 for all i.

Example: Is S = {(1,−2,3),(5, 6,−1),(3 ,2, 1) } a linearly independent set in ℝ3 ?

Example: Is p⃗1 = 1− x , p⃗2 = 5 + 3 x − 2 x 2 , p⃗3 = 1 + 3 x − x 2 a linearly independent set in P3 ?

66
Theorem: (i) A finite set that contains ⃗
0 is not linearly independent.

(ii) A set of two vectors is linearly independent if they are not scalar multiples of each
other.

Example:

Example:

Definition: If f 1 , f 2 , ⋯ , f n are n − 1 times differentiable on (−∞ , ∞) , then

the wronskian w (x) is:

f1 f2 ⋯ fn

[ f1'
w (x) = det f 1 ''
⋮
(n−1)
f1
⋮
(n −1)
f2
f 2'

⋱ ⋮
⋯
f 2 '' ⋯ f n ''

(n−1)
⋯ fn
f n'

]
Theorem: If w ( x)≠ 0 (the zero function), then { f 1 , f 2 , ⋯ , f n } is linearly independent in
C(−∞ ,∞) .

Example: Let S = { sin x , cos x , x cos x }

67
§10.2 – Coordinates and Basis

Definition: Let S = { v⃗1 , v⃗2 ,⋯, v⃗n } be a set of n vectors from a vector space V.
S is called a basis for V if:

(i) S spans V and

(ii) S is linearly independent

Examples: 1. The standard basis for ℝn is:

2. The standard basis for Pn is:

3. The standard basis for M 2,3 is:

4.

Example: Is S = {(1, 2,1),(2, 9, 0) ,(3 ,3 ,4) } a basis for ℝ3 ?

68
Theorem: If β = { v⃗1 , v⃗2 ,⋯, v⃗n} is a basis for a vector space V, then each ⃗
v ∈ V can be expressed
as a linear combination of vectors in β in exactly one way.

Proof:

Definition: If β = { v⃗1 , v⃗2 ,⋯, v⃗n} is a basis for V and ⃗ v = k 1 v⃗1 + k 2 v⃗2 +⋯+ k n v⃗n , then the coordinate
vector of ⃗ v relative to β is:
k1
(⃗v )β = ( k 1 , k 2 ,⋯, k n ) or [ ⃗
[]
v ]β= k 2
⋮
kn

Remark: The order of coordinates in [⃗v ] β depends on the order of the vectors in β .
As a result, β is sometimes referred to as an ordered basis.

Examples: 1. Using the standard (ordered) basis for ℝn :

2. Using the standard (ordered) basis for Pn :

69
3. Using the standard (ordered) basis for M 2,3 :

4. Find the coordinate vector of (5,1 ,9) relative to basis S = {(1, 2,1),(2, 9, 0) ,(3 ,3 ,4) } .

70
§10.3 – Dimension

Definition: All bases for a finite dimensional vector space have the same number of vectors. We
define the dimension of a vector space V, denoted dim V , as the number of vectors in a
basis for V.

Examples:

Theorem: Let S be a set of vectors from an n-dimensional vector space V.

(a) If S has more than n vectors, then S is not linearly independent.

(b) If S has less than n vectors, then S does not span V.

Corollary: Let V be an n-dimensional vector space and let S be a set of n vectors from V. Then S is
a basis for V as long as S either spans V or is linearly independent.

2 2
Example: Is β = { 1 + x , 1 − x + x , 1− x } a basis for P2 ?

Theorem: If W is a subspace of a vector space V, then dim W ≤ dim V , and dim W = dim V if
and only if W = V .

Remark: Any set of vectors that spans V but is not linearly independent can be made into a basis
of V by removing particular vectors. Any set of linearly independent vectors that doesn't
span V can be made into a basis by adding particular vectors to the set.

71
Theorem: Elementary row operations do not affect the row space or nullspace of a matrix.

Example: Find a basis for the nullspace and a basis for the row space of
1 −3 4 −2 5 4
A=
[ 2 −6 9 −1 8
2 −6 9 −1 9
−1 3 −4 2 −5 −4
2
7
]

72
Elementary row operations do change the column space of a matrix, but do not change the linear
independence relations on the column vectors. That is,

Theorem: If a subset of the column vectors in the REF (or RREF) of a matrix is linearly
independent, then so is the corresponding subset of column vectors in A.

1 −3 4 −2 5 4
Example: Find a basis for the column space of A =
[
2 −6 9 −1 8
2 −6 9 −1 9
2
7
−1 3 −4 2 −5 −4
.
]

In summary, to find a basis for the

(a) nullspace of A:

• Put A into RREF

• Solve each variable in terms of the free variables
• Write the vector equation of the solution
• The vectors in this form will be a basis for the nullspace of A

(b) row space of A:

• Put A into REF

• The non-zero row vectors of the REF will be a basis for the row space of A

(c) column space of A:

• Put A into REF

• The columns of A corresponding to the columns with leading entries in the REF will be
a basis for the column space of A.

73
§10.4 – Rank, Nullity, and the Fundamental Matrix Spaces

If we consider an m × n matrix A and its transpose A T , there are six vector spaces of interest:

row space A row space A T

column space A column space A T
nullspace A nullspace A T

However, row space A = column space A T and column space A = row space A T

This leaves the four fundamental matrix spaces associated with A:

row space A column space A

nullspace A nullspace A T

Theorem: If A is any matrix, then row space A and column space A have the same dimension.

Proof:

Definition: The common dimension of the row space and column space of a matrix A is called the
rank of A, denoted rank ( A ) .

The dimension of the nullspace of A is called the nullity of A, denoted nullity ( A ) .

−1 2 0 4 5 −3
Example: Let A =
[
3 −7 2 0
2 −5 2 4
1
6
4 −9 2 −4 −4 7
4
1
]
. Calculate rank ( A ) and nullity( A ) .

74
Dimension Theorem for Matrices:

If A is a matrix with n columns, then rank ( A )+ nullity ( A)= n

Remarks: 1. If A is an m × n matrix with rank r, then A T is an n × m matrix with rank r. So

rank ( A T )= and nullity ( A T )= .

2. The maximum rank an m × n matrix could possibly have is .

75
 Chapter#11 – Change of Basis
§11.1 – Change of Basis

Theorem: If we change the basis for a vector space V from an old basis β = { v⃗1 , v⃗2 ,⋯, v⃗n} to a new
basis β ' = {v⃗1 ' , v⃗2 ' ,⋯, v⃗n ' } , then for each vector ⃗ v ∈ V , the old coordinate vector [ ⃗
v ]β
is related to the new coordinate vector [ ⃗ v ]β ' by the equation:

[ ⃗v ] β' = Pβ '←β [ ⃗v ] β , where Pβ' ← β = [ [ v⃗1 ] β' [ v⃗2 ] β' ⋯ [ v⃗n ] β' ] .

Remarks: 1. That is, the columns of Pβ' ← β are the coordinate vectors of the vectors in β relative
to basis β' .

2. The matrix Pβ' ← β is called the transition matrix from β to β ' .

3. The order of the columns in Pβ' ← β depend on the order of the vectors in β and β ' ,
so these bases are referred to as ordered bases.

2 2 1 3 1 −1
Example:
{[ ] [ ] [ ]}
Let β = 1 , −1 , 2
1 1 1
Find Pβ' ← β .
and β ' =
{[ ] [ ] [ ]}
1 , 1 , 0
−5 −3 2
be two (ordered) bases for ℝ3 .

76
Theorem: Pβ' ← β is invertible, and Pβ'−1← β = Pβ ← β' .

Theorem: Let β and β ' be (ordered) bases for a vector space V. Then for any basis ϵ ,

Pβ' ← β = Pβ' ← ϵ P ϵ ← β

Remark: Since the above theorem works for any basis ϵ , it is easiest to use the standard basis. In
this case, Pϵ ← β = [ [ v⃗1 ] ϵ [ v⃗2 ] ϵ ⋯ [ v⃗n ] ϵ ] = [ v⃗1 | v⃗2 | ⋯ | v⃗n ] .

Example: Let β and β ' be bases for ℝ2 . If β ' =

1 , 2
2 3 {[ ] [ ] } and the change of basis matrix from

1 −1
β to β ' is Pβ' ← β = [
1 2 ]
, find β .

77
§11.2 – Orthogonal and Orthonormal Bases

An orthogonal set of vectors is a set of vectors that are mutually orthogonal to each other.

Example: S = { (1,1,1,1) , ( 1,−1,1 ,−1) , (1,1 ,−1 ,−1) } is an orthogonal set of vectors. You may
wish to verify, using dot product, that each pair of vectors in S are orthogonal.

An orthonormal set of vectors is an orthogonal set of vectors such that each vector is a unit vector.

Example: If we normalize each vector in S above we obtain the orthonormal set

1 1 1 1 1 1 1 1 1 1 1 1
T= {( )(
, , , , ,− , ,− , , ,− ,−
2 2 2 2 2 2 2 2 )(
2 2 2 2 )}
It is often convenient to use an orthonormal basis for a vector space since it is quite easy to compute
coordinate vectors relative to an orthonormal basis.

Theorem: If γ = { u⃗1 , u⃗2 , ⋯ u⃗n } is an (ordered) orthonormal basis for vector space V, and if
⃗v ∈ V , then the coordinate vector of ⃗v relative to γ is:
⃗v⋅⃗
u1

[]
[ ⃗v ] γ = ⃗v⋅⃗
⃗
⋮
u2

v⋅⃗
un

Proof:

Now, all we need is a way to convert a general basis into an orthonormal basis. For this, see the next
section.

78
§11.3 – Gram-Schmidt Process

The Gram-Schmidt Process is an algorithm for converting a general basis β = { b⃗1 , b⃗2 , ⋯ b⃗n } into an
orthonormal basis γ = { u⃗1 , u⃗2 , ⋯ u⃗n } .

Gram-Schmidt Process

The idea is to first form an orthogonal basis β' = { v⃗1 , v⃗2 , ⋯ v⃗n } using the following steps, and
afterwards normalize each vector v⃗i in β' to complete the orthonormal basis
γ = { u⃗1 , u⃗2 , ⋯ u⃗n } .

Step#1: v⃗1 = b⃗1

Step#2: v⃗2 = b⃗2 − projv⃗ b⃗2

1

Step#3: v⃗3 = b⃗3 − projv⃗ b⃗3 − projv⃗ b⃗3

1 2

Step#4: v⃗4 = b⃗4 − projv⃗ b⃗4 − projv⃗ b⃗4 − projv⃗ b⃗4

1 2 3

Example: Let S be the subspace of ℝ4 spanned by v⃗1 =(1,1,1,1) , v⃗2 =(2,0,2,0) , v⃗3 =(5,9 ,−3,1) .
v ] γ where ⃗v =(1,2,3,4) .
Find an orthonormal basis γ for S, and use it to determine [ ⃗

79
§11.4 – Dimension Formula for Orthogonal Complements

Lemma: Let V be a subspace of ℝn , then V ∪V ⊥ =ℝ n .

Proof:

Theorem: Let V be a subspace of ℝn , then dim( V ) + dim(V ⊥ ) = n .

Proof:

80
 Chapter#12 – Linear Transformations
§12.1 – Linear Transformations

A function f in mathematics is a rule that associates with each element of one set A (called the domain)
exactly one element in another set B (called the codomain). If f associates the element b ∈ B to the
element a ∈ A , we write f (a)= b and call b the image of a under f. The set of all b ∈ B that are
images of elements in the domain is called the range of f.

Remarks: 1. Every element in the domain must get mapped to exactly one element in the
codomain.

2. Two different elements in the domain can get mapped to the same element in the
codomain.

3. Not every element in the codomain needs to be in the range of f.

4. To indicate that f is a function that maps elements from A to B, we write f : A → B .

Definition: When the domain V and codomain W are (possibly different) vector spaces, we call a
function T : V → W a transformation from V to W. If V and W are the same vector
space, then T : V → V is called an operator on V.

Example: Consider the transformation T : ℝ 2 → ℝ3 where T ((x , y)) =( x , y , x + y ) .

81
Definition: Let T : V → W be a transformation. If for every v⃗1 , v⃗2 ∈ V and k ∈ ℝ

(i) T ( v⃗1) + T ( v⃗2 )= T ( v⃗1 + v⃗2 ) and

(ii) k T ( v⃗1)= T (k v⃗1 )

then T is called a linear transformation.

Example: Consider the last example where T : ℝ 2 → ℝ3 was given by T ((x , y)) =( x , y , x + y ) .

82
Example: Consider T : ℝ n → ℝ m where T (⃗v )= A ⃗
v for a particular m× n matrix A.

Theorem: Every linear transformation between finite dimensional vector spaces can be expressed
by multiplication by a matrix. We call this matrix the standard matrix for the
transformation.

To find the standard matrix [T ] for the transformation, we find where the standard basis vectors are
sent by the transformation. That is,

[T ] = [ T ( e⃗1) ∣ T ( e⃗2 ) ∣ ⋯ ∣ T ( e⃗n) ]

Example: The standard matrix for the transformation T : ℝ 2 → ℝ3 given by

T ((x , y)) =( x , y , x + y )

83
§12.2 – Linear Transformations in ℝ2 and ℝ3

In ℝ2 and ℝ3 the following are examples of linear operators:

• reflection about a line through the origin ( ℝ2 )

• reflection about a plane through the origin ( ℝ3 )
• projection onto a line through the origin ( ℝ2 or ℝ3 )
• projection onto a plane through the origin ( ℝ3 )
• rotation about the origin ( ℝ2 )
• rotation about a line through the origin ( ℝ3 )
• shear

Example: In ℝ2 reflection about a line that creates angle θ with x-axis.

Example: In ℝ3 , reflection about the xz-plane.

84
Example: In ℝ2 , orthogonal projection onto a line that creates an angle θ with x-axis.

Example: In ℝ3 projection onto the xy-plane.

85
Example: in ℝ2 , counter-clockwise rotation of angle θ about the origin.

Example: In ℝ3 , rotation of θ about the x-axis with positive (right-hand rule) orientation.

Example: In ℝ3 , expansion of factor of k.

Remark: If k > 1 , we call this an dilation. If 0 < k < 1 , we call this a contraction.

86
Example: Shear in ℝ2 parallel to the x-axis

87
§12.3 – Properties of Matrix Transformations

Definition: Performing successive transformations, one after another, are called compositions of
transformations.

If T 1 : V → W and T 2 : W → X , then the composition T 2 ∘ T 1 : V → X .

That is, T 2 ∘ T 1 (⃗
v )= T 2 (T 1 (⃗
v )) .

Theorem: If T 1 and T 2 are linear transformations, then T 2 ∘ T 1 is also a linear transformation.

Moreover, [ T 2 ∘ T 1 ] = [T 2] [T 1 ] .

2 2
Example: Let T 1 : ℝ2 → ℝ2 be projection onto the y-axis and T 2 : ℝ → ℝ be counter-clockwise
rotation of π radians about the origin.
4

Definition: A transformation T : V → W is said to be one-to-one if T maps different vectors in V to

different vectors in W.

Remarks: 1. T is one-to-one if each vector in the range of T is the image of exactly one vector in V.

2. T is one-to-one if T ( v⃗1 )= T ( v⃗2 ) implies that v⃗1 = v⃗2 .

88
Examples:

Example: Is T : ℝ 2 → ℝ3 where T (( x , y ))= (x , y , x + y ) a one-to-one linear transformation?

Definition: Let T : V → W be a linear transformation. The set of all vectors ⃗ v ∈ V such that
T (⃗v )= ⃗
0 W is called the kernal of T, denoted ker (T) . The range of T is also denoted
R (T ) .

n
Remarks: 1. If T : ℝ n → ℝm has standard matrix [T ] , then ker (T )= {⃗
v ∈ ℝ ∣ [T ] ⃗v = ⃗
0} .
That is, ker (T) = nullspace of [T ] .
m n
2. R (T ) = {w ⃗ for some ⃗v ∈ℝ } . That is, R (T )= column space of [T ] .
⃗ ∈ ℝ ∣ [T ]⃗v = w

3. To describe the kernal and range of T, we find bases for the nullspace and
column space of [T ] respectively. Take note that the kernal of T is a subspace of ℝn
and the range of T is a subspace of ℝm .

89
Definition: Given linear transformation T : V → W , ker (T) and R (T ) are subspace of V and W
respectively. Thus it makes sense to talk about the dimensions of these subspaces. The
dimension of the kernal of T is called the nullity of T and the dimension of the range of
T is called the rank of T. That is,

dim(ker (T ))=nullity(T ) and dim( R(T ))=rank(T )

Remark: Notice that if [T ] is the matrix for a linear transformation, then

nullity (T ) = nullity ([T ]) and rank (T ) = rank([T ]) . Therefore...

Theorem: Given linear transformation T : V → W , rank (T ) + nullity (T )= dim (V ) .

Theorem: A linear transformation T : V → W is one-to-one if and only if ker (T ) = {⃗

0}.

Proof:

Remark: This gives us another way of determining whether a linear transformation is one-to-one.
Determine ker (T ) . If ker (T ) = { ⃗
0 } , then T is one-to-one.

Example: Let T ((x , y)) =( x , y , x + y ) .

90
§12.4 – Invertible Linear Transformations

If T : ℝ n → ℝn is a one-to-one linear operator, then the matrix [T ] for T will be invertible. It follows
that the matrix [T ]−1 will also be the matrix for a linear operator from ℝn to ℝn and is called the
inverse of T, denoted T −1 . The linear operators T and T −1 cancel the effect of one another in the sense
that for all ⃗x ∈ ℝ n
T ∘ T −1 (⃗x )= [T ][T ]−1 ⃗x = ⃗x and T −1 ∘ T (⃗x )= [T ]−1 [T ]⃗x = ⃗x

Example: Let T : ℝ 2 → ℝ 2 be the linear operator so T ( x , y)= (3 x + y , x−2 y ) . Show that T is

one-to-one and find an expression for T −1 .

91
§12.5 – Orthogonal Matrices

An n×n matrix with the property A−1 = A T is called an orthogonal matrix.

4 3

[ ]
0 −
5 5
9 4 12
Example: A = − −
25 5 25
12 3 16
25 5 25

Example: A = [ cos θ
sin θ
−sin θ
cos θ ]
Theorem: Given an n×n matrix A, the following are equivalent:

1) A is an orthogonal matrix

2) The row vectors of A form an orthonormal set

3) The column vectors of A form an orthonormal set

Proof:

92
Properties of Orthogonal Matrices:

1. The inverse of an orthogonal matrix is an orthogonal matrix.

2. The product of two orthogonal matrices is an orthogonal matrix.

3. If A is orthogonal, then det( A)= 1 or det( A)=−1

Proof: (of 1 and 2)

Theorem: A is an n×n orthogonal matrix if and only if ‖ A ⃗x‖=‖⃗x‖ for all x ∈ ℝ n .

Proof:

Remark: It follows that if [T ] is an orthogonal matrix, then T is a linear operator that preserves
distances. That is, ‖T (⃗x )‖ =‖⃗x‖ for all x ∈ℝ n .

For example the standard matrices for rotation operators as well as for reflection
operators are orthogonal matrices.

93
§12.6 – Custom Bases

At times it can be convenient to use a different coordinate system other than the standard coordinates in
ℝn . In particular, if we are rotating around a vector that is not that is not one of the axes, reflecting
about a plane that is not one of the standard planes, or projecting onto a plane that is not one of the
standard ones. In this case we would like to have a transformation matrix that acts on (coordinate)
vectors relative to a certain basis. We use the notation [T ]β to denote the matrix for T relative to basis
β and it works as:
[T ]β [⃗x ]β =[T (⃗x )]β

To build [T ]β , where β = {b⃗1 , b⃗2 , ⋯ , b⃗n } we use [T ]β = [ [T ( b⃗1)]β | [T ( b⃗2)]β | ⋯ | | [T ( b⃗n)]β ] .

That is, we find where T send each basis vector, then for each image we find its coordinate vector
relative to basis β .

One of the most natural ways to use a transformation matrix relative to a particular basis is to
accompany it with some change of basis matrices. This allows us to take a vector in ℝn , find its
coordinate vector relative to a convenient basis, transform it with [T ]β , and finally convert it back to
standard coordinates.

In matrix notation this looks like T (⃗x )= P ϵ←β [T ]β Pβ← ϵ ⃗x .

(Recall that Pϵ←β = [ b⃗1 | b⃗2 | ⋯ | b⃗n ] and Pβ← ϵ = P−1

ϵ←β )

Example: Find the matrix [T ] for the linear operator projection onto the plane x + 2 y−3 z = 0 .

94
 Chapter#13 – Eigenvalues and Eigenvectors
§13.1 – Eigenvalues and Eigenvectors

Definition: If A is an n× n matrix, then a non-zero vector ⃗x in ℝn is called an eigenvector of A if

A⃗ x = λ ⃗x for some scalar λ ∈ ℝ .

In this case, λ is called an eigenvalue of A and ⃗x is an eigenvector corresponding to

eigenvalue λ.

Remark: Although ⃗
0 cannot be an eigenvector, λ = 0 can be an eigenvalue.

Example:

How do we find eigenvalues?

Theorem: If A is an n× n matrix, then λ is an eigenvalue of A if and only if it satisfies

det (λ I − A) = 0 . This is called the characteristic equation of A.

Proof:

3 0
Example: A= [ 8 −1 ]

95
Remark: The characteristic equation of A is always a polynomial of degree n. Thus an n × n
matrix always has n eigenvalues (though some of them may be repeated and/or complex.

Theorem: If A is a triangular matrix (upper, lower, diagonal), then the eigenvalues of A are the
entries on the main diagonal.

How do we find eigenvectors?

So ⃗x is an eigenvector corresponding to eigenvalue λ if ⃗x is non-zero and is in the null space of

λ I − A . It follows that the set of eigenvectors corresponding to λ along with ⃗
0 is a subspace of
n
ℝ . It is called the eigenspace corresponding to eigenvalue λ.

1 −3 3
Example:
[ ]
Find the eigenvalues of A = 3 −5 3 and bases for the eigenspaces corresponding

to each eigenvalue.
6 −6 4

96
97
§13.2 – Diagonalization

Definition: Two n × n matrices A and B are said to be similar if B = P −1 A P for some invertible
matrix P.

Theorem: If A and B are similar matrices, then they have the same:

• determinant • characteristic polynomial

• rank • eigenvalues
• nullity • eigenspace dimensions
• trace • invertibility

Remark: Since diagonal matrices are so nice to work with, we are curious whether a given matrix
A is similar to a diagonal matrix.

Definition: An n × n matrix A is said to be diagonalizable if it is similar to a diagonal matrix D.

That is, if there exists an invertible matrix P such that P−1 A P = D .

Definition: The algebraic multiplicity of an eigenvalue λ is the number of times λ appears as a

root of the characteristic polynomial of A.

The geometric multiplicity of an eigenvalue λ is the dimension of the eigenspace

corresponding to λ .

Theorem: An n × n matrix A is diagonalizable if and only if for every eigenvalue λ , its algebraic
multiplicity matches its geometric multiplicity. Moreover, in this case, P is the matrix
whose columns are the basis vectors of the eigenspaces, and D is the diagonal matrix
whose entries on the main diagonal are the eigenvalues of A.

Remarks: 1.

2.

3.

98
 2 0 −2
Example:
[ ]
Is A = 0 3 0 diagonalizable? If so, then find a matrix P so that P−1 A P = D .
0 0 3

99
 2 0 −2
Example: 6

[ ]
Find A where A = 0 3 0 .
0 0 3

100
§13.3 – Orthogonal Diagonalization

At times an n×n diagonalizable matrix A can be expressed in the form A = PDP−1 where P is an
orthogonal matrix (and D is a diagonal matrix). In this case we get that A = PDPT , and we say that A
is orgonally diagonalizable.

Notice that in this case, transposing both sides will give that

It follows that:

Theorem: An n×n matrix A is orthonally diagonalizable if and only if A is a symmetric matrix.

Here is an algorithm for obtaining an orthogonal matrix P which diagonalizes a symmetric matrix A.

Step#1: Find a basis for each eigenspace of A

Step#2: Use the Gram-Schmidt Process on each basis to convert them into orthonormal bases.

Step#3: Form the matrix P whose columns are the basis vectors from Step#2.

Remark: This proof that this algorithm works relies on the following lemma.

Lemma: If A is a symmetrix matrix, then the eigenvectors from different eigenspaces are
orthogonal.

Proof:

101
§13.4 – Complex Vector Spaces

Definition: A vector space where the scalars can be complex numbers is called a complex vector
space. The set of all n-tuples of complex numbers is a complex vector space and is
denoted ℂn . Scalars are complex numbers and vector addition and scalar multiplication
are defined component-wise.

Example: Find the eigenvalues and bases for the eigenspaces for the matrix A= 4 −5 .
[ ]
1 0

Theorem: If λ is a complex eigenvalue of a real n×n A, then λ̄ is also an eigenvalue.

x is an eigenvector corresponding to eigenvalue λ , then ¯⃗

If, ⃗ x is an eigenvector
corresponding to eigenvalue λ̄ .

102
103