MATH10212 Linear Algebra

Lecture Notes

Last change: 23 Apr 2018

Textbook
Students are strongly advised to ac-
quire a copy of the Textbook:
D. C. Lay. Linear Algebra and its
Applications. Pearson, 2006. ISBN
0-521-28713-4.
Other editions can be used as well; the
book is easily available on Amazon (this
includes some very cheap used copies)
and in Blackwell’s bookshop on campus.
These lecture notes should be
treated only as indication of the
content of the course; the text-
book contains detailed explana-
tions, worked out examples, etc.

About Homework
The Undergraduate Student Hand- Normally, homework assignments will
book 1 says: consist of some odd numbered exercises
from the sections of the Textbook cov-
As a general rule for each hour in class
ered in the lectures up to Wednesday in
you should spend two hours on indepen-
particular week. The Textbook contains
dent study. This will include reading lec-
answers to most odd numbered exercises
ture notes and textbooks, working on ex-
(but the numbering of exercises might
ercises and preparing for coursework and
change from one edition to another).
examinations.
Homework N (already posted on the
In respect of this course, MATH10212
webpage and BB9 space of the course)
Linear Algebra B, this means that stu-
should be returned to Supervision
dents are expected to spend 8 (eight!)
Classes teachers early in Week N, for
hours a week in private study of Linear
marking and discussion at supervision
Algebra.
classes on Tuesday and Wednesday.

1
http://www.maths.manchester.ac.uk/study/undergraduate/information-for-current-students/
undergraduatestudenthandbook/teachingandlearning/timetabledclasses/.
 2

Communication
The Course Webpage is
http://www.maths.manchester.ac.
uk/~avb/math10212-Linear-Algebra-B.
html
The Course Webpage page is updated
almost daily, sometimes several times a
day. Refresh it (and files it is linked to)
in your browser, otherwise you may miss
the changes.
Email: Feel free to write to me with
questions, etc., at the address
alexandre.borovik@manchester.ac.
uk
but only from your university e-mail
account.
Emails from Gmail, Hotmail, etc. auto-
matically go to spam.

What is Linear Algebra?

It is well-known that the total cost of where
a purchase of amounts g1 , g2 , g3 of some  
p1
 
g1
goods at prices p1 , p2 , p3 , respectively, is P = p2  and G = g2  ,
an expression p3 g3
3
X and T (transposition) turns row
p1 g1 + p2 g2 + p3 g3 = pi gi . vectors into column vectors, and
i=1
vice versa:
Expressions of this kind,  T
p1
T
 
a1 x 1 + · · · + an x n P = p2  = p 1 p2 p3 ,

p3
are called linear forms in variables
and
x1 , . . . , xn with coefficients a1 , . . . , an .  
 T p1
p1 p2 p3 = p2  .

• Linear Algebra studies the mathe-
p3
matics of linear forms.

• Over the course, we shall develop • Physicists use even more short no-
increasingly compact notation for tation and, instead of
operations of Linear Algebra. In 3
X
particular, we shall discover that p1 g1 + p2 g2 + p3 g3 = pi gi
i=1
p1 g1 + p2 g2 + p3 g3 write
can be very conveniently written as p1 g 1 + p2 g 2 + p3 g 3 = p i g i ,
 
  g1 omitting the summation sign en-
p1 p2 p3 g2  tirely. This particular trick was
g3 invented by Albert Einstein, of all
people. I do not use “physics”
and then abbreviated tricks in my lectures, but am
 
prepared to give a few addi-
 g1
p1 p2 p3 g2  = P T G,

tional lectures to physics stu-
g3 dents.
 3

• Warning: Increasingly compact • Unlike, say, Calculus, Linear Alge-

notation leads to increasingly com- bra focuses more on the develop-
pact and abstract language. ment of a special mathematics lan-
guage rather than on procedures.
MATH10212 • Linear Algebra B • Lecture 1 • Linear systems • Last change: 23 Apr 2018
Lecture 1 Systems of linear equations [Lay 1.1]
A linear equation in the variables
x1 , . . . , xn is an equation that can be
written in the form
a1 x 1 + a2 x 2 + · · · + an x n = b
where b and the coefficients a1 , . . . , an
are real numbers. The subscript n can
be any natural number.
A system of simultaneous linear
equations is a collection of one or more
linear equations involving the same vari-
ables, say x1 , . . . , xn . For example,
x1 + x2 = 3
x1 − x2 = 1
We shall abbreviate the words “a sys-
tem of simultaneous linear equa-
tions” just to “a linear system”.
A solution of the system is a list
(s1 , . . . , sn ) of numbers that makes each
equation a true identity when the values
s1 , . . . , sn are substituted for x1 , . . . , xn ,
respectively. For example, in the system
above (2, 1) is a solution.
The set of all possible solutions is called
the solution set of the linear system.
Two linear systems are equivalent if the
have the same solution set.
We shall be use the following elemen-
tary operations on systems od simul-
taneous liner equations:
Replacement Replace one equation by
the sum of itself and a multiple of
another equation.
Interchange Interchange two
tions.
Scaling Multiply all terms in a equa-
tion by a nonzero constant.
Note: The elementary operations are re-
versible.
4
Theorem: Elementary operations
preserve solutions.
If a system of simultaneous linear equa-
tions is obtained from another system by
elementary operations, then the two sys-
tems have the same solution set.
We shall prove later in the course that a
system of linear equations has either
• no solution, or
• exactly one solution, or
• infinitely many solutions,
under the assumption that the coeffi-
cients and solutions of the systems are
real or complex numbers.
A system of linear equations is said to
be consistent it if has solutions (either
one or infinitely many), and a system in
inconsistent if it has no solution.
Solving a linear system
The basic strategy is
to replace one system with
an equivalent system (that is,
with the same solution set)
which is easier to solve.
Existence and uniqueness
questions
• Is the system consistent?
equa-
• If a solution exist, is it unique?
Equivalence of linear systems
• When are two linear systems
equivalent?
MATH10212 • Linear Algebra B • Lecture 1 • Linear systems • Last change: 23 Apr 2018 5

Checking solutions

Given a solution of the system of linear equations, how to check that it is correct?
MATH10212 • Linear Algebra B • Lecture 2 • Row reduction and echelon forms • Last change: 23 Apr 2018 6

Lecture 2 Row reduction and echelon forms [Lay 1.2]

Matrix notation Scaling Multiply all entries in a row by
a nonzero constant.
It is convenient to write coefficients of a
linear system in the form of a matrix, a
rectangular table. For example, the sys- The two matrices are row equivalent if
tem there is a sequence of elementary row op-
erations that transforms one matrix into
x1 − 2x2 + 3x3 = 1 the other.
x1 + x2 = 2 Note:
x2 + x3 = 3

has the matrix of coefficients • The row operations are reversible.

 
1 −2 3
1 1 0 • Row equivalence of matrices is an
equivalence relation on the set of
0 1 1
matrices.
and the augmented matrix
Theorem: Row Equivalence.
 
1 −2 3 1
1 1 0 2 ;
If the augmented matrices of two linear
0 1 1 3
systems are row equivalent, then the two
notice how the coefficients are aligned systems have the same solution set.
in columns, and how missing coefficients A nonzero row or column of a matrix is
are replaced by 0. a row or column which contains at least
The augmented matrix in the example one nonzero entry.
above has 3 rows and 4 columns; we say We can now formulate a theorem (to be
that it is a 3×4 matrix. Generally, a ma- proven later).
trix with m rows and n columns is called
an m × n matrix. Theorem: Equivalence of linear
systems.
Elementary row operations
Two linear systems are equivalent if and
Replacement Replace one row by the only if the augmented matrix of one of
sum of itself and a multiple of an- them can be obtained from the aug-
other row. mented matrix of another system by frow
operations and insertion / deletion of
Interchange Interchange two rows. zero rows.
MATH10212 • Linear Algebra B • Lecture 3 • Solution of Linear Systems • Last change: 23 Apr 2018
Lecture 3 Solution of Linear Systems [Lay 1.2]
A nonzero row or column of a matrix is
a row or column which contains at least
one nonzero entry. A leading entry of
a row is the leftmost nonzero entry (in a
non-zero row).
Definition. A matrix is in echelon
form (or row echelon form) if it has
the following three properties:
1. All nonzero rows are above any row
of zeroes.
2. Each leading entry of a row is in
column to the right of the leading
entry of the row above it.
3. All entries in a column below a
leading entry are zeroes.
If, in addition, the following two
conditions are satisfied,
4. All leading entries are equal 1.
5. Each leading 1 is the only non-zero
entry in its column
then the matrix is in reduced echelon
form.
An echelon matrix is a matrix in ech-
elon form.
Any non-zero matrix can be row re-
duced (that, transformed by elementary
row operations) into a matrix in echelon
form (but the same matrix can give rise
to different echelon forms).
Examples. The following is a schematic
presentation of an echelon matrix:
 
 ∗ ∗ ∗ ∗
 0  ∗ ∗ ∗
0 0 0  ∗
and this is a reduced echelon matrix:
 
1 0 ∗ 0 ∗
0 1 ∗ 0 ∗
0 0 0 1 ∗
7
Theorem 1.2.1: Uniqueness of the
reduced echelon form.
Each matrix is row equivalent to one and
only one reduced echelon form.
Definition. A pivot position in a ma-
trix A is a location in A that corresponds
to a leading 1 in the reduced echelon
form of A. A pivot column is a col-
umn of A that contains a pivot position.
Example for solving in the lecture
(The Row Reduction Algorithm):
 
0 2 2 2 2
1 1 1 1 1

1 1 1 3 3

1 1 1 2 2
A pivot is a nonzero number in a pivot
position which is used to create zeroes in
the column below it.
A rule for row reduction:
1. Pick the leftmost non-zero column
and in it the topmost nonzero en-
try; it is a pivot.
2. Using scaling, make the pivot
equal 1.
3. Using replacement row opera-
tions, kill all non-zero entries in the
column below the pivot.
4. Mark the row and column contain-
ing the pivot as pivoted.
5. Repeat the same with the matrix
made of not pivoted yet rows and
columns.
6. When this is over, interchange
the rows making sure that the re-
sulting matrix is in echelon form.
7. Using replacement row opera-
tions, kill all non-zero entries in the
column above the pivot entries.
MATH10212 • Linear Algebra B • Lecture 3 • Solution of Linear Systems • Last change: 23 Apr 2018 8

Solution of Linear Systems

When we converted the augmented ma- the augmented matrix has no row of the
trix of a linear system into its reduced form
row echelon form, we can write out the  
entire solution set of the system. 0 ··· 0 b with b nonzero
Example. Let
  If a linear system is consistent, then the
1 0 −5 1 solution set contains either
0 1 1 4
0 0 0 0
(i) a unique solution, when there are
be the augmented matrix of a a linear no free variables, or
system; then the system is equivalent to
(ii) infinitely many solutions, when
x1 − 5x3 = 1
there is at least one free variable.
x2 + x3 = 4
0 = 0
Using row reduction to solve a lin-
The variables x1 and x2 correspond to ear system
pivot columns in the matrix and a re
called basic variables (also leading or 1. Write the augmented matrix of the
pivot variables). The other variable, system.
x3 is a free variable.
Free variables can be assigned arbitrary 2. Use the row reduction algorithm
values and then leading variables ex- to obtain an equivalent augmented
pressed in terms of free variables: matrix in echelon form. Decide
whether the system is consistent.
x1 = 1 + 5x3
x2 = 4 − x3 3. if the system is consistent, get the
x3 is free reduced echelon form.

Theorem 1.2.2: Existence and 4. Write the system of equations cor-

Uniqueness
A linear system is consistent if and only
if the rightmost column of the aug-
mented matrix is not a pivot column—
that is, if and only if an echelon form of
responding to the matrix obtained
in Step 3.
5. Express each basic variable in
terms of any free variables appear-
ing in the equation.
MATH10212 • Linear Algebra B • Lecture 4 • Vector equations • Last change: 23 Apr 2018 9

Lecture 4 Vector equations [Lay 1.3]

A matrix with only one column is called 1. u + v = v + u
a column vector, or simply a vector.
2. (u + v) + w = u + (v + w)
Rn is the set of all column vectors with
n entries. 3. u + 0 = 0 + u = u
A row vector: a matric with one row. 4. u + (−u) = −u + u = 0
Two vectors are equal if and only if they 5. c(u + v) = cu + cv
have
6. (c + d)u = cu + du
• the same shape,
7. c(du) = (cd)u
• the same number of rows,
8. 1u = u
• and their corresponding entries are
equal. (Here −u denotes (−1)u.)
The set of al vectors with n entries is
denoted Rn . Linear combinations
The sum u+v of two vectors u and v in
Rn is obtained by adding corresponding Given vectors v1 , v2 , . . . , vp in Rn and
entries in u and v. For example in R2 scalars c1 , c2 , . . . , cp , the vector
     
1 −1 0 y = c1 v1 + · · · cp vp
+ = .
2 −1 1
is called a linear combina-
The scalar multiple cv of a vector v tion of v1 , v2 , . . . , vp with weights
and a real number (“scalar”) c is the c1 , c2 , . . . , cp .
vector obtained by multiplying each en-
try in v by c. For example in R3 ,
    Rewriting a linear system as
1 1.5
1.5  0 =
  0 . a vector equation
−3 −2
Consider an example: the linear system
The vector whose entries are all zeroes is
x2 + x3 = 2
called the zero vector and denoted 0:
  x 1 + x2 + x3 = 3
0
0 x1 + x2 − x3 = 2
0 =  ..  .
 
. can be written as equality of two vectors:
0    
x2 + x3 2
x1 + x2 + x3  = 3
Operations with row vectors are defined
in a similar way. x1 + x2 − x3 2

which is the same as

Algebraic properties of Rn        
0 1 1 2
For all u, v, w ∈ Rn and all scalars c and x1 1 + x2 1 + x3  1  = 3
d: 1 1 −1 2
MATH10212 • Linear Algebra B • Lecture 4 • Vector equations • Last change: 23 Apr 2018 10

Let us write the matrix In particular b can be generated by a lin-

  ear combination of a1 , a2 , . . . , an if and
0 1 1 2
1 only if there is a solution of the corre-
1 1 3
sponding linear system.
1 1 −1 2
in a way that calls attention to its
Definition. If v1 , . . . , vp are in Rn ,
columns:
then the set of all linear combination of
v1 , . . . , vp is denoted by
 
a1 a2 a3 b

Denote Span{v1 , . . . , vp }
     
0 1 1
and is called the subset of Rn spanned
a1 = 1 , a2 = 1 , a3 =  1
(or generated) by v1 , . . . , vp ; or the
1 1 −1
span of vectors v1 , . . . , vp .
and
That is, Span{v1 , . . . , vp } is the collec-
 
2
b = 3 , tion of all vectors which can be written
2 in the form
then the vector equation can be written
c1 v1 + c2 v2 + · · · + cp vp
as
x1 a1 + x2 a2 + x3 a3 = b.
with c1 , . . . , cp scalars.
Notice that to solve this equation is the
We say that vectors v1 , . . . , vp span Rn
same as
if
express b as a linear combi- Span{v1 , . . . , vp } = Rn
nation of a1 , a2 , a3 , and find
all such expressions. We can reformulate the definition of
span as follows: iv a1 , . . . , ap ∈ Rn , then
Therefore
Span{a1 , . . . , ap } = {b ∈ Rn such that
solving a linear system is the same as [a1 · · · ap |b]
finding an expression of the vector of the
is the augmented matrix
right part of the system as a linear com-
bination of columns in its matrix of co- of a consistent system
efficients. of linear equations},
A vector equation
or, in simpler words,
x1 a1 + x2 a2 + · · · + xn an = b.
Span{a1 , . . . , ap } = {b ∈ Rn such that
has the same solution set as the linear the system of equations
system whose augmented matrix is
x1 a1 + · · · + xp ap = b
 
a1 a2 · · · an b has a solution}.
MATH10212 • Linear Algebra B • Lecture 5 • The matrix equation Ax = b • Last change: 23 Apr 2018 11

Lecture 5: The matrix equation Ax = b [Lay 1.4]

Definition. If A is an m × n matrix, Theorem 1.4.3: If A is an m × n ma-
with columns a1 , . . . , an , and if x is in trix, with columns a1 , . . . , an , and if x is
Rn , then the product of A and x, de- in Rn , the matrix equation
noted Ax, is the linear combination of
the columns of A using the correspond- Ax = b
ing entries in x as weights:
has the same solution set as the vector
  equation
x
   .1 
Ax = a1 a2 · · · an  ..  x1 a1 + x2 a2 + · · · + xn an = b
xn
which has the same solution set as the
= x1 a1 + x2 a2 + · · · + xn an system of linear equations whose aug-
mented matrix is
 
a1 a2 · · · an b .
Example. The system

x2 + x3 = 2
Existence of solutions. The equation
x1 + x2 + x3 = 3 Ax = b has a solution if and only if b is
x1 + x2 − x3 = 2 a linear combination of columns of A.

was written as Theorem 1.4.4: Let A be an m×n ma-

trix. Then the following statements are
x1 a1 + x2 a2 + x3 a3 = b. equivalent.

where (a) For each b ∈ Rn , the equation

      Ax = b has a solution.
0 1 1
a1 = 1 , a2 = 1 , a3 =
     1 (b) Each b ∈ Rn is a linear combina-
1 1 −1 tion of columns of A.

and   (c) The columns of A span Rn .

2
b = 3 . (d) A has a pivot position in every row.
2
Row-vector rule for computing Ax.
In the matrix product notation it be- If the product Ax is defined then the ith
comes entry in Ax is the sum of products of

0 1
   
1 x1 2 corresponding entries from the row i of
1 1 1 x2  = 3 A and from the vector x.
1 1 −1 x3 2 Theorem 1.4.5: Properties of the
or matrix-vector product Ax.
Ax = b If A is an m × n matrix, u, v ∈ Rn , and
where c is a scalar, then
 
  x1 (a) A(u + v) = Au + Av;
A = a1 a2 a3 , x = x2  .
x3 (b) A(cu) = c(Au).
MATH10212 • Linear Algebra B • Lecture 5 • The matrix equation Ax = b • Last change: 23 Apr 2018
Homogeneous linear systems
A linear system is homogeneous if it
can be written as
Ax = 0.
A homogeneous system always has at
least one solution x = 0 (trivial solu-
tion).
Therefore for homogeneous systems an
important question os existence of a
12
nontrivial solution, that is, a nonzero
vector x which satisfies Ax = 0:
The homogeneous system Ax = b has
a nontrivial solution if and only if the
equation has at least one free variable.
Example.
x1 + 2x2 − x3 = 0
x1 + 3x3 + x3 = 0
MATH10212 • Linear Algebra B • Lecture 6 • Solution sets of linear equations • Last change: 23 Apr 2018 13

Lecture 6: Solution sets of linear equations [Lay 1.5]

Nonhomogeneous systems is consistent for some given b, and p
be a solution. Then the solution set of
When a nonhomogeneous system has
Ax = b is the set of all vectors of the
many solutions, the general solution can
form
be written in parametric vector form a
one vector plus an arbitrary linea com- w = p + vh ,
bination of vectors that satisfy the cor-
responding homogeneous system.
where vh is any solution of the homoge-
Example. neous equation
x1 + 2x2 − x3 = 0
x1 + 3x3 + x3 = 5
Ax = 0.
Theorem 1.5.6: Suppose the equation
Ax = b
MATH10212 • Linear Algebra B • Lecture 7 • Linear independence • Last change: 23 Apr 2018 14

Lecture 7: Linear independence [Lay 1.7]

Definition. An indexed set of vectors Therefore the columns of matrix A are
linearly independent if and only if the
{ v1 , . . . , vp } equation
Ax = 0
in Rn is linearly independent if the
vector equation has only the trivial solution.

x1 v1 + · · · + xp vp = 0 A set of one vectors { v1 } is linearly de-

pendent if v1 = 0.
has only trivial solution.
A set of two vectors { v1 , v2 } is linearly
The set dependent if at least one of the vectors
{ v1 , . . . , vp } is a multiple of the other.
in Rn is linearly dependent if there ex- Theorem 1.7.7: Characterisation of
ist weights c1 , . . . , cp , not all zero, such linearly dependent sets. An indexed
that set
c1 v1 + · · · + cp vp = 0 S = { v1 , . . . , vp }
of two or more vectors is linearly depen-
dent if and only if at least one of the
Linear independence of matrix
vectors in S is a linear combination of
columns. The matrix equation
the others.
Ax = 0 Theorem 1.7.8: dependence of
“big” sets. If a set contains more vec-
where A is made of columns
tors than entries in each vector, then the

A = a1 · · · an
 set is linearly dependent. Thus, any set

can be written as { v1 , . . . , vp }

x1 a1 + · · · + xn an = 0 in Rn is linearly dependent if p > n.

MATH10212 • Linear Algebra B • Lecture 8 • Linear transformations • Last change: 23 Apr 2018 15

Lecture 8: Introduction to linear transformations [1.8]

Transformation. A transformation The identity matrix. An n × n ma-
T from Rn to Rm is a rule that assigns trix with 1’s on the diagonal and 0’s else-
to each vector x in Rn a vector T (x) in where is called the identity matrix In .
Rm . For example,
The set Rn is called the domain of T ,
 
  1 0
the set Rm is the codomain of T . I1 = 1 , I2 = ,
0 1
Matrix transformations. With every  
  1 0 0 0
m × n matrix A we can associated the 1 0 0 0 1 0 0
transformation I3 = 0 1 0 , I4 = 

0 0 1 0 .

0 0 1
T : Rn −→ Rm 0 0 0 1
x 7→ Ax. The columns of the identity matrix In
will be denoted
In short: e1 , e2 , . . . , en .
T (x) = Ax.
For example, in R3
The range of a matrix transforma-      
tion. The range of T is the set of all 1 0 0
linear combinations of the columns of A. e1 = 0 , e2 = 1 , e3 = 0 .
0 0 1
Indeed, this can be immediately seen
from the fact that each image T (x) has Obsetrve that if x is an arbitrary vector
the form in Rn ,  
x1
T (x) = Ax = x1 a1 + · · · + xn an  .. 
x =  . ,
xn
Definition: Linear transformations.
then
A transformation 
x1
T : Rn −→ Rm  x2 
 
x =  x3 
 
is linear if:  .. 
.
xn
• T (u + v) = T (u) + T (v) for all
vectors u, v ∈ Rn ;      
1 0 0
• T (cu) = cT (u) for all vectors u 0 1 0
     
= x1   + x2   + · · · + xn 0
and all scalars c. 0 0  
 ..   ..   .. 
. . .
Properties of linear transforma- 0 0 1
tions. If T is a linear transformation
then = x1 e1 + x2 e2 + · · · + xn en .
T (0) = 0
The identity transformation. It is
and easy to check that

T (cu + dv) = cT (u) + dT (v). In x = x for all x ∈ Rn .

MATH10212 • Linear Algebra B • Lecture 8 • Linear transformations • Last change: 23 Apr 2018 16

Therefore the linear transformation as- identity transformation of Rn :

sociated with the identity matrix is the
Rn −→ Rn
x 7→ x

The matrix of a linear transformation [Lay 1.9]

Theorem 1.9.10: The matrix of a The matrix A is called the standard
linear transformation. Let matrix for the linear transforma-
tion T .
T : Rn −→ Rm
Definition. A transformation
be a linear transformation.
T : Rn −→ Rm
Then there exists a unique matrix A such
that is onto Rm if each b ∈ Rm is the image
of at least one x ∈ Rn .
T (x) = Ax for all x ∈ Rn .
A transformation T is one-to-one if
In fact, A is the m × n matrix whose jth each b ∈ Rm is the image of at most
column is the vector T (ej ) where ej is one x ∈ Rn .
the jth column of the identity matrix in
Theorem: One-to-one transforma-
Rn :
tions: a criterion. A linear transfor-
mation
 
A = T (e1 ) · · · T (en )
T : Rn −→ Rm
is one-to-one if and only if the equation
Proof. First we express x in terms of
e1 , . . . , en : T (x) = 0

x = x1 e1 + x2 e2 + · · · + xn en has only the trivial solution.

and compute, using definition of linear Theorem: “One-to-one” and
transformation “onto” in terms of matrices. Let

T (x) = T (x1 e1 + x2 e2 + · · · + xn en ) T : Rn −→ Rm
= T (x1 e1 ) + · · · + T (xn en ) be linear transformation and let A be the
= x1 T (e1 ) + · · · + xn T (en ) standard matrix for T . Then:

and then switch to matrix

• T maps Rn onto Rm if and only if
notation: the columns of A span Rm .
 
x
   .1  • T is one-to-one if and only if the
= T (e1 ) · · · T (en )  .. 
columns of A are linearly indepen-
xn dent.
= Ax.
MATH10212 • Linear Algebra B • Lecture 9 • Matrix operations: Addition • Last change: 23 Apr 2018 17

Lecture 9: Matrix operations: Addition [Lay 2.1]

Labeling of matrix entries. Let A be Matrices
an m × n matrix.    
  1 0 0 π √0 0

A = a1 a2 · · · an
 1 0 0 0 0 ,
, 0 2 0
0 2
0 0 2 0 0 3
a11 · · · ···
 
a1j a1n
 .. .. .. 
are all diagonal. The identity matrices
 . . . 
=  ai1 · · · aij ··· ain 
 
 .  
 .. .. ..    1 0 0
. .    1 0
1 , , 0 1 0 , . . .
am1 · · · amj · · · amn 0 1
0 0 1
Notice that in aij the first subscript i
denotes the row number, the second sub- are diagonal.
script j the column number of the entry Zero matrix. By definition, 0 is a m×n
aij . In particular, the column aj is matrix whose entries are all zero. For ex-

a1j
 ample, matrices
 .. 
 .   

0 0 0

aj =  aij  .
  0 0 ,
 .  0 0 0
 .. 
amj are zero matrices. Notice that zero
square matrices, like
Diagonal matrices, zero matrices.  
The diagonal entries in A are   0 0 0
0 0 0 0 0
,
a11 , a22 , a33 , . . . 0 0
0 0 0
For example, the diagonal entries of the
matrix are diagonal!
 
1 2 3 Sums. If
A = 4 5 6
7 8 9 
A = a1 a2 · · · an

are 1, 5, and 9.
A square matrix is a matrix with equal and
 
numbers of rows and columns. B = b1 b2 · · · bn

A diagonal matrix is a square matrix are m × n matrices then we define the

whose non-diagonal entries are zeroes. sum A + B as

 
A+B = a1 + b1 a2 + b2 ··· an + bn
a11 + b11 · · · ···
 
a1j + b1j a1n + b1n
.. .. ..
. . .
 
 
=  ai1 + bi1 · · · aij + bij ··· ain + bin 
 
 .. .. .. 
 . . . 
am1 + bm1 · · · amj + bmj · · · amn + bmn
MATH10212 • Linear Algebra B • Lecture 9 • Matrix operations: Addition • Last change: 23 Apr 2018 18

Scalar multiple. If c is a a scalar then trices of the same size and r and s be
we define scalars.
 
cA = ca1 ca2 · · · can
1. A + B = B + A
ca11 · · · ···
 
ca1j ca1n 2. (A + B) + C = A + (B + C)
 .. .. .. 
 . . . 
3. A + 0 = A
=  cai1 · · · caij ··· cain 
 
 . .. .. 
 .. . .  4. r(A + B) = rA + rB
cam1 · · · camj · · · camn
5. (r + s)A = rA + sA
Theorem 2.1.1: Properties of ma- 6. r(sA) = (rs)A.
trix addition. Let A, B, and C be ma-
MATH10212 • Linear Algebra B • Lecture 10 • Matrix multiplication • Last change: 23 Apr 2018 19

Lecture 10: Matrix multiplication [Lay 2.1]

Composition of linear transforma- S ◦ T and it will be natural to call C the
tions. Let B be an m × n matrix and product of A and B and denote
A an p × m matrix. They define linear
C =A·B
transformations
(but the multiplication symbol “·” is fre-
T : Rn −→ Rm , x 7→ Bx quently skipped).

and Definition: Matrix multiplication.

If A is an p × m matrix and B is an
S : Rm −→ Rp , y 7→ Ay. m × n matrix with columns

Their composition b1 , . . . , bn
then the product AB is the p×n matrix
(S ◦ T )(x) = S(T (x)) whose columns are
is a linear transformation Ab1 , . . . , Abn :

S ◦ T : Rn −→ Rp .
 
AB = A b1 · · · bn
 
From the previous lecture, we know that = Ab1 · · · Abn .
linear transformations are given by ma-
trices. What is the matrix of S ◦ T ?
Columns of AB. Each column Abj of
Multiplication of matrices. To an- AB is a linear combination of columns of
swer the above question, we need to com- A with weights taken from the jth col-
pute A(Bx) in matrix form. umn of B:
Write x as
 
b
  1j
Abj = a1 · · · am  ... 
   
x1
x =  ...  bmj
 
xn = b1j a1 + · · · + bmj am
and observe Mnemonic rules
Bx = x1 b1 + · · · + xn bn .
[m × n matrix] · [n × p matrix] =
Hence [m × p matrix]

A(Bx) = A(x1 b1 + · · · + xn bn )
= A(x1 b1 ) + · · · + A(xn bn ) columnj (AB) = A · columnj (B)

= x1 A(b1 ) + · · · + xn A(bn )
  rowi (AB) = rowi (A) · B
= Ab1 Ab2 · · · Abn x
Theorem 2.1.2: Properties of ma-
Therefore multiplication by the matrix
trix multiplication. Let A be an m×n
matrix and let B and C be matrices for
 
C = Ab1 Ab2 · · · Abn
which indicated sums and products are
transforms x into A(Bx). Hence C is defined. Then the following identities
the matrix of the linear transformation are true:
MATH10212 • Linear Algebra B • Lecture 10 • Matrix multiplication • Last change: 23 Apr 2018 20

1. A(BC) = (AB)C

2. A(B + C) = AB + AC

3. (B + C)A = BA + CA

4. r(AB) = (rA)B = A(rB) for any

scalar r

5. Im A = A = AIn

Powers of matrix. As it is usual in

algebra, we define, for a square matrix
A,

Ak = A · · · A (k times)

If A 6= 0 then we set

A0 = I

The transpose of a matrix. The

transpose AT of an m × n matrix A is
the n × m matrix whose rows are formed
from corresponding columns of A:
 
 T 1 4
1 2 3
= 2 5
4 5 6
3 6

Theorem 2.1.3: Properties of trans-

pose. Let A and B denote matrices
whose sizes are appropriate for the fol-
lowing sums and products. Then we
have:

1. (AT )T = A

2. (A + B)T = AT + B T

3. (rA)T = r(AT ) for any scalar r

4. (AB)T = B T AT
MATH10212 • Linear Algebra B • Lecture 11 • The inverse of a matrix • Last change: 23 Apr 2018 21

Lecture 11: The inverse The quantity ad − bc is called the de-

terminant of A:
of a matrix [Lay 2.2]
det A = ad − bc
Invertible matrices
An n × n matrix A is invertible if there Theorem 2.2.5: Solving matrix
is an n × n matrix C such that equations. If A is an invertible n × n
matrix, then for each b ∈ Rn , the equa-
CA = I and AC = I tion Ax = b has the unique solution

C is called the inverse of A. x = A−1 b.

The inverse of A, if exists, is unique (!)
and is denoted A−1 :
Theorem 2.2.6: Properties of in-
−1
A A = I and AA −1
= I. vertible matrices.

(a) If A is an invertible matrix, then

Singular matrices. A non-invertible
A−1 is also invertible and
matrix is called a singular matrix.
An invertible matrix is nonsingular. (A−1 )−1 = A

Theorem 2.2.4: Inverse of a 2 × 2 (b) If A and B are n×n invertible ma-

matrix. Let trices, then so is AB, and

(AB)−1 = B −1 A−1
 
a b
A=
c d
(c) If A is an invertible matrix, then
If ad − bc 6= 0 then A is invertible and so is AT , and
−1
(AT )−1 = (A−1 )T
  
a b 1 d −b
=
c d ad − bc −c a
MATH10212 • Linear Algebra B • Lecture 12 • Characterizations of invertible matrices • Last change: 23 Apr 2018 22

Lecture 12: Characterizations of invertible matrices.

Definition: Elementary matrices. (f) The linear transformation x 7→ Ax
An elementary matrix is a matrix ob- is one-to-one.
tained by performing a single elementary
row operation on an identity matrix. (g) Ax = b has at least one solution
for each b ∈ Rn .
If an elementary row transformation is
performed on an n × n matrix A, the (h) The columns of A span Rn .
resulting matrix can be written as EA,
where the m × m matrix is made by the (i) x 7→ Ax maps Rn onto Rn .
same row operations on Im .
(j) There is an n × n matrix C such
Each elementary matrix E is invertible.
that CA = I.
The inverse of E is the elementary ma-
trix of the same type that transforms E
(k) There is an n × n matrix D such
back into I.
that AD = I.

Theorem 2.2.7: Invertible matrices. (l) AT is invertible.

An n × n matrix A is invertible if and
only if A is row equivalent to In , and One-sided inverse is the inverse. Let
in this case, any sequence of elementary A and B be square matrices.
row operations that reduces A to In also
transforms In into A−1 . If AB = I then both A and B are in-
vertible and
Computation of inverses.
B = A−1 and A = B −1 .
• Form
  the augmented matrix
A I and row reduce it.
Theorem 2.3.9: Invertible linear
• If A  is row equivalent to I, transformations. Let

then A I is row equivalent to
−1 T : Rn −→ Rn
 
I A .
• Otherwise A has no inverse. be a linear transformation and A its
standard matrix.
The Invertible Matrix Theorem
2.3.8: For an n × n matrix A, the fol- Then T is invertible if and only if A is
lowing are equivalent: an invertible matrix.

In that case, the linear transformation

(a) A is invertible.
(b) A is row equivalent to In . S(x) = A−1 x

(c) A has n pivot positions. is the only transformation satisfying

(d) Ax = 0 has only the trivial solu-
S(T (x)) = x for all x ∈ Rn
tion.
T (S(x)) = x for all x ∈ Rn
(e) The columns of A are linearly in-
dependent.
MATH10212 • Linear Algebra B • Lecture 13 • Partitioned matrices • Last change: 23 Apr 2018 23

Lecture 13 Partitioned matrices [Lay 2.4]

Example: Partitioned matrix AB can be computed by the usual row-
  column rule, with blocks treated as ma-
1 2 a trix entries.
A = 3 4 b     
p q z 1 2 a α β
  A = 3 4 b  , B =  γ δ 
A11 A12 p q z Γ ∆
=
A21 A22
Example
A is a 3 × 3 matrix which can be viewed  
as a 2 × 2 partitioned (or block) ma- 1 2 a  
A11 A12
trix with blocks A= 3 4 b =
  ,
A21 A22
    p q z
1 2 a
A11 = , A12 = , 
α β

3 4 b  
B1
B= γ δ =
 
B2
Γ ∆
   
A21 = p q , A22 = z

Addition of partitioned matrices. If   

matrices A and B are of the same size A11 A12 B1
AB =
and partitioned the same way, they can A21 A22 B2
 
be added block-by-block. A11 B1 + A12 B2
=
A21 B1 + A22 B2
Similarly, partitioned matrices can be
multiplied by a scalar blockwise. 
1 2 α β
  
a  
 3 4 γ + Γ ∆ 
Multiplication of partitioned ma- δ b
=    
trices. If the column partition of A   α β   
matches the row partition of B then p q + z Γ ∆
γ δ

Theorem 2.4.10: Column-Row Expansion of AB. If A is m × n and B is n × p

then
 
row1 (B)
 row2 (B) 

 
AB = col1 (A) col2 (A) · · · coln (A)  .. 
 . 
rown (B)

= col1 (A)row1 (B) + · · · + coln (A)rown (B)

Example Partitioned matrices natu- sysetm of equations:

rally arise in analysis of systems of si-
multaneous linear equations. Consider a
x1 + 2x2 + 3x3 + 4x4 = 1
x1 + 3x2 + 3x3 + 4x4 = 2
MATH10212 • Linear Algebra B • Lecture 13 • Partitioned matrices • Last change: 23 Apr 2018 24

In matrix form, it looks as Observe that

 
 x1
 
   −1 3 −2
1 2 3 4   = 1
x 2
 A =
−1 1
1 3 3 4 x3  2
x4
and
Denote   

1 2

−1 3 −2 3 4
A = A C = ·
1 3 −1 1 3 4
 

3 4
 3 4
C = =
3 4 0 0
 
x1
Y1 = which gives us an answer:
x2
 
x3  
x1
   
3 −2 1
 
3 4 t
Y2 = = −
x4 x2 −1 1 2 0 0 s
 
1   
−1 3t + 4s

B = = −
2 1 0
then we can rewrite the matrices in-
   
x3 t
volved in the matrix equation ar parti- =
x4 s
tioned matrices:
 
  Y1 We may wish to bring it in a more tra-
A C =B
Y2 ditional form:
and, multiplying matrices as partitioned  
x1
  
−1 −3t − 4s

matrices, x2 
  =  1 + 0
   

AY1 + CY2 = B. x3  0  t 
x4 0 s
Please notice that matrix A is invertible      
−1 −3 −4
– at this stage it should be an easy men- 1 0 0
tal calculation for you. Multiplying the = 
 0  + t 1  + s 0 
    
both parts of the last equation by A−1 ,
0 0 1
we get
Y1 + A−1 CY2 = A−1 B Of course, when we solving a singles sys-
tem of equations, the formulae of the
and kind
Y1 = A−1 B − A−1 CY2
Y1 = A−1 B − A−1 CY2
The entries of
 
x do not save time and effort, but in many
Y2 = 3
x4 applications you have to solve millions
of systems of linear equations with the
can be taken for free parameters: x3 = t, same matrix of coefficients but different
x4 = s, and x1 and x2 expressed as right-hand parts, and in this situation
   
x1 −1 −1 t shortcuts based on compound matrices
=A B−A C become very useful.
x2 s
MATH10212 • Linear Algebra B • Lecture 14 • Subspaces • Last change: 23 Apr 2018 25

Lecture 14: Subspaces of Rn [Lay 2.8]

Subspace. A subspace of Rn is any set The null space of a matrix A is the
H that has the properties set Nul A of all solutions to the homo-
geneous system
(a) 0 ∈ H.
Ax = 0
(b) For each vectors u, v ∈ H, the sum
u + v ∈ H.
Theorem 2.8.12: The null space.
(c) For each vector u ∈ H and each The null space of an m × n matrix A is
scalar c, cu ∈ H. a subspace of Rn .
Equivalently, the set of all solutions to a
Rn is a subspace of itself.
system
{ 0 } is a subspace, called the zero sub- Ax = 0
space.
of m homogeneous linear equations in n
unknowns is a subspace of Rn .
Span. Span{ v1 , . . . , vp } is the sub-
space spanned (or generated) by Basis of a subspace. A basis of a sub-
v1 , . . . , vp . space H of Rn is a linearly independent
set in H that spans H.
The column space of a matrix A is the
set Col A of all linear combinations of Theorem 2.8.13: Pivot columns of a
columns of A. matrix A form a basis of Col A.

Dimension and rank [Lay 2.9]

Theorem: Given a basis b1 , . . . , bp in a such that
subspace H of Rn , every vector u ∈ H is
uniquely expressed as a linear combina- x = c1 b1 + · · · + cp bp .
tion of b1 , . . . , bp .
Solving homogeneous systems. The vector
Finding a parametric solution of a sys-  
c1
tem of homogeneous linear equations    .. 
x B=.
Ax = 0 cp

means to find a basis of Nul A. is the coordinate vector of x.

Coordinate system. Let Dimension. The dimension of a
nonzero subspace H, denoted by dim H,
B = { b1 , . . . , bp } is the number of vectors in any basis of
H.
be a basis for a subspace H.
The dimension of the zero subspace {0}
For each x ∈ H, the coordinates of x
is defined to be 0.
relative to the basis B are the weights
Rank of a matrix. The rank of a ma-
c1 , . . . , c p trix A, denoted by rank A, is the dimen-
MATH10212 • Linear Algebra B • Lecture 14 • Subspaces • Last change: 23 Apr 2018 26

sion of the column space Col A of A:
rank A = dim Col A
The Rank Theorem 2.9.14: If a ma-
trix A has n columns,
rank A + dim Nul A = n.
The Basis Theorem 2.9.15: Let H be
a p-dimensional subspace of Rn .
• Any linearly independent set of ex-
actly p elements in H is automati-
cally a basis for H.
form a basis of R3 : there are three of
them and they are linearly independent
(the latter should be obvious to students
at this stage).
The Invertible Matrix Theorem
(continued) Let A be an n × n matrix.
Then the following statements are each
equivalent to the statement that A is an
invertible matrix.
(m) The columns of A form a basis of
Rn .
(n) Col A = Rn .

• Also, any set of p elements of H (o) dim Col A = n.

that spans H is automatically a ba-
sis of H. (p) rank A = n.

For example, this means that the vectors (q) Nul A = {0}.
     
1 2 4
0 , 3 , 5 (r) dim Nul A = 0.
0 0 6
MATH10212 • Linear Algebra B • Lecture 15 • Introduction to determinants • Last change: 23 Apr 2018 27

Lecture 15: Introduction to determinants [Lay 3.1]

The determinant det A of a square ma- The determinant of a 2 × 2 matrix
trix A is a certain number assigned to the is defined by the formula
matrix; it is defined recursively, that
is, we define first determinants of matri-
ces of sizes 1 × 1, 2 × 2, and 3 × 3, and 
a11 a12

then supply a formula which expresses det = a11 a22 − a12 a21 .
a21 a22
determinants of n × n matrices in terms
of determinants of (n − 1) × (n − 1) ma-
trices. The determinant of a 3 × 3 matrix
 
a11 a12 a13
The determinant of a 1 × 1 matrix
  A = a21 a22 a23 
A = a11 is defined simply as being
a31 a32 a33
equal its only entry a11 :
 
det a11 = a11 . is defined by the formula

 
a11 a12 a13      
a22 a23 a21 a23 a21 a22
det a21 a22 a23 = a11 ·
  − a12 · + a13 ·
a32 a33 a31 a33 a31 a32
a31 a32 a33

Example. The determinant det A of the For example, if

matrix  
2 0 7 
1 2 3

A = 0 1 0 A = 4 5 6 ,
1 0 4 7 8 9
equals

1 0
 
0 1
 then
2 · det + 7 · det
0 4 1 0    
1 3 2 3
which further simplifies as A22 = , A31 =
7 9 5 6
2 · 4 + 7 · (−1) = 8 − 7 = 1.
Recursive definition of determi-
Submatrices. By definition, the sub- nant. For n > 3, the determinant of
matrix Aij is obtained from the matrix an n × n matrix A is defined as the ex-
A by crossing out row i and column j. pression

det A = a11 det A11 − a12 det A12 + · · · + (−1)1+n a1n det A1n
n
X
= (−1)1+j a1j det A1j
j=1
MATH10212 • Linear Algebra B • Lecture 15 • Introduction to determinants • Last change: 23 Apr 2018 28

which involves the determinants of is based on induction on n, which, to

smaller (n−1)×(n−1) submatrices A1j , avoid the use of “general” notation is il-
which, in their turn, can be evaluated by lustrated by a simple example. Let
a similar formula wich reduces the cal-  
cualtions to the (n − 2) × (n − 2) case, a11 0 0 0 0
a21 a22 0 0 0
and can be repeated all the way down to  
determinants of size 2 × 2. A = a31 a32 a33 0
 0 .
a41 a42 a43 a44 0 
Cofactors. The (i, j)-cofactor of A is a51 a52 a53 a54 a55

Cij = (−1)i+j det Aij All entries in the first row of A–with pos-
sible exception of a11 —are zeroes,
Then
a12 = · · · = a15 = 0,
det A = a11 C11 + a12 C12 + · · · + a1n C1n
therefore in the formula
is cofactor expansion across the first det A = a11 det A11 + · · · + a55 det A55
row of A.
all summand with possible exception of
Theorem 3.1.1: Cofactor expan-
the first one,
sion. For any row i, the result of the
cofactor expansion across the row i is the a11 det A11 ,
same:
are zeroes, and therefore
det A = ai1 Ci1 + ai2 Ci2 + · · · + ain Cin
det A = a11 det A11
 
a22 0 0 0
The chess board pattern of signs in a32 a33 0 0
cofactor expansions. The signs = a11 det 
a42 a43 a44 0  .


a52 a53 a54 a55

(−1)i+j
But the smaller matrix A11 is also lower
which appear in the formula for co- triangle, and therefore we can conclude
factors, form the easy-to-recognise and by induction that
easy-to-remember “chess board” pat-  
tern: a22 0 0 0
a32 a33 0 0
a42 a43 a44 0  = a22 · · · a55
det  
 
+ − + ···
− + −  a52 a53 a54 a55
 
+ − +  hence
 
.. ...
. det A = a11 · (a22 · · · a55 )
= a11 a22 · · · a55
Theorem 3.1.2: The determinant
is the product of diagonal entries of A.
of a triangular matrix. If A is a tri-
angular n × n matrix then det A is the The basis of induction is the case n = 2;
product of the diagonal entries of A. of course, in that case
 
a11 0
det = a11 a22 − 0 · a21
Proof: This proof contains more details a21 a22
than the one given in the textbook. It = a11 a22
MATH10212 • Linear Algebra B • Lecture 15 • Introduction to determinants • Last change: 23 Apr 2018 29

is the product of the diagonal entries of Corollary. The determinant of the

A. identity matrix equals 1:

det In = 1.
Corollary. The determinant of a diag-
onal matrix equals is the product of its
diagonal elements:
  Corollary. The determinant of the zero
d1 0 · · · 0 matrix equals 0:
 0 d2 0
det  .. ..  = d1 d2 · · · dn .
 
. . det 0 = 0.
. . .
0 · · · dn
MATH10212 • Linear Algebra B • Lectures 15–16 • Properties of determinants • Last change: 23 Apr 2018 30

Lectures 15–16: Properties of determinants [Lay 3.2]

Theorem 3.2.3: Row Operations. Similarly
Let A be a square matrix.  
T 1 3
det A = 1 · det +0+0
(a) If a multiple of one row of A is 0 4
added to another row to produce
= 1 · 4 = 4;
a matrix B, then

det B = det A. we got the same value.

(b) It two rows of A are swapped to Corollary: Cofactor expansion

produce B, then across a column. For any column j,

det B = − det A. det A = a1j C1j + a2j C2j + · · · + anj Cnj

(c) If one row of A is multiplied by k

to produce B, then Example. We have already computed
det B = k det A. the determinant of the matrix
 
2 0 7
Theorem 3.2.5: The determinant of A = det 0 1 0
the transposed matrix. If A is an 1 0 4
n × n matrix then
by expansion across the first row. Now
det AT = det A. we do it by expansion across the third
column:
 
Example. Let 1+3 0 1
det A = 7 · (−1) · det
  1 0
1 2 0  
2+3 2 0
A = 0 1 0 , +0 · (−1) · det
1 0
0 3 4
 
3+3 2 0
then +4 · (−1) · det

1 0 0
 0 1
AT = 2 1 3 = −7 − 0 + 8
0 0 4 = 1.
and
 Proof: Columns of A are rows of AT
1 0 which has the same determinant as A.
det A = 1 · det
3 4
  Corollary: Column operations. Let
0 0 A be a square matrix.
−2 · det
0 4
 
0 1 (a) If a multiple of one column of A
+0 · det
0 3 is added to another column to pro-
duce a matrix B, then
= 1·4−2·0+0·0
= 4. det B = det A.
MATH10212 • Linear Algebra B • Lectures 15–16 • Properties of determinants • Last change: 23 Apr 2018 31

(b) It two columns of A are swapped • If a row or a column has a conve-

to produce B, then nient scalar factor, take it out of
the determinant.
det B = − det A.
• If convenient, swap two rows or
(c) If one column of A is multiplied by two columns— but do not forget
k to produce B, then to change the sign of the determi-
nant.
det B = k det A.
• Adding to a row (column) scalar
multiples of other rows (columns),
Proof: Column operations on A are row simplify the matrix to create a row
operation of AT which has the same de- (column) with just one nonzero en-
terminant as A. try.
Computing determinants. In com- • Expand across this row (column).
putations by hand, the quickest method
of computing determinants is to work • Repeat until you get the value of
with both columns and rows: the determinant.

Example.
   
1 0 3 1 0 0
C3 −3C1
det 1 3 1  = det 1 3 −2
1 1 −1 1 1 −4
 
1 0 0
2 out of C3
= 2 · det 1 3 −1
1 1 −2
 
1 0 0
−1 out of C3
= −2 · det 1 3 1
1 1 2
 
expand 1+1 3 1
= −2 · 1 · (−1) · det
1 2
 
R1 −3R2 0 −5
= −2 · det
1 2
 
R1 ↔R2 1 2
= −2 · (−1) · det
0 −5
= 2 · (−5)
= −10.

Example. Compute the determinant Solution: Subtracting Row 1 from


1 1 1 1 0
 Rows 2, 3, and 4, we rearrange the de-
1 1 1 0 1
 
∆ = det 
1 1 0 1 1 

 
MATH10212 • Linear Algebra B • Lectures 15–16 • Properties of determinants • Last change: 23 Apr 2018 32

terminant as the first column we have

 
2+1 −1 1
∆ = −(−1) · (−1) · det
  1 3
1 1 1 1 0
0 0 0 −1 1  
  −1 1
∆ = det 0 0 −1 0 1
 = − det
0 −1 0
 1 3
0 1
0 1 1 1 1
 
R3 +R2 −1 1
= − det
0 4

After expanding the determinant across = −(−4)

the first column, we get = 4.

  As an exercise, I leave you with this ques-

0 0 −1 1 tion: wouldn’t you agree that the deter-
 0 −1 0 1 minant of the similarly constructed ma-
∆ = 1 · (−1)1+1 · det 
−1 0

0 1 trix of size n × n should be ±(n − 1)?
1 1 1 1 But how can one determine the correct
  sign?
0 0 −1 1
 0 −1 0 1 And one more exercise: you should now
= det −1 0
 be able to instantly compute
0 1
1 1 1 1
 
1 2 3 4 0
  1 2 3 0 5
0 0 −1 1
 
det 1 2 0 4 5 .
0 −1 0 1
 
R4 +R3   1 0 3 4 5
= det 
−1 0 0 1 0 2 3 4 5
0 1 1 2
Theorem 3.2.4: Invertible matri-
ces. A square matrix A is invertible if
and only if
After expansion across the 1st column
det A 6= 0.

 
0 −1 1
∆ = (−1) · (−1)3+1 · det −1 0 1 Recall that this theorem has been al-
1 1 2 ready known to us in the case of 2 × 2
matrices A.
 
0 −1 1 Example. The matrix
= − det −1 0 1  
1 1 2 2 0 7
A = 0 1 0

0 −1 1
 1 0 4
R3 +R2
= − det −1 0 1 has the determinant
0 1 3    
1 0 0 1
det A = 2 · det + 7 · det
0 4 1 0
= 2 · 4 + 7 · (−1)
After expanding the determinant across = 8−7=1
MATH10212 • Linear Algebra B • Lectures 15–16 • Properties of determinants • Last change: 23 Apr 2018 33

hence A is invertible. Indeed, and

 
4 0 −7
A−1 =  0 1 0 det(AB) = 1 · 17 − 5 · 3 = 2,
−1 0 2
which is exactly the value of
—check!

(det A) · (det B).

Theorem 3.2.6: Multiplicativity
Property. If A and B are n × n ma-
trices then Corollary. If A is invertible,
det AB = (det A) · (det B)
det A−1 = (det A)−1 .

Example. Take
Proof. Set B = A−1 , then
   
1 0 1 5
A= and B = ,
3 2 0 1
then AB = I

det A = 2 and det B = 1.

and Theorem 3.2.6 yields
But
  
1 0 1 5 det A · det B = det I = 1.
AB =
3 2 0 1
  Hence
1 5
= det B = (det A)−1 .
3 17

Cramer’s Rule [Lay 3.3]

Cramer’s Rule is an explicit (closed) for- is given by
mula for solving systems of n linear equa- det Ai (b)
tions with n unknowns and nonsigular xi = , i = 1, 2, . . . , n.
det A
(invertible) matrix of coefficients. It has
important theoretical value, but is un-
suitable for practical application. For example, for a system
x1 + x2 = 3
n
For any n × n matrix A and any b ∈ R , x1 − x2 = 1
denote by Ai (b) the matrix obtained
Cramer’s rule gives the answer
from A by replacing column i by the vec-  
tor b: 3 1
det

Ai (b) = a1 · · · ai−1 b ai+1 · · · an .
 1 −1 −4
x1 =  = =2
1 1 −2
det
Theorem 3.3.7: Cramer’s Rule. Let 1 −1
A be an invertible n × n matrix. For 
1 3

any b ∈ Rn , the unique solution x of the det
1 1 −2
linear system x2 =  = =1
1 1 −2
Ax = b det
1 −1
MATH10212 • Linear Algebra B • Lectures 15–16 • Properties of determinants • Last change: 23 Apr 2018 34

Theorem 3.3.8: An Inverse For- the cofactors are

mula. Let A be an invertible n × n ma-
trix. Then
  C11 = (−1)1+1 · d = d
C11 C21 · · · Cn1
C21 = (−1)2+1 · b = −b
1   C12 C22 · · · Cn2 

A−1 =  .. .. ..  C12 = (−1)1+2 · c = −c
det A  . . . 
C1n C2n · · · Cnn C22 = (−1)2+2 · a = a,

where cofactors Cji are given by

and we get the familiar formula
Cji = (−1)j+i det Aji
and Aji is the matrix obtained from A  
by deleting row j and column i. −1 1 d −b
A =
det A −c a
Notice that for a 2 × 2 matrix
   
a b 1 d −b
A= = .
c d ad − bc −c a
MATH10212 • Linear Algebra B • Lecture 17 • Eigenvalues and eigenvectors • Last change: 23 Apr 2018 35

Lecture 17: Eigenvalues and eigenvectors

Quiz x is called an eigenvector correspond-
ing to λ.
What is the value of this determinant?
Eigenvalue. A scalar λ is an eigenvalue

1 2 3
 of A iff the equation
∆ = det 0 2 3
1 2 6 (A − λI)x = 0

Quiz has a non-trivial solution.

Is Γ positive, negative, or zero? Eigenspace. The set of all solutions of
 
0 0 71 (A − λI)x = 0
Γ = det  0 93 0
87 0 0 is a subspace of Rn called the
eigenspace of A corresponding to the
Eigenvectors. An eigenvector of an
eigenvalue λ.
n × n matrix A is a nonzero vector x
such that Theorem: Linear independence of
Ax = λx eigenvectors. If v1 , . . . , vp are eigen-
for some scalar λ. vectors that correspond to distinct eigen-
values of λ1 , . . . , λp of an n×n matrix A,
then the set
Eigenvalues. A scalar λ is called an
eigenvalue of A if there is a non-trivial { v1 , . . . , vp }
solution x of
Ax = λx; is linearly independent.
MATH10212 • Linear Algebra B • Lecture 17 • The characteristic equation • Last change: 23 Apr 2018 36

Lecture 18: The characteristic equation

Characteristic polynomial (s) The number 0 is not an eigenvalue
of A.
The polynomial
(t) The determinant of A is not zero.
det(A − λI)
Theorem: Eigenvalues of a triangu-
in variable λ is characteristic polyno-
lar matrix. The eigenvalues of a trian-
mial of A.
gular matrix are the entries on its main
For example, in the 2 × 2 case diagonal.

a−λ b
 Proof. This immediately follows from
det = λ2 −(a+d)λ+(ad−bc). the fact the the determinant of a trian-
c d−λ
gular matrices is the product of its diag-
onal elements. Therefore, for example,
det(A − λI) = 0 for the 3 × 3 triangular matrix
 
is the characteristic equation for A. d1 a b
A =  0 d2 c 
Characterisation of eigenvalues
0 0 d3
Theorem. A scalar λ is an eigenvalue
of an n × n matrix A if and only if λ its characteristic polynomial det(A−λI)
satisfies the characteristic equation equals
 
det(A − λI) = 0 d1 − λ a b
det  0 d2 − λ c 
0 0 d3 − λ

Zero as an eigenvalue. λ = 0 is an and therefore

eigenvalue of A if and only if det A = 0.
det(A − λI) = (d1 − λ)(d2 − λ)(d3 − λ)

The Invertible Matrix Theorem has roots (eigenvalues of A)

(continued). An n × n matrix A is in-
vertible iff : λ = d1 , d 2 , d 3 .
MATH10212 • Linear Algebra B • Lectures 18–21 • Diagonalisation 37

Lectures 19–21: Diagonalisation • Last change: 23

Apr 2018
Similar matrices. A is similar to B if Theorem: Similar matrices have
there exist an invertible matrix P such the same characteristic polynomial.
that If matrices A and B are similar then they
A = P BP −1 . have the same characteristic polynomials
and eigenvalues.
Example:
Diagonalisable matrices. A square
Occasionally the similarity relation is de- matrix A is diagonalisable if it is simi-
noted by symbol ∼. lar to a diagonal matrix, that is,
Similarity is an equivalence relation: it
is A = P DP −1

for some diagonal matrix D and some

• reflexive: A ∼ A
invertible matrix P .
• symmetric: A ∼ B implies B ∼ A Of course, the diagonal entries of D are
• transitive: A ∼ B and B ∼ C im- eigenvalues of A.
plies A ∼ C. The Diagonalisation Theorem. An
n × n matrix A is diagonalisable iff A
Conjugation. Operation has n linearly independent eigenvectors.
In fact,
AP = P −1 AP
A = P DP −1
is called conjugation of A by P .
where D is diagonal iff the columns of
Properties of conjugation P are n linearly independent eigenvec-
tors of A.
• IP = I
Theorem: Matrices with all eigen-
P P P
• (AB) = A B ; as a corollary: values distinct. An n × n matrix with
n distinct eigenvalues is diagonalisable.
• (A + B)P = AP + B P
Example: Non-diagonalisable ma-
P P
• (c · A) = c · A trices  
2 1
• (Ak )P = (AP )k A=
0 2
• (A−1 )P = (AP )−1 has repeated eigenvalues λ1 = λ2 = 2.
• (AP )Q = AP Q A is not diagonalisable. Why?
MATH10212 • Linear Algebra B • Lecture • Vector spaces • Last change: 23 Apr 2018 38

Lecture 22: Vector spaces

A vector space is a non-empty space
of elements (vectors), with operations of
addition and multiplication by a scalar.
By definition, the following holds for all
u, v, w in a vector space V and for all
scalars c and d.
1. The sum of u and v is in V .
Definitions
Most concepts introduced in the previ-
ous lectures for the vector spaces Rn can
be transferred to arbitrary vector spaces.
Let V be a vector space.
Given vectors v1 , v2 , . . . , vp in V and
scalars c1 , c2 , . . . , cp , the vector

2. u + v = v + u. y = c1 v1 + · · · cp vp
3. (u + v) + w = u + (v + w). is called a linear combina-
4. There is zero vector 0 such that tion of v1 , v2 , . . . , vp with weights
u + 0 = u. c1 , c2 , . . . , cp .
If v1 , . . . , vp are in V , then the set of
5. For each u there exists −u such
all linear combination of v1 , . . . , vp is de-
that u + (−u) = 0.
noted by Span{v1 , . . . , vp } and is called
6. The scalar multiple cu is in V . the subset of V spanned (or gener-
ated) by v1 , . . . , vp .
7. c(u + v) = cu + cv.
That is, Span{v1 , . . . , vp } is the collec-
8. (c + d)u = cu + du. tion of all vectors which can be written
in the form
9. c(du) = (cd)u.
c1 v1 + c2 v2 + · · · + cp vp
10. 1u = u.
with c1 , . . . , cp scalars.
Examples. Of course, the most impor-
Span{v1 , . . . , vp } is a vector subspace
tant example of vector spaces is provided
of V , that is, it is closed with respect to
by standard spaces Rn of column vectors
addition of vectors and multiplying vec-
with n components, with the usual op-
tors by scalars.
erations of component-wise addition and
multiplication by scalar. V is a subspace of itself.
Another example is the set Pn of poly- { 0 } is a subspace, called the zero sub-
nomials of degree at most n in variable space.
x,

Pn = {a0 + a1 x + · · · + an xn } An indexed set of vectors

with the usual operations of addition of { v1 , . . . , vp }

polynomials multiplication by constant.
in V is linearly independent if the vec-
And here is another example: the set tor equation
C[0, 1] of real-valued continuous func-
tions on the segment [0, 1], with the x1 v1 + · · · + xp vp = 0
usual operations of addition of functions
and multiplication by constant. has only trivial solution.
MATH10212 • Linear Algebra B • Lecture • Vector spaces • Last change: 23 Apr 2018 39

The set A basis of V is a linearly independent

{ v1 , . . . , vp } set which spans V . A vector space V
is called finite dimensional if it has
in V is linearly dependent if there ex-
a finite basis. Any two bases in a fi-
ist weights c1 , . . . , cp , not all zero, such
nite dimensional vector space have the
that
same number of vectors; this number is
c1 v1 + · · · + cp vp = 0
called the dimension of V and is de-
noted dim V .
Theorem: Characterisation of lin-
early dependent sets. An indexed set If B = (b1 , . . . , bn ) is a basis of V , every
vector x ∈ V can be written, and in a
S = { v1 , . . . , vp } unique way, as a linear combination

of two or more vectors is linearly depen- x = x1 b1 + · · · xn bn ;

dent if and only if at least one of the
vectors in S is a linear combination of the scalars x1 , . . . , xn are called coordi-
the others. nates of x with respect to the basis B.
MATH10212 • Linear Algebra B • Lectures 23 and 24 • Linear transformations • Last change: 23 Apr 2018 40

Lectures 23–24: Linear transformations of abstract

vector spaces
A transformation T from a vector The identity transformation of V is
space V to a vector space W is a rule
that assigns to each vector x in V a vec- V −→ V
tor T (x) in W . x 7→ x.
The set V is called the domain of T , the
set W is the codomain of T . The zero transformation of V is
A transformation
V −→ V
T : V −→ W
x 7→ 0.
is linear if:

• T (u + v) = T (u) + T (v) for all If

vectors u, v ∈ V ; T : V −→ W
• T (cu) = cT (u) for all vectors u is a linear transformation, its kernel
and all scalars c.
Ker T = {x ∈ V : T (x) = 0}
Properties of linear transforma-
tions. If T is a linear transformation
then is a subspace of V , while its image
T (0) = 0
Im T = {y ∈ W : T (x) = y for some x ∈ V }
and
T (cu + dv) = cT (u) + dT (v). is a subspace of W .
MATH10212 • Linear Algebra B • Lecture 25 • Symmetric matrices and inner product • Last change: 23 Apr 2018 41

Lecture 25: Symmetric matrices and inner product

Symmetric matrices. A matrix A is
symmetric if AT = A.
Properties of dot product
Theorem 6.1.1. Let u, v, w ∈ Rn , and
An example:
  c be a scalar. Then
1 2 3
A = 2 3 4 .
3 4 5 (a) u  v = v  u
Notice that symmetric matrices are nec-
essarily square.
(b) (u + v)  w = u  w + v  w
Inner (or dot) product. For vectors
u, v ∈ Rn their inner product (also
called scalar product or dot product) (c) (cu)  v = c(u  v)
u  v is defined as
u  v = uT v   (d) u  u > 0, and u  u = 0 if and only
v1 if u = 0
  v2 
 
= u1 u2 · · · un  .. 
.
vn Properties (b) and (c) can be combined
= u1 v1 + u2 v2 + · · · + un vn as

(c1 u1 + · · · + cp up )  w = c1 (u1  w) + · · · + cp (up  w)

The length of a vector v is defined as Theorem. Vectors u and v are orthog-

√ q onal if and only if

kvk = v v = v12 + v22 + · · · + vn2
kuk2 + kvk2 = ku + vk2 .

That is, vectors u and v are orthogonal

In particular,
if and only if the Pythagoras Theorem
kvk2 = v  v. holds for the triangle formed by u and
v as two sides (so that its third side is
u + v).

We call v a unit vector if kvk = 1. Proof is a straightforward computation:

Orthogonal vectors. Vectors u and v ku + vk2 = (u + v)  (u + v)

are orthogonal to each other if = uu + uv + vu + vv
= uu + uv + uv + vv
uv = 0
= u  u + 2u  v + v  v
= kuk2 + 2u  v + kvk2 .
MATH10212 • Linear Algebra B • Lecture 25 • Symmetric matrices and inner product • Last change: 23 Apr 2018 42

Hence is an orthogonal set of non-zero vectors

in Rn then S is linearly independent.
ku + vk2 = kuk2 + kvk2 + 2u  v,
and Proof. Assume the contrary that S
ku + vk2 = kuk2 + kvk2 iff u  v = 0. is linearly dependent. This means that
there exist scalars c1 , . . . , cp , not all of
 them zeroes, such that

Orthogonal sets. A set of vectors c1 u1 + · · · + cp up = 0.

{ u1 , . . . , up }
Forming the dot product of the left hand
in Rn is orthogonal if side and the right hand side of this equa-
ui  uj = 0 whenever i 6= j. tion with ui , we get

(c1 u1 + · · · + cp up )  ui = 0  ui = 0
Theorem 6.2.4. If
S = { u1 , . . . , up } After opening the brackets we have

c1 u1  ui + · · · + ci−1 ui−1 + ci ui  ui + ci+1 ui+1  ui + · · · + cp up  ui = 0.

In view of orthogonality of the set S, all be eigenvectors of a symmetric matrix A

dot products on the left hand side, with for pairwise distinct eigenvalues
the exception of ui  u, equal 0. Hence
what remains from the equality is
λ1 , . . . , λp .
ci ui  ui = 0
Since ui is non-zero,
Then
ui  ui 6= 0
and therefore ci = 0. This argument { v1 , . . . , vp }
works for every index i = 1, . . . , p. We
get a contradition with our assumption
that one of ci is non-zero.  is an orthogonal set.
Proof. We need to prove the following:
n
An orthogonal basis for R is a basis
which is also an orthogonal set.
For example, the two vectors If u and v are eigenvectors
 
1
 
1 for A for eigenvalues λ 6= µ,
, then
1 −1
form an orthogonal basis of R2 (check!). u  v = 0.
Theorem: Eigenvectors of symmet-
ric matrices. Let
v1 , . . . , vp For a proof, consider the following se-
MATH10212 • Linear Algebra B • Lecture 25 • Symmetric matrices and inner product • Last change: 23 Apr 2018 43

quence of equalities: Notice that this definition can reformu-

lated as
λu  v = (λu)  v 
= (Au)  v  1, if i = j
ui uj = .
0, if i 6= j
= (Au)T v
= (uT AT )v
Theorem: Coordinates with re-
= (uT A)v spect to an orthonormal basis. If
= uT (Av)
= uT (µv) { u1 , . . . , un }
= µuT v
is an orthonormal basis and v a vector
= µu  v. in Rn , then
Hence
v = x1 u1 + · · · + xn un
λu  v = µu  v
and where
(λ − µ)u  v = 0.
xi = v  ui , for all i = 1, 2, . . . , n.
Since λ − µ 6= 0, we have u  v = 0. 

Corollary. If A is an n × n symmetric Proof. Let

matrix with n distinct eigenvalues
v = x1 u1 + · · · + xn un ,
λ1 , . . . , λ n
then, for i = 1, 2, . . . , n,
then the corresponding eigenvectors

v1 , . . . , vn v  ui = x1 u1  ui + · · · + xn un  ui
= xi ui  ui
form an orthogonal basis of Rn . = xi .
An orthonormal basis in Rn is an or-
thogonal basis Theorem: Eigenvectors of symmet-
ric matrices. Let A be a symmetric
u1 , . . . , un
n × n matrix with n distinct eigenvalues
made of unit vectors,
λ1 , . . . , λ n .
kui k = 1 for all i = 1, 2, . . . , n.
Then Rn has an orthonormal basis made
of eigenvectors of A.
MATH10212 • Linear Algebra B • Lecture 26 • Orthogonal matrices • Last change: 23 Apr 2018 44

Lecture 26: Orthogonal matrices

A square matrix A is said to be orthog- An example of an orthogonal matrix:
onal if "√ √ #
AAT = I. 2 2
A= √2 2√
2 2
2
− 2

Some basic properties of orthogonal ma-

trices: Theorem: Characterisation of Or-
thogonal Matrices. For a square n × n
If A is orthogonal then matrix A, the following conditions are
equivalent:
• AT is also orthogonal.

• A is invertible and A−1 = AT . (a) A is orthogonal .

• det A = ±1. (b) Columns of A form an orthogonal

basis of Rn .
• The identity matrix is orthogonal.
MATH10212 • Linear Algebra B • Lectures 25 and 26 • Inner product spaces • Last change: 23 Apr 2018 45

Lectures 25 and 26: Inner product spaces

Inner (or dot) product is a function u  u > 0 and u  u = 0 if and
which associate, with each pair of vec- only if u = 0,
tors u and v in a vector space V a real
number denoted that is, if f is a continuous function on
u  v, [0, 1] and
Z 1
subject to the following axioms:
f (x)2 dx = 0
0
1. u  v = v  u
then f (x) = 0 for all x ∈ [0, 1]. This re-
2. (u + v)  w = u  w + v  w quires the use of properties of continuous
3. (cu)  v = c(u  v) functions; since we study linear algebra,
not analysis, I am leaving it to the read-
4. u  u > 0 and u  u = 0 if and only ers as an exercise.
if u = 0

Inner product space. A vector space Length. The length of the vector u is
with an inner product is called an inner defined as
product space. √
kuk = u  u.
Example: School Geometry. The or-
dinary Euclidean plane of school geome- Notice that
try is an inner product space:
kuk2 = u  u,
• Vectors: directed segments start- and
ing at the origin O kuk = 0 ⇔ u = 0.
• Addition: parallelogram rule
Orthogonality. We say that vectors u
• Product-by-scalar: stretching the and v are orthogonal if
vector by factor of c.
u  v = 0.
• Dot product:
u  v = kukkvk cos θ,
where θ is the angle between vec- The Pythagoras Theorem. Vectors
tors u and v. u and v are orthogonal if and only if

kuk2 + kvk2 = ku + vk2 .

Example: C[0, 1]. The vector space
C[0, 1] of real-valued continuous func-
tions on the segment [0, 1] becomes an
inner product space if we define inner Proof is a straightforward computation,
product by the formula exactly the same as in Lectures 25–27:
Z 1
 ku + vk2 = (u + v)  (u + v)
f g= f (x)g(x)dx.
0 = uu + uv + vu + vv
In this example, the tricky bit is to show = uu + uv + uv + vv
that the dot product on C[0, 1] satisfies = u  u + 2u  v + v  v
the axiom: = kuk2 + 2u  v + kvk2 .
MATH10212 • Linear Algebra B • Lectures 25 and 26 • Inner product spaces • Last change: 23 Apr 2018 46

Hence Theorem: The Cauchy-Schwarz In-

equality. In any inner product space,
ku + vk2 = kuk2 + kvk2 + 2u  v,
and |u  v| 6 kukkvk.
ku + vk2 = kuk2 + kvk2
if and only if
The Cauchy-Schwarz Inequality:
u  v = 0.
an example. In the vector space Rn
 with the dot product of
  
The Cauchy-Schwarz Inequality. In u1 v1
 ..   .. 
the Euclidean plane, u =  .  and v =  . 
un vn
u  v = kukkvk cos θ
hence defined as
|u  v| 6 kukkvk.

The following Theorem shows that this u  v = uT v = u1 v1 + · · · + un vn ,

is is a general property of inner product
spaces. the Cauchy-Schwarz Inequality becomes

q q
|u1 v1 + · · · + un vn | 6 u21 + ··· + u2n · v12 + · · · + vn2 .
or, which is its equivalent form,

(u1 v1 + · · · + un vn )2 6 (u21 + · · · + u2n ) · (v12 + · · · + vn2 ).

The Cauchy-Schwarz Inequality: uct space C[0, 1] the Cauchy-Schwarz In-

one more example. In the inner prod- equality takes the form

Z 1
 sZ 1
s
Z 1
 

 f (x)g(x)dx 6 f (x)2 dx · g(x)2 dx,
0 0 0

or, equivalently,
Z 1 2 Z 1  Z 1 
2 2
f (x)g(x)dx 6 f (x) dx · g(x) dx .
0 0 0

Proof of the Cauchy-Schwarz In- Let

equality given here is different from the
q(t) = at2 + bt + c
one in the textbook by Lay. Our proof is
based on the following simple fact from be a quadratic function in
school algebra: variable t with the property
MATH10212 • Linear Algebra B • Lectures 25 and 26 • Inner product spaces • Last change: 23 Apr 2018 47

that 2. d(u, v) > 0.

q(t) > 0
3. d(u, v) = 0 iff u = v.
for all t. Then
4. The Triangle Inequality:
b2 − 4ac 6 0.
d(u, v) + d(v, w) > d(u, w).
Now consider the function q(t) in vari-
able t defined as

q(t) = (u + tv)  (u + tv) Axioms 1–3 immediately follow from the

= u  u + 2tu  v + t2 v  v axioms for dot product, but the Triangle
Inequality requires some attention.
= kuk2 + (2u  v)t + (kvk2 )t2 .
Proof of the Triangle Inequality. It
Notice q(t) is a quadratic function in t suffices to prove
and
q(t) > 0. ku + vk 6 kuk + kvk,
Hence or
ku + vk2 6 (kuk + kvk)2 ,
(2u  v)2 − 4kuk2 kvk2 6 0
or
which can be simplified as
(u+v)  (u+v) 6 kuk2 +2kukkvk+kvk2 ,
|u  v| 6 kukkvk.
or

u  u+2u  v+v  v 6 u  u+2kukkvk+v  v,
Distance. We define distance between
vectors u and v as or
2u  v 6 2kukkvk
d(u, v) = ku − vk.
But this is the Cauchy-Schwarz Inequal-
It satisfies axioms of metric: ity. Now observe that all rearrange-
ments are reversible, hence the Triangle
1. d(u, v) = d(v, u). Inequality holds. 
MATH10212 • Linear Algebra B • Lecture 27 • Orthogonalisation and the Gram-Schmidt Process • Last change: 23 Apr 201848

Lecture 27: Orthogonalisation and the Gram-Schmidt

Process
Orthogonal basis. A basis v1 , . . . , vn is called orthonormal if it is orthogonal
in the inner product space V is called and kvi k = 1 for all i.
orthogonal if vi  vj = 0 for i 6= j.
Equivalently,
Coordinates in respect to an or- 
thogonal basis: 1 if i = j
vi  vj = .
0 if i 6= j
u = c1 v1 + · · · + cn vn
Coordinates in respect to an or-
iff thonormal basis:
u  vi
ci =
vi  vi u = c1 v1 + · · · + cn vn

Orthonormal basis. A basis iff

v1 , . . . , vn in the inner product space V ci = u  vi

The Gram-Schmidt Orthogonalisation Process makes an orthogonal basis from a

basis x1 , . . . , xp :

v1 = x1
x2  v1
v2 = x2 − v1
v1  v1
x3  v1 x3  v2
v3 = x3 −  v1 −  v2
v1 v1 v2 v2
..
.
xp  v1 xp  v2 xp  vp−1
vp = xp −  v1 −  v2 − · · · − vp−1
v1 v1 v2 v2 vp−1  vp−1
Main Theorem about inner prod- the standard dot product
uct spaces. Every finite dimensional  
inner product space has an orthonormal v
  .1 
basis and is isomorphic to on of Rn with u  v = uT v = u1 · · ·

un  ..  .
vn
MATH10212 • Linear Algebra B • Lectures 28–30 • Revision • Last change: 23 Apr 2018 49

Lectures 28–30: Revision, Systems of Linear Equa-

tions

Systems of linear equations

Here A is an m × n matrix. and equals to the span of the system of
vectors a1 , . . . , an , that is, the set of all
We associate with A systems of simul-
possible linear combinations of vec-
taneous linear equations
tors a1 , . . . , an .
Ax = b,
The column space of A has another im-
m
where b ∈ R and portant characterisation:
 
x1
x =  ... 
  Col A = {b ∈ Rm : Ax = b has a solution}.
xn
is the column vector of unknowns. Assume now that A is a square n×n
matrix.
Especially important is the homoge-
neous system of simultaneous linear If we associate with A the linear trans-
equations formation
Ax = 0.
The set TA : Rn −→ Rn
v 7→ Av
null A = {x ∈ Rn : Ax = 0}
is called the null space of A; it is a vec- then the column space of A gets yet an-
tor subspace of Rn and coincides with other interpretation: it is the image of
the solution space of the homogeneous the linear transformation TA :
system Ax = 0.
If a1 , . . . , an are columns of A, so that Col A = Im TA
 
A = a1 · · · an , = {v ∈ Rn : ∃v ∈ Rn s.t. TA (w) = v},

then the column space of A is defined

and the null space of A is the kernel of
as
TA :
Col A = {c1 a1 + · · · + cn an : ci ∈ R}
= Span {a1 , . . . , an } null A = ker TA
= {v ∈ R : TA (v) = 0}.
MATH10212 • Linear Algebra B • Lectures 28–30 • Revision • Last change: 23 Apr 2018 50

Linear dependence
If a1 , . . . , ap are vectors in a vector space and form a matrix A using vectors
V , an expression a1 , . . . , ap as columns, then
c1 a1 + · · · + cp ap , ci ∈ R  
is called a linear combination of the c
   .1 
vectors a1 , . . . , ap , and coefficients ci are c1 a1 + · · · + cp ap = a1 · · · ap  .. 
called weights in this linear combina- cp
tion. If = Ac
c1 a1 + · · · + cp ap = 0
we say that weights c1 , . . . , cp de- and linear dependencies between vec-
fine a linear dependency of vectors tors a1 , . . . , ap are nothing else but solu-
a1 , . . . , ap . We always have a trivial tions of the homogeneous system of lin-
(or zero) dependency, in which all ear equation
ci = 0. We say that vectors a1 , . . . , ap
are linearly dependent if the admit a
Ax = 0.
non-zero (or non-trivial) linear depen-
dency, that is, a dependency in which at
least one weight ci 6= 0. Therefore vectors a1 , . . . , ap are linearly
dependent if and only if the system
Let us arrange the weights c1 , . . . , cp in a
column  
c1 Ax = 0
 .. 
c=.
cp has a non-zero solution.

Elementary row transformation

Elementary row transformations and therefore do not change dependen-
cies between columns:
Ri ↔ Rj , i 6= j
Ri ← Ri + λRj , i 6= j Ac = 0 ⇔ (EA)c = 0.
Ri ← λRi , λ 6= 0
Therefore if certain columns of A were
performed on a matrix A amount to mul-
linearly dependent (respectively, inde-
tiplication of A on the left by corre-
pendent) then the same columns of EA
sponding elementary matrices,
remain linearly dependent (respectively,
A ← EA, independent)

Rank of a matrix
We took it without proof that if U is a U and is denoted dim U .
vector subspace of Rn , then the maximal
A maximal system of linearly indepen-
systems of linearly independent subsets
dent vectors in U is called a basis of U .
in U contain the same number of vectors.
This number is called the dimension of Each basis of U contains dim U vectors.
MATH10212 • Linear Algebra B • Lectures 28–30 • Revision • Last change: 23 Apr 2018 51

An important theorem: .
dim null A + dim Col A = n Theorem. If rank A = p then among
columns of A there are p linearly inde-
dim Col A is called the rank of A and is pendent columns
denoted
Theorem. Since elementary row trans-
rank A = dim Col A. formations of A do not change depen-
dency of columns, e=elementary row op-
Therefore
eration do not change rank A.
dim null A + rank A = n