1

1 Systems of linear equations

One of the most important problems in mathematics is that of solving a sys-

tem of linear equations. Such linear systems arise in many applications in en-
gineering, including electrical networks, mechanical frameworks, traffic flow,
economic models, and optimisation problems.

The following equations are linear:

1
2 + 3y = 0, y=  + 4z + 6, 1 − 22 − 33 + 4 = 7
2

The following equations are not linear:

p
 + 3 y = 5, 3 + 2y − z = 0, y = sn()
 2

An example
Using Kirchhoff’s laws, the following circuit

20 Ω 10 Ω
Q

i1 i3

80 V 10 Ω 90 V

i2

P
15 Ω

gives a system of four linear equations in three unknowns:

Node P: 1 − 2 + 3 = 0
Node Q: −1 + 2 − 3 = 0
Right loop: 102 + 253 = 90
Left loop: 201 + 102 = 80

To find the values of the currents 1 , 2 , and 3 , we need to solve this system.
 3

Another example

Is there a parabola which passes through the three points (1, 1), (3, 2) and
(9, 10)? If so, what is its equation?
 4

Goals

In this topic we will answer the following questions:

• How can we find a solution to a linear system?
• How many solutions can there be?
• How can we describe all of the solutions to a system?
• How can we interpret the set of solutions geometrically?
 5

Terminology

A linear system of m equations in the n unknowns (or variables)

1 , 2 , 3 , . . . , n
is a system of the form
11 1 + 12 2 + · · · + 1j j + · · · + 1n n = b1
21 1 + 22 2 + · · · + 2j j + · · · + 2n n = b2
...
1 1 + 2 2 + · · · + j j + · · · + n n = b (the th eqn)
...
m1 1 + m2 2 + · · · + mj j + · · · + mn n = bm
where the j s and b s are all numbers. The numbers j are the coefficients
of the system, and the numbers b are sometimes called the right-hand side
(or RHS).
(The first subscript of j refers to the equation and the second to the variable.
For example, 23 is the coefficient of 3 in the second equation.)
We call this an m × n linear system. By a solution to an m × n system of
equations, we mean an ordered n-tuple of numbers (1 , 2 , . . . n ) that satisfies
all the equations of the system. We aim to find all solutions to a given system:
the general solution or solution set.
If a system of equations has no solution, we say it is an inconsistent system.
 6

Exercise 1 A simple case - 1 × 1 linear systems.

Solve the following equations.

(a) 3 = 2 (b) 0 = 2 (c) 0 = 0

What do we mean by a solution to an equation?

What do we mean by the solution, or the general solution, to an equation.

What are the three possible cases for the solution of  = b?

 7

2 Triangular systems and back-substitution

The simplest equation you can solve is of the form  = b. So the simplest
systems to solve are diagonal systems such as
5 = 15
2y = 4
3z = −6 ,
which has the (unique) solution (, y, z) = (3, 2, −2).
An upper-triangular system is also easy to solve. For example, consider
4 + y + 2z = 15
2y + z = 4
3z = −6 .
• The last equation gives z = −2.
• Substituting this into the second equation and solving for y gives y = 3.
• Substituting these both into the first equation and solving for  gives  = 4.
• So the (unique) solution is (, y, z) = (4, 3, −2).

This method for solving an upper-triangular system is called back-substitution.

 8

Exercise 2 Back Substitution

Solve the following linear system using back substitution.
−y−z = 2
y + 3z = 5
5z = 10
 9

Echelon form
Back-substitution works more generally for matrices in echelon form

• A system is in Echelon Form (EF) if the first variable in

each equation is further to the right as we move down the
equations.

• The pivot variables (or leading variables) of a system in

EF are those variables that are first in one of the equations.

• The variables that are not pivot variables are free variables

Example: A system in echelon form

2 + 3y − 4z = 1
5z = 16 .

Example: A system not in echelon form

2 + 3y − 4z = 1
2 = 16 .
 10

Exercise 3 Back-substitution example

Identify the free variables in the following system and solve by back-substitution.
 + y + z −  = 2
2z + 2 = 0
 11

Exercise 4 Pivots and free variables

Identify the free variables in the following system and solve it. Give a geometric
interpretation of the solution.
3 + 2y − z = 3
 12

Exercise 5 Pivots and free variables

Identify the free variables in the following system and solve it. Give a geometric
interpretation of the solution.
3 + 2y − z = 3
y =1
 13

Exercise 6 A linear system not in EF

Solve the following linear system and give a geometric interpretation of the
solution.
+y=1
− y = 2.
 14

3 Elimination and back-substitution

To solve a linear system one can use an algorithm called Gaussian Elimina-
tion to obtain an equivalent linear system in echelon form, which can then be
solved by back-substitution. This algorithm is based on the following:
Elementary operations
1. Multiplying an equation through by a nonzero constant.
2. Interchanging two equations.
3. Adding a multiple of one equation to another equation.

Definition: Two linear systems are equivalent if one can be obtained from
the other via a sequence of elementary operations.

Note: If two systems are equivalent, then they have the same set of solutions.
 15

Exercise 7 Solving a system using elimination and back-substitution.

Use elementary operations to find an equivalent system in echelon form and

then solve it using back-substitution. How can you interpret your solution geo-
metrically?
 + 2y + z = 2
3 + 4y + z = 12
 16

Augmented Matrices

For more complicated systems it is worth introducing the shorthand notation of

augmented matrices. For example, the system of equations

2 + 3y + 4z = 19
−2 + 2y − 13z = −22
4 + 16y − 6z = 40
can be written as the augmented matrix
 
2 3 4 : 19
−2 2 −13 : −22
4 16 −6 : 40
Exercise 8 Augmented matrices

Write each system of equations as an augmented matrix.

2 − 5y = 0 2 − y + 3z = 0
(a) (b) .
7  + 2y = 0 2y − z = 6
 17

Elementary row operations

The elementary operations for the system correspond to the following elemen-
tary row operations, which can be used to obtain a (row) equivalent matrix
in echelon form:

we have
Elementary row operations
1. Multiplying a row through by a nonzero constant.
2. Interchanging two rows.
3. Adding a multiple of one row to another different row.

Exercise 9 Using elementary row operations

Use EROs to find a matrix in row echelon form equivalent to the augmented
matrix  
1 −1 : 6
3 2 : 1
 18

Exercise 10 Solving our introductory example

Write the following system of equations as an augmented matrix and then solve
it using Gaussian elimination and back substitution
1 − 2 + 3 =0
− 1 + 2 − 3 =0
102 + 253 = 90
201 + 102 = 80
19
 20

Exercise 11 Gaussian elimination

Solve these systems of equations by Gaussian elimination.

(a) 1 + 32 − 3 = 4
1 + 22 + 3 = 2
31 + 72 + 3 = 9
21
 22

(b) 1 + 22 + 33 + 44 = 5

21 + 42 + 63 + 94 = 12
31 + 62 + 103 + 144 = 18
41 + 82 + 103 + 144 = 18
23
 24

4 Determining the number of solutions to a linear system

There are three possibilities for the number of solutions to a linear system.
These can easily be read off from the (row) echelon form of the system (or its
augmented matrix).
 
• If there is a row of the form 0 . . . 0 : c with c 6= 0, then there is no solution.
• Otherwise, there is at least one solution:
– If there are no free variables, then there is a unique solution.
– Otherwise there are infinitely many solutions. In this case the general
solution will be described using one free parameter for each free variable.

Exercise 12 The number of solutions

For each of the following augmented matrices state the number of solutions.
 
1 2 3 : 0
(a) 0 2 2 : 0
0 0 2 : 1
 25

 
1 2 3 : 0
(b) 0 2 2 : 0
0 0 0 : 1

 
1 2 3 : 0
(c) 0 0 2 : 0
0 0 0 : 0

 
1 2 3 4 5 6 : 0
(d) 0 0 0 0 3 2 : −1
0 0 0 0 0 2 : 1
 26

Exercise 13 Additional Exercises

(a) Find the value(s) of p for which the following system of equations has solu-
tions.
1 + 22 + 43 = p
31 + 2 + 43 = p + 1
1 + 72 + 123 = p + 2
 27

(b) Discuss the nature of the solution of the system of equations

1 + 2 + 3 = 1
1 − 2 = 
1 − k3 = 2
where  and k are constants.
 28

Applications of linear systems

Exercise 14 Traffic Networks

The accompanying figure shows a network of one-way streets with traffic flow-
ing in the directions indicated. The flow rates along the streets are measured
as the average number of vehicles per hour.
Set up a linear system whose solution provides the unknown flow rates and
then solve this system.
Given that flows must be non-negative (traffic cannot go against the one-way
system), what is the minimum flow rate for each of the unknowns?

400 750
300 x3 250
C A
x2 x4
400 D B 200
x1
100 300
29
 30

5 Matrices and their algebra (A.L. 1.3)

What is a Matrix? A matrix is a rectangular array of numbers. If A is a matrix,

we use Aj to denote the entry in row  and column j.

What are matrices good for? Anything that involves linear algebra.

Goals of the next few sections

• learn matrix algebra, i.e., the rules for addition and multipli-
cation of matrices.
• learn to write a system of linear equations as a single alge-
braic equation with matrices
• learn how to find a matrix inverse to solve such equations
• understand the connection between inverses and solutions
of linear systems
 31

How big is a matrix?

The size (or dimension or order) of a matrix is determined by the number of
rows and columns. A matrix with m rows and n columns has size m × n.
Exercise 15 Matrix orders
What are the orders of the following matrices?
 
1 2
(a) 3 4
5 6

 
(b) 1 2 3 (a row vector)

 
1
(c) 2 (a column vector)
3

(d) [ 3] (a scalar)
 32

Some special matrices

• square matrix an n × n matrix;

• diagonal matrix: square, and the only non-zero entries are on the main
diagonal.

• identity matrix: diagonal matrix with all main diagonal entries 1 (denoted 
or n ).

• block matrix: these are built out of smaller matrices. Many configurations
are possible, subject to consistency in number of rows and/or columns.=
 33

Adding and scalar multiplication

• Addition and scalar multiplication of matrices are done entry by entry.

• Matrices can only be added together when they have the same order; we
then say that the matrices conform for addition or subtraction.

If A and B conform then the j entry of the matrix A + B is Aj + Bj .

Exercise 16 Matrix operations

Using the following matrices

       
2 1 1 −2 0 1 1 0 −1 0 0 0
A= , B= , C= , D=
3 0 −1 2 3 5 0 1 1 0 0 0
find
(a) 2 C

(b) −A

(c) A + (−A)
 34

       
2 1 1 −2 0 1 1 0 −1 0 0 0
A= , B= , C= , D=
3 0 −1 2 3 5 0 1 1 0 0 0

(d) B + D

(e) A + B

(f) B + A
 35

Matrix multiplication

Matrix multiplication is not done entry by entry.

Matrices A and B can be multiplied to form the matrix AB if the rows of A are
the same size as the columns of B. In other words if A is m × n and B is p × q
then AB is only defined when n = p.
If A is m × n and B is n × p then the product AB is the m × p matrix with j entry
(AB)j = 1 b1j + 2 b2j + 3 B3j + · · · n nnj
Xn
= k bkj .
k=1

In words to find the j entry of the product

go across the -th row of A and down the j-th column of B.
To illustrate the row–column product we calculate an entry in the product AB,
where    
1 2 −2 2 1
A =  2 1 0  and B = 4 −2 .
−3 1 1 5 3
 36

Exercise 17 Matrix multiplication

Multiply the following matrices. (You should first check the order of the matri-
ces.)
   
2 3 2
(a)
6 7 −1

       
2 3 1 2 3 0
(b) and
6 7 0 6 7 1

 
  1 2
1 2 3 
(c) −1 0
4 5 6
1 1
 37

 
1 2  
1 −1 1
(d) 3 4
1 0 1
5 6

 
1  
(e) 3 1 −1 2
5

 
  1
(f) 1 −1 2 3
5
 38

Exercise 18 Matrix multiplication

Do the following multiplications. Each example is meant to illustrate an idea.

   
1 2 1 −1
(a) 3 4 1 0 
5 6 2 3

   
3 2 2 4
(b)
−6 −4 −3 −6
 39

   
3 2 1 0
(c)
−6 −4 0 1

   
3 0 7 5 −7 −28
(d) 2 4 1 −2 3 11 
0 −1 1 −2 3 12
 40

Exercise 19 Matrix multiplication

Find AB, BA, (AB)C and A(BC) where

     
2 1 −2 0 1 0
A= , B= , C=
3 0 2 3 4 1
 41

Note: Matrix multiplication is not commutative. The order matters! In

general, AB 6= BA. But multiplication is associative, i.e. (AB)C = A(BC).
 42

6 Linear systems and matrix equations

It is easy to write a system of equations as a matrix equation. For example,

 
  
+ cy = b becomes  c =b
y
and
2 + 3y + 4z = 8
6 + 7y + 8z = 20
becomes the matrix equation
 
    
2 3 4   8
y =
6 7 8 20
z
or
Ax = b

Here A is the coefficient matrix, x is the unknown column vector and b is the
known RHS column vector.
 43

Exercise 20 Matrix equations

Write the following system of equations as a matrix equation.

1 + 32 − 3 = 4
1 + 22 + 3 = 2
31 + 72 + 3 = 9
 44

More generally, if
     
11 12 . . . 1j . . . 1n 1 b1
 21 22 . . . 2j . . . 2n   2   b2 
 .  .   . 
 ..  ..   .. 

  
A = j =   1
, x =   and b = 
 b  ,

2 . . . j . . . n   j 
 ...  ...   ... 
     

m1 m2 . . . mj . . . mn n bm

then the matrix equation Ax = b represents the m× n linear system of equations

11 1 + 12 2 + . . . + 1j j + . . . + 1n n = b1
21 1 + 22 2 + . . . + 2j j + . . . + 2n n = b2
...
1 1 + 2 2 + . . . + 2j j + . . . + n n = b
...
m1 1 + m2 2 + . . . + 2j j + . . . + mn n = bm

The th entry of the product Ax is given by

n
X
(Ax) = 1 1 + 2 2 + · · · + n n = j j (1 ≤  ≤ m) .
j=1
 45

Example: The system 3 + 2y = 5

+y=2
can be written as the matrix equation
       
3 2  5 3 2
= where A = .
1 1 y 2 1 1

Exercise 21
Solve this equation by multiplying both sides from the left by the matrix
 
1 −2
C = .
−1 3
 46

7 Invertible matrices (A.L. 1.4)

Recall (see Exercise 18(c)) that the identity matrix  is a square matrix with
1s down the main diagonal, and 0s everywhere else. Multiplying by an identity
matrix (of the appropriate size) doesn’t change a matrix.
A square matrix A is invertible if there is some (square) matrix
C such that
C A =  = A C.

Comments
1
• C is called the inverse of A. We write C = A−1 . NEVER WRITE .
A

• Invertible matrices are sometimes called non-singular.

• If a square matrix is not invertible, it is called singular.
• If A is invertible, then we can solve any linear system Ax = b by multiply-
ing both sides of the equation on the left by the inverse of A to obtain the
(unique) solution x = A−1 b.
Ax = b
=⇒ A−1 A x = A−1 b
=⇒  x = A−1 b
=⇒ x = A−1 b .
 47

In particular, if b = 0, then the only solution to the homogeneous equation,

Ax = 0, is the trivial one, x = 0.
• When A is square, to show that C = A−1 , you need to show either A C =  or
C A = . Either one of these statements implies the other one.
• For large matrices A, solving a matrix system Ax = b by first finding A−1
and then multiplying b by A−1 is inefficient. Solving the system by Gaussian
elimination is more efficient as it takes fewer arithmetic operations.
• Matrix equations are easy to solve if we have the inverse. But, finding the
inverse can take many operations.
 48

Exercise 22 Solving a system with an inverse

   
1 2 3 −40 16 9
Consider the matrices A = 2 5 3 and A−1 =  13 −5 −3 .
1 0 8 5 −2 −1
(Check that these two matrices do multiply together to give 3 .)
 
0
Solve for x in the matrix equation Ax = b where b = 1.
1
 49

2 × 2 inverses
−1
1
  
 b d −b
= , when d − bc 6= 0 .
c d d − bc −c 

Check:
1 1
       
d −b  b d − bc 0 1 0
A−1 A = = =
d − bc −c  c d d − bc 0 d − bc 0 1
(and similarly, AA−1 = 2 ).
(Optional exercise: Show that if d − bc = 0 then A has no inverse.)
 50

Exercise 23 Matrix inverses

   
1 2 1 2
Use the formula to find, if possible, the inverses of and .
3 4 0 0
 51

A method for finding A−1 when A is not 2 × 2 (A.L. 1.5)

The following gives an algorithm to compute inverses (if they exist); it can be
programmed to handle large matrices.

If A−1 exists, we can find it by row-reducing [ A | ] to [  | C]. Then

C = A−1 .

Justification: If
C = [ c1 |c2 | . . . |cj | . . . |cn ]
is the inverse of A then we need to solve the matrix equation AC =  for C where
 = [ e1 |e2 | . . . |ej | . . . |en ] .
Here ej is the j-th column of the n × n identity matrix n . It is the column vector
with a 1 in the j-th position and zeros elesewhere.
This can be done a column at a time, n times, by solving the system:
Acj = ej for cj , that is, by row reducing [ A | ej ] to [  | cj ].

Since the coefficient matrix A is the same for each of the n systems, the algo-
rithm above carries out the elimination process on all n equations simultane-
ously.
 52

Exercise 24 Matrix inverses

 
1 1 2
Find the inverse of A = 1 1 1.
2 1 1
53
 54

Elementary matrices
Definition: An elementary matrix is a square matrix which as a pre-multiplier
performs an elementray row operation. The elementary matrix performing a
particular row operation can be found by performing the same operation on the
identity.

Exercise 25 Elementary matrices

• Find the 3 × 3 elementary matrix which as a premultiplier adds two times
row two to row three.
• Calculate EA where A is the matrix
 
 b c
d e ƒ  .
g h 

Row elimination can now be viewed as pre-multiplying by a finite sequence of

elementary matrices E1 , E2 , . . . , Ep . So the combination of all the operations
can be achieved by pre-multiplying by the single matrix B = Ep . . . E2 E1 .
 55

Exercise 26 Inverses of products

Show that if A and B are invertible n × n matrices then so is AB, and

(AB)−1 = B−1 A−1 .
 56

When is A Invertible?

Theorem 27 Let A be an n × n matrix. Then the following conditions

are equivalent.
1. A is invertible.
2. For each n-vector b the matrix equation A x = b has a unique solu-
tion.
3. Ax = 0 implies that x = 0. That is, the only solution to the homoge-
neous equation is the trivial one.
4. If A and B are row-equivalent then B has no rows of zeros.
5. A is row equivalent to n . That is A can be reduced to n by elemen-
tary row operations.
 57

Proof of Theorem 27

(a) (1)⇒(2): If A is invertible, then the equation Ax = b can be solved by multi-

plying by A−1 . This yields the unique solution x = A−1 b.

(b) (2)⇒(3): If (2) holds, then Ax = 0 has a unique solution. Since x = 0 is a

solution, it must be the only solution.

(c) (3)⇒(4): We prove that if (4) is false then (3) must be false. Suppose that A
is row-equivalent to a matrix B with a row of zeros. Perform row operations
to convert A into B; then perform further row operations to put B into REF.
Let this REF be C. Then C has at least one row of zeros, and A and C are
row-equivalent. Hence the systems A x = 0and C x = 0 all have the same
solutions. The latter system has at most (n − 1) pivots (since C has a row of
zeros), so there is at least one free variable. Thus there is v 6= 0 such that
C(t v) = 0 for all t ∈ R. Since x = v solves A x = 0, (3) is false.

(d) (4)⇒(5): Apply Gaussian elimination to obtain a REF of A. Call it B. Since

A and B are row-equivalent, B has no row of zeros, so every row contains a
pivot and there are n pivots. Thus, every column contains a pivot and further
elementary row operations can be performed to reduce A to n .

(e) (5)⇒(1): Row operations can be used to reduce [ A : n ] to [ n : C]. Then

C = A−1 .
 58

Theorem 28 Let A, B and C be n × n matrices.

(a) If A is invertible then A−1 is invertible and (A−1 )−1 = A.
(b) If A and B are invertible then A B is invertible and (A B)−1 = B−1 A−1 .
(c) If C A =  then A is invertible and A−1 = C.
(d) If B C =  then B is invertible and B−1 = C.

Proof of Theorem 28

(a): Immediate from using C = A−1 in the definition of invertibility: A−1 A = n =

A A−1 .
(b): Done earlier; see exercise 26.
(c): Suppose CA = . Then any equation Ax = b can be solved by left-multiplying
by C to get x = CAx = Cb. In particular, we see that there is a unique solu-
tion. So A is invertible by Theorem 27. Moreover, A−1 =  A−1 = (C A) A−1 =
C (A A−1 ) = C  = C.
(d): By (c), C is invertible and C−1 = B. By (a), B = C−1 is invertible, and B−1 =
(C−1 )−1 = C.
 59

8 The transpose of a matrix

Given an m × n matrix A, it is often useful to form a new n × m matrix, whose

columns are the rows of A. We call this new matrix the transpose of A and
denote it by AT .
T
Thus if the j entry of A is j, then  n × m matrix B = A is
 the j entryof the
1 2 1 3
bj = j . For example, if A = then AT = .
3 4 2 4

Exercise 29 Transposing
 a matrix
1 2 3
(a) If A = , find AT and (AT )T .
4 5 6
 60

   
−3 2 1 0 1 3
(b) If B = and C = , verify that (B + C)T = BT + CT .
4 3 2 −1 1 2

In general it is true that (B+ C)T = BT + CT . Is it true that (B+ C)−1 = B−1 + C−1 ?
 61

   
−1 4
(c) If x =  2  and y =  7 , find xT , yT , xT y and yT x.
−1 10
What do you notice?

 
1 2
(d) If C = , find CT .
2 3
 62

Exercise30 Transpose
 of a product
2 1  
2 4
1. If C =  6 3 and D = , verify that (CD)T = DT CT .
1 6
−2 4

2. Show that for arbitrary matrices A and B, we have (AB)T = BT AT whenever

one of the products is defined.
63