Week 2 Notes:

1) Elementary Row Operations (EROs)

Here are the 3 elementary operations on any matrices which are helpful in solving linear equations:

1) Row Replacement    (It is DIFFERENT from   

: the row that gets changed “ ” is placed last)

⋮ ⋮
 
    
  
⋯
   ⋯   
 ⋮  ↦  ⋮ 


⋯   

⋯
⋮   ⋮ 

For example, 5   gives

1 2 3 5  1 5  2 5  3
  ↦  
   

2) Row Interchange  ↔ 

⋮ ⋮
 
  ⋯ 

⋯
   
 ⋮  ↦  ⋮ 


⋯   ⋯
⋮  ⋮ 

For example,  ↔  gives

1 2 3  
  ↦  
  1 2 3

3) Row Scaling  (by a non-zero number )

⋮ ⋮ ⋮ ⋮
  ⋯ ↦    ⋯
⋮ ⋮ ⋮ ⋮

For example, 5 gives

1 2 3 1 2 3
  ↦  
  5 5 5
We can do these operations on any matrices. For instance, we often do these operations on augmented
matrices. In doing so, the solutions of the system of equations will not be changed.
Remarks:

1) Don’t use equal sign = when doing EROs. The matrices before and after operations are 2 different
matrices.
2) Although the matrices we get before and after EROs are different, they are called row equivalent.

1 1 2
3) The operations are invertible, that is, we can ‘undo’ an ERO by another ERO.

4) The order of ERO matters, e.g. for " = 3 2 5 . Try the following:
4 1 3
(a) first do −2   , then do  ↔ %

(b) first do  ↔ % , then do −2  

It will give you two different matrices (but they are row equivalent)

2) Row echelon form (REF) and reduced row echelon form (RREF)
Recall that we (1) have a system of linear equations, then (2) write the system into augmented matrix, then
we can (3) use ERO on the augmented matrix to solve the system.

We cannot use ERO forever. We usually stop using ERO when the matrix is nice enough to be solved, e.g.

(  (%  2() = 3
(  2( − (% = 5
'
2(  ( − 5(% = 0
( − 3(%  8() = −3
0 1 1 2 | 3 1 2 −1 0 | 5
1 2 −1 0 | 5 0 1 1 2 | 3
has augmented matrix , . . Use ERO and we can get the matrix , ., which
2 1 −5 0 | 0 0 0 0 6 | −1
1 0 −3 8 | −3 0 0 0 0 | 0
corresponds to the system

(  2( − (% = 5
(  (%  2() = 3
'
6() = −1
0 = 0
This is much nicer because we can use backwards substitution to solve it. The final matrix is actually called row
echelon form (REF) as we define below:
Definition:

Any matrix 0 is in row echelon form (REF) if

⋯ ⋯ ⋯
1. The rows with all zero entries (if any) must be at the bottom, i.e. 1⋯ ⋯ ⋯2. If M has no rows of zeros, this
0 ⋯ 0
condition is vacuous, and is automatically true.

2. The leading entry (the first non-zero entry on each row) must move to the right by at least one column

⋯ ⋯ ⋯ ⋯
when going down a row:

0 ∗ ⋯ ⋯
3 0 0 … 6
0 0 0 …

3. All entries in a column below a leading entry must be zeros.

Examples:

7 1 2 2 3 0 0 8 2 3
0 8 3 5 1 0 0 0 0 8
, ., , .
0 0 0 9 1 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
are in REF, while

0 8 2 2 3 0 8 5 2 3
8 0 0 1 1 0 8 0 1 1
, . , , .
0 0 0 0 0 0 0 9 4 0
0 0 0 0 0 0 0 0 0 0
are not in REF. (why?)

Algorithm to get REF from a random matrix:

1. Stop when all entries are zeros, or no visible entry.

2. Locate the left-most non-zero column, select a non-zero entry, use (2 row interchange) to
move it to the top row.
3. Use (1 row replacement) to make all entries below 0.
4. Forget the top row, and repeat Step 1-3 for the submatrix below that row.

Exercise:

(  (  2(% = 
Write the following system in REF (there may be more than one possible answer):

2(  3(  2(% = 

'
( − 2(% = %
3(  5(  2(% = )
Solution: THERE CAN BE DIFFERENT CORRECT ANSWERS FOR THIS EXERCISE. ONE WAY TO DO IT IS:

1 1 2 |  1 1 2 | 
2 3 2 |  0 1 −2 |  − 2 −2;  ;
,  . → ,  .
0 1 −2 | % 0 1 −2 | %
3 5 2 | ) 0 2 −4 | ) − 3 −3;  ;)

1 1 2 | 
0 1 −2 |  − 2
→ , . −;  ;
0 0 0 | % −   2  %
0 0 0 | )   − 2 −2;  ;)

In fact we can simplify an REF matrix further into an RREF matrix as defined below:

Definition:

A reduced row echelon form (RREF) of any matrix is a REF with the extra properties:

⋯ ⋯ ⋯ ⋯
5. On a non-zero row, the leading entry must be 1.

0 1 ⋯ ⋯
3 0 0 1 … 6
0 0 0 …

⋯ 0 0 ⋯
6. On the columns containing the leading entry 1, the “1” is the ONLY non-zero entry.

0 1 0 ⋯
, 0 … .
0 1
0 0 0 …

Algorithm from a REF to a RREF:

5. Use (3 row scaling) to scale all the leading entries to 1.

6. Working from the rightmost leading entries to left, use (1 row replacement) to make all
entries above each of them 0.

Example:

To transform a REF to a RREF:

 1 2 −1 0 | 5 8 < −3 < | −5/3
0 1 1 2 | 3 0 8 1 < | 10/3
, . → , .
0 0 0 6 | −1 0 0 0 8 | −1/6
0 0 0 0 | 0 0 0 0 0 | 0

Here is the most important aspect of RREF:

Theorem:

Each matrix is row equivalent to a unique RREF.

With this theorem, we can define the notion of pivot columns, which gives us information of free parameters.

Definition:

The pivot positions of any matrix 0 are the positions of the leading 1’s in the RREF of 0. (equivalently, the
leading non-zero entry in the REF of 0)

The pivot columns of any matrix 0 are the columns containing the pivot positions of 0.

The free columns of any matrix 0 are the columns NOT containing the pivot positions of 0.

Example:

1 2 −1 0 8 0 −3 0
0 1 1 2 0 8 1 0
, . → , .
0 0 0 6 0 0 0 8
0 0 0 0 0 0 0 0

The pivot positions (of both matrices) are the (1,1), (2,2), (3,4) entries. The pivot columns (of both matrices)
are 1,2,4.

0 1 2 1 3
Exercise: Find the pivot positions and pivot columns of 1 2 3 2 1
2 1 0 3 2

Solution:

0 1 2 1 3 1 2 3 2 1 1 2 3 2 1 1 2 3 2 1
1 2 3 2 1 → 0 1 2 1 3 → 0 1 2 1 3 → 0 1 2 1 3 (REF)
2 1 0 3 2 2 1 0 3 2 0 −3 −6 −1 0 0 0 0 2 9
We can reduce further to get the RREF:

1 2 3 2 1 1 2 3 2 1 1 2 3 0 −8 1 0 −1 0 −5
0 1 2 1 3 → 0 1 2 1 3 → 0 1 2 0 −3/2 → 0 1 2 0 −3/2 (RREF)
0 0 0 2 9 0 0 0 1 9/2 0 0 0 1 9/2 0 0 0 1 9/2

One can read from the REF or RREF matrix that the pivot positions are (1,1), (2,2), (3,4), and the pivot
columns are 1,2,4.
3) A Systematic Way to solve system of linear equations
To determine whether a system is consistent or not, look at the RREF of augmented matrix B0′|D′E:

Theorem:

A system of linear equations is consistent (i.e. you can solve the system) ⇔ the last column of the augmented
matrix is not a pivot column.

(i.e. if a system has solution(s), then the last column of its augmented matrix is not pivot. Conversely, if a
system has no solution, then the last column of its augmented matrix is a pivot.)

Suppose the system is consistent. Then look at the coefficient matrix 0.

Definition

Let 0 be a coefficient matrix corresponding to a system of equations.

The variable corresponding to a pivot column of 0 is a basic variable.

The variable corresponding to a free column of 0 is a free variable.

For example:

1 2 −1 0 | 5 8 0 −3 0 | −5/3
0 1 1 2 | 3 0 8 1 0 | 10/3
, . → , .
0 0 0 6 | −1 0 0 0 8 | −1/6
0 0 0 0 | 0 0 0 0 0 | 0

The 4 × 4 matrix on the left is the coefficient matrix 0. It has pivot columns 1,2,4; and free column 3. So
( , ( , () are basic variables, (% is a free variable.

We can always assign a free parameter to a free variable, e.g. (% = H to be any number. Then the solutions of
the system are

( = −5/3  3H
( = 10/3 − H
' , where H is free
(% = H
() = −1/6

Finally, we give the definition of rank of any matrix, which will be a main theme of the course:

Definition:

The rank of a matrix 0 is the number of pivots, or the number of pivot columns.

As above, we relate the notion of rank to solving system of linear equations.

Theorem:

A linear system with coefficient matrix 0 and augmented matrix B0|DE is consistent (i.e. have solution(s)) iff
 PQ(0) = PQ(B0|DE)

Suppose B0|RE is a consistent system with P variables (i.e. 0 has P columns). Then the number of free
parameters of the solutions is equal to P − PQ(0).

For instance,

1 2 −1 0 | 5 8 0 −3 0 | −5/3
0 1 1 2 | 3 0 8 1 0 | 10/3
, . → , .
0 0 0 6 | −1 0 0 0 8 | −1/6
0 0 0 0 | 0 0 0 0 0 | 0

has rank 3. There is only (4 − 3) = 1 free parameter, namely ((% = H)

As a conclusion – to solve a system of linear equations:

Write the system as augmented matrix B0 |DE.

Use EROs to reduce B0 |DE into RREF B0S |D′E (as a bonus, 0′ is also the RREF of 0)
a)

Locate the pivot columns of B0S |D′E

b)

if D′ is a pivot column, the system is inconsistent; if not, the system is consistent.

c)

If consistent, locate the free columns of 0′.

d)
e)
f) If there is a free column, then infinitely many solutions; if not, unique solution.

Exercises:

( − 2(2 − (3  3(4 = 0
Solve the following system of linear equations:

( − 3(2 − 5(3 = 0
T (  ( = 3 , U −2(1  4(2  5(3 − 5(4 = 3
2 3
3(1 − 6(2 − 6(3  8(4 = 2

1 −3 −5| 0 1 0 −2| 9
V W → V W (3;  ; )
0 1 1 | 3 0 1 1 | 3
(a)

(the pivot positions are colored in BLUE). So (% = H is free and

( = 2H  9, ( = −H  3, (% = H

where H ∈ Y is free.

1 −2 −1 3 | 0 1 −2 −1 3 | 0
Z−2 4 5 −5 | 3[ → Z0 0 3 1 | 3[ (2;  ; , −3;  ;% )
3 −6 −6 8 | 2 0 0 −3 −1 | 2
(b)

1 −2 −1 3 | 0
→ Z0 0 3 1 | 3[ (;  ;% )
0 0 0 0 | 5

since the last column of the augmented matrix is pivot, the system is inconsistent
4) Vectors
As promised in the introduction last week, we want to move our attention back to studying vectors, and will
relate it to solving linear equations.

Here is the geometric picture of a vector:

The vector \ carries the information of (a) length and direction.

Therefore, two vectors are (b) equal if they have the same length and points to the same direction -

^^^^^_ and ^^^^^_

The vectors "] `a are therefore equal.

Here are some operations on vectors. We can (c) add vectors using the head-to-tail rule:

We can (d) multiply a vector with a number, so that we can elongate, shrink the vector, or move it into the
opposite direction.
In order to do mathematics, we need to use numbers to describe vectors. Here is one way to do it, using
coordinates:

9
The coordinate point (9, b, 7) on the cde-space is now treated as a position vector - \ = Zb[. You can
7
even treat \ as a (3 × 1)-matrix!

Denoting vectors by numbers, we can describe the pictorial properties (a) – (d) mathematically:

(a) Length of \ = √2  3  5

2 
(b) Equality of vectors: Z3[ = g h ijj  = 2,  = 3, = 5, i.e. all entries agree.
5
 k k
(c) Addition of vectors: g h  Z [ = Z   l [
 l
j j
2 2
(d) Scalar multiplication:  ⋅ Z3[ = Z3[
5 5
0
(e) Zero vector: < = Z0[
0
2 1 2×1
WARNING: You cannot multiply two vectors like Z3[ × Z4[ ≠ Z3 × 4[! It has no geometric meaning at all.
5 7 5×7
Notions (a) – (e) above can be extended beyond the (pq-space: We introduce a space with P entries as follows:

Notation: The collection of all (real) vectors of size P:



Ys = {u ⋮ w |  ,  … , v ∈ Y}


v

e.g. when P = 3, Yb is just the usual (pq-space, and when P = 2, Y9 is just the usual (p-plane, and Y is just
a real line.
The new space Ys has the same addition and scalar multiplication rule as the (pq-space Yb . And there are NO
2
relations between vectors of different sizes. So Z3[  (1) makes no sense at all.
5

Here are the rules for vectors in Yv : For any \, y, z in Yv and any numbers c and d (in Y):

1. u + v = v + u
2. (u + v) + w = u + (v + w)
3. u + 0 = 0 + u = u
4. u + (-u) = (-u) + u = 0
5. c(u + v) = cu + cv
6. (c + d)u = cu + du
7. c(du) = (cd)u
8. 1u = u

5) Linear Combination and Span

Definition:

Let { = {\8 , \9, … , \| } be a collection of vectors in Yv . Another vector y ∈ Yv is a linear combination (l.c.) of
{ if

y =  \8   \9  ⋯  } \|

for some scalars  ,  , … , } ∈ Y.

The linear span of ~, {P({), is the collection of all vectors of the form  \8   \9  ⋯  } \| , i.e.

{P({) ≔ {  \8   \9  ⋯  } \| |  ,  , … , } ∈ Y}

So {P({) contains all possible linear combinations of {.

Remarks:

1) The word span in the above context is a noun. Later on, we will give a definition of span as a verb.

2) To understand the definitions of l.c. and span, let’s look at the pictures in Y9 and Yb . The idea is, { specifies
the directions you can walk along -

(a) If you can reach the point y ∈ Yv by only moving along the directions specified by {, then y is a l.c. of {.
(b) the span of { are all the positions { can reach:
Exercise:

For the { = {\8 , … , \| } specified below, draw {P {, and determine whether y is a l.c. of { (or equivalently,
whether y ∈ {P {).

If so, express y in the form of y =  \8  ⋯  } \| as in the definition.

If not, explain why you cannot express y in the form of y =  \8  ⋯  } \|.

0 1
1 1 4
(a) { = { ‚ ,  ‚}, y =  ‚ in Y9 ; (b) { = {Z0[}, y = Z0[ in Yb
1 −1 0
0 0