Linear Algebra

Department of Mathematics
Indian Institute of Technology Guwahati

January – May 2019

MA 102 (RA, RKS, MGPP, KVK)

1 / 28
System of Linear Equations

Topics:
Matrices
Gaussian elimination
Row echelon form (ref)
Gauss-Jordan elimination and reduced row echelon form (rref)
Rank of a matrix

2 / 28
Matrices
Definition: A matrix is an array of numbers called entries or
elements of the matrix. The size of a matrix A is a description of
the number of rows and columns of the matrix A. An m × n
matrix A has m rows and n columns and is of the form
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A= . ..  .
 
. ..
 . . ··· . 
am1 am2 · · · amn

3 / 28
Matrices
Definition: A matrix is an array of numbers called entries or
elements of the matrix. The size of a matrix A is a description of
the number of rows and columns of the matrix A. An m × n
matrix A has m rows and n columns and is of the form
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A= . ..  .
 
. ..
 . . ··· . 
am1 am2 · · · amn

Let aj := [a1j , . . . , amj ]> be the

 j-th column of Afor j = 1 : n.
Then we represent A as A = a1 a2 · · · an .

3 / 28
Matrices
Definition: A matrix is an array of numbers called entries or
elements of the matrix. The size of a matrix A is a description of
the number of rows and columns of the matrix A. An m × n
matrix A has m rows and n columns and is of the form
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A= . ..  .
 
. ..
 . . ··· . 
am1 am2 · · · amn

Let aj := [a1j , . . . , amj ]> be the  j-th column of Afor j = 1 : n.

Then we represent A as A = a1 a2 · · · an .
Let Ai := [ai1 , ai2 , . . . , a
in ] be the
 i-th row of A for i = 1 : m. Then
A1
we represent A as A =  ...  .
 

Am
3 / 28
Special matrices

Let A be an m × n matrix with (i, j)-th entry aij . Set

p := min(m, n). Then
aii for i = 1 : p are called the diagonal entries of A;
A is said to be a diagonal matrix if aij = 0 for all i 6= j;

4 / 28
Special matrices

Let A be an m × n matrix with (i, j)-th entry aij . Set

p := min(m, n). Then
aii for i = 1 : p are called the diagonal entries of A;
A is said to be a diagonal matrix if aij = 0 for all i 6= j;
A is said to be an upper triangular if aij = 0 for all i > j;
A is said to be a lower triangular if aij = 0 for all i < j;
A is said to be a square matrix if m = n.

4 / 28
Special matrices

Let A be an m × n matrix with (i, j)-th entry aij . Set

p := min(m, n). Then
aii for i = 1 : p are called the diagonal entries of A;
A is said to be a diagonal matrix if aij = 0 for all i 6= j;
A is said to be an upper triangular if aij = 0 for all i > j;
A is said to be a lower triangular if aij = 0 for all i < j;
A is said to be a square matrix if m = n.

Identity matrix: An n × n diagonal matrix with all diagonal entries

equal to 1 is called the identity matrix and is denoted by In or I .

Zero matrix: An m × n matrix with all entries 0 is called the zero

matrix and is denoted by Om×n or simply by O.

4 / 28
Special matrices

Let A be an m × n matrix with (i, j)-th entry aij . Set

p := min(m, n). Then
aii for i = 1 : p are called the diagonal entries of A;
A is said to be a diagonal matrix if aij = 0 for all i 6= j;
A is said to be an upper triangular if aij = 0 for all i > j;
A is said to be a lower triangular if aij = 0 for all i < j;
A is said to be a square matrix if m = n.

Identity matrix: An n × n diagonal matrix with all diagonal entries

equal to 1 is called the identity matrix and is denoted by In or I .

Zero matrix: An m × n matrix with all entries 0 is called the zero

matrix and is denoted by Om×n or simply by O.

4 / 28
Linear combination
Let u and v be vectors in Rn . Let α and β be scalars. Adding αu
and βv gives the linear combination αu + βv.
Example: Let u := [1, 1, −1]> , v := [2, 3, 4]> and w := [4, 5, 2]> .
Then w = 2u + v. Thus w is a linear combination of u and v.

5 / 28
Linear combination
Let u and v be vectors in Rn . Let α and β be scalars. Adding αu
and βv gives the linear combination αu + βv.
Example: Let u := [1, 1, −1]> , v := [2, 3, 4]> and w := [4, 5, 2]> .
Then w = 2u + v. Thus w is a linear combination of u and v.

Definition: Let v1 , . . . , vm be vectors in Rn and let α1 , . . . , αm be

scalars. Then the vector u := α1 v1 + · · · + αm vm is called a linear
combination of v1 , . . . , vm .

5 / 28
Linear combination
Let u and v be vectors in Rn . Let α and β be scalars. Adding αu
and βv gives the linear combination αu + βv.
Example: Let u := [1, 1, −1]> , v := [2, 3, 4]> and w := [4, 5, 2]> .
Then w = 2u + v. Thus w is a linear combination of u and v.

Definition: Let v1 , . . . , vm be vectors in Rn and let α1 , . . . , αm be

scalars. Then the vector u := α1 v1 + · · · + αm vm is called a linear
combination of v1 , . . . , vm .

Problem: Let a1 , . . . , an and b be vectors in Rm . Find scalars

x1 , . . . , xn , if exist, such that x1 a1 + · · · + xn an = b.
Example: Vector equation
       
1 0 0 1
x1  −1  + x2  1  + x3  0  =  3  .
0 −1 1 5

5 / 28
Matrix times vector
 combination x1 a1 + · · · + xn an using a
We rewrite the linear
matrix. Set A := a1 · · · an and x := [x1 , . . . , xn ]> . We
define the matrix A times the vector x to be the same as the
combination x1 a1 + · · · + xn an .

6 / 28
Matrix times vector
 combination x1 a1 + · · · + xn an using a
We rewrite the linear
matrix. Set A := a1 · · · an and x := [x1 , . . . , xn ]> . We
define the matrix A times the vector x to be the same as the
combination x1 a1 + · · · + xn an .
Definition: Matrix-vector multiplication
 
x1
Ax = a1 · · · an  ...  = x1 a1 + · · · + xn an .
  

xn

The matrix A acts on the vector x and the result Ax is a linear

combination of the columns of A.

6 / 28
Matrix times vector
 combination x1 a1 + · · · + xn an using a
We rewrite the linear
matrix. Set A := a1 · · · an and x := [x1 , . . . , xn ]> . We
define the matrix A times the vector x to be the same as the
combination x1 a1 + · · · + xn an .
Definition: Matrix-vector multiplication
 
x1
Ax = a1 · · · an  ...  = x1 a1 + · · · + xn an .
  

xn

The matrix A acts on the vector x and the result Ax is a linear

combination of the columns of A.
Example: Compact notation for vector equation
          
1 0 0 x1 1 0 0 1
 −1 1 0   x2  = x1  −1 +x2  1 +x3  0  =  3  .
0 −1 1 x3 0 −1 1 5

6 / 28
Matrix times vector
 
A row vector ai1 · · · ain is a 1 × n matrix. Therefore
 
x1
  . 
ai1 · · · ain  ..  = ai1 x1 + · · · + ain xn .
xn

7 / 28
Matrix times vector
 
A row vector ai1 · · · ain is a 1 × n matrix. Therefore
 
x1
  . 
ai1 · · · ain  ..  = ai1 x1 + · · · + ain xn .
xn

Example: Matrix-vector multiplication in two ways

       
1 0 0 x1 1 0 0
 1 1 0   x2  = x1  1  + x2  1  + x3  0 
0 1 1 x3 0 1 1
 
   1 0 0 x 
x1
   
=  x1 + x2  =  1 1 0 x 
x2 + x2  
0 1 1 x

7 / 28
Matrix-vector multiplication
More
 generally      
a11 · · · a1n x1 a11 a1n
 .. ..   ..  .   . 
 = x1  ..  + · · · + xn  .. 

 . ··· .  .
a · · · amn xn a a
m1    m1   mn
a11 x1 + · · · + a1n xn a11 · · · a1n x
= .. ..
= .
   
.  . 
am1 x1 + · · · + amn xn am1 · · · amn x

8 / 28
Matrix-vector multiplication
More
 generally      
a11 · · · a1n x1 a11 a1n
 .. ..   ..  ..   ..
 = x1  .  + · · · + xn  .
 
 . ··· .  . 
a · · · amn xn a a
m1    m1   mn
a11 x1 + · · · + a1n xn a11 · · · a1n x
= .. ..
= .
   
.  . 
am1 x1 + · · · + amn xn am1 · · · amn x
 
A1
   ..
Now represent A := a1 · · · an by its rows: A =  . .


Am
Then we have
   
a11 x1 + · · · + a1n xn A1 x
Ax = x1 a1 + · · · + xn an =  ..   .. 
 =  . .

.
am1 x1 + · · · + amn xn Am x

8 / 28
Linear equations
Definition: A linear equation in the n variables x1 , . . . , xn is an
equation of the form

a1 x1 + · · · + an xn = b (1)

where the coefficients a1 , . . . , an and the constant term b are

constants.

9 / 28
Linear equations
Definition: A linear equation in the n variables x1 , . . . , xn is an
equation of the form

a1 x1 + · · · + an xn = b (1)

where the coefficients a1 , . . . , an and the constant term b are

constants.
The equation (1) can be rewritten as
 
x1
a1 · · · an  ...  = b.
 
(2)


xn

9 / 28
Linear equations
Definition: A linear equation in the n variables x1 , . . . , xn is an
equation of the form

a1 x1 + · · · + an xn = b (1)

where the coefficients a1 , . . . , an and the constant term b are

constants.
The equation (1) can be rewritten as
 
x1
a1 · · · an  ...  = b.
 
(2)


xn

A vector s := [s1 , . . . , sn ]> is said to be a solution of the linear

equation (1) if it satisfies the equation (2).

9 / 28
System of linear equations
An m × n system of linear equations is a set of m equations in the
n variables x1 , . . . , xn of the form

a11 x1 + · · · + a1n xn = b1
.. .. (3)
. .
am1 x1 + · · · + amn xn = bm

where aij and bi are constants for i = 1 : m and j = 1 : n.

10 / 28
System of linear equations
An m × n system of linear equations is a set of m equations in the
n variables x1 , . . . , xn of the form

a11 x1 + · · · + a1n xn = b1
.. .. (3)
. .
am1 x1 + · · · + amn xn = bm

where aij and bi are constants for i = 1 : m and j = 1 : n.

The system of equations in (3) can be rewritten as matrix equation
    
a11 · · · a1n x1 b1
Ax = b  ... .. ..   ..  =  ..  (4)

. .  .   . 
am1 · · · amn xn bm
where A is called the coefficient matrix and b is called the constant
vector.

10 / 28
System of linear equations
An m × n system of linear equations is a set of m equations in the
n variables x1 , . . . , xn of the form

a11 x1 + · · · + a1n xn = b1
.. .. (3)
. .
am1 x1 + · · · + amn xn = bm

where aij and bi are constants for i = 1 : m and j = 1 : n.

The system of equations in (3) can be rewritten as matrix equation
    
a11 · · · a1n x1 b1
Ax = b  ... .. ..   ..  =  ..  (4)

. .  .   . 
am1 · · · amn xn bm
where A is called the coefficient matrix and b is called the constant
vector. A vector s := [s1 , . . . , sn ]> is said to be a solution of (3) if
it satisfies (4).
10 / 28
System of linear equations
An m × n system of linear equations is a set of m equations in the
n variables x1 , . . . , xn of the form

a11 x1 + · · · + a1n xn = b1
.. .. (3)
. .
am1 x1 + · · · + amn xn = bm

where aij and bi are constants for i = 1 : m and j = 1 : n.

The system of equations in (3) can be rewritten as matrix equation
    
a11 · · · a1n x1 b1
Ax = b  ... .. ..   ..  =  ..  (4)

. .  .   . 
am1 · · · amn xn bm
where A is called the coefficient matrix and b is called the constant
vector. A vector s := [s1 , . . . , sn ]> is said to be a solution of (3) if
it satisfies (4). We refer to Ax = b as a linear system.
10 / 28
Augmented system
The system of equations in (3) can also rewritten as a vector
equation
     
a11 a1n b1
x1  ...  + · · · +  ...  xn −  ...  = 0 (5)
     

am1 amn bm
which shows that solving the system amounts to expressing b as a
linear combination of the columns of A. Rewriting (5) as a matrix
equation yields the augmented system

11 / 28
Augmented system
The system of equations in (3) can also rewritten as a vector
equation
     
a11 a1n b1
x1  ...  + · · · +  ...  xn −  ...  = 0 (5)
     

am1 amn bm
which shows that solving the system amounts to expressing b as a
linear combination of the columns of A. Rewriting (5) as a matrix
equation yields the augmented system
   
  x1 0
a11 · · · a1n b1  .   . 
 .. .. ..   ..   .. 
 . . .  = 
 xn   0 
am1 · · · amn bm
| {z } −1 0
augmented matrix
 
where A b is called the augmented matrix.
11 / 28
Augmented system
Note that x := [x1 , . . . , xn ]> is a solution of
a11 x1 + · · · + a1n xn = b1
.. ..
. .
am1 x1 + · · · + amn xn = bm
if and only if x satisfies the augmented system
   
  x1 0
a11 · · · a1n b1  ..   .. 
 .. .. .. .  . 
=
 
 . . .  
 xn   0 
am1 · · · amn bm
−1 0

12 / 28
Augmented system
Note that x := [x1 , . . . , xn ]> is a solution of
a11 x1 + · · · + a1n xn = b1
.. ..
. .
am1 x1 + · · · + amn xn = bm
if and only if x satisfies the augmented system
   
  x1 0
a11 · · · a1n b1  ..   .. 
 .. .. .. .  . 
=
 
 . . .  
 xn   0 
am1 · · · amn bm
−1 0

Strategy: We solvethe augmented

 system by reducing the
augmented matrix A b to row echelon form.

12 / 28
Augmented system
Note that x := [x1 , . . . , xn ]> is a solution of
a11 x1 + · · · + a1n xn = b1
.. ..
. .
am1 x1 + · · · + amn xn = bm
if and only if x satisfies the augmented system
   
  x1 0
a11 · · · a1n b1  ..   .. 
 .. .. .. .  . 
=
 
 . . .  
 xn   0 
am1 · · · amn bm
−1 0

Strategy: We solvethe augmented

 system by reducing the
augmented matrix A b to row echelon form.
Remark: There are two matrices associated with a linear system
Ax = b, namely, the coefficient matrix A and the augmented
matrix A b .
12 / 28
Equivalent systems
Definition: Two linear systems Ax = b and Uy = d are said to be
equivalent if they have the same solution, where the matrices A
and U have the same size.

13 / 28
Equivalent systems
Definition: Two linear systems Ax = b and Uy = d are said to be
equivalent if they have the same solution, where the matrices A
and U have the same size.
Strategy: Transform a given linear system to an equivalent linear
system that is easier to solve.

13 / 28
Equivalent systems
Definition: Two linear systems Ax = b and Uy = d are said to be
equivalent if they have the same solution, where the matrices A
and U have the same size.
Strategy: Transform a given linear system to an equivalent linear
system that is easier to solve.
Example: Gaussian (forward) elimination
 
x −y −z =2 1 −1 −1 2
3x − 3y + 2z = 16 ⇐⇒  3 −3 2 16 
2x − y + z = 9 2 −1 1 9
| {z }
augmented matrix

13 / 28
Equivalent systems
Definition: Two linear systems Ax = b and Uy = d are said to be
equivalent if they have the same solution, where the matrices A
and U have the same size.
Strategy: Transform a given linear system to an equivalent linear
system that is easier to solve.
Example: Gaussian (forward) elimination
 
x −y −z =2 1 −1 −1 2
3x − 3y + 2z = 16 ⇐⇒  3 −3 2 16 
2x − y + z = 9 2 −1 1 9
| {z }
augmented matrix

Use first equation to eliminating x from 2nd and 3rd equation

 
x −y −z =2 1 −1 −1 2
5z = 10 ⇐⇒  0 0 5 10  .
y + 3z = 5 0 1 3 5
13 / 28
Example (cont.)

Now interchange 2nd and 3rd equations

 
x −y −z =2 1 −1 −1 2
y + 3z = 5 ⇐⇒  0 1 3 5 .
5z = 10 0 0 5 10

Solving equivalent upper triangular system (back substitution), we

have the solution [x, y , z]> = [3, −1, 2]> .

14 / 28
Example (cont.)

Now interchange 2nd and 3rd equations

 
x −y −z =2 1 −1 −1 2
y + 3z = 5 ⇐⇒  0 1 3 5 .
5z = 10 0 0 5 10

Solving equivalent upper triangular system (back substitution), we

have the solution [x, y , z]> = [3, −1, 2]> .
Observation: Elementary operations (scalar multiplication,
addition, interchange) on equations correspond to elementary row
operations on the augmented matrix.

14 / 28
Row echelon form (ref)

Pivot: First nonzero entry in a row is called a pivot (leading entry).

Pivot column: A column containing a pivot (leading entry) is

called a pivot column.

15 / 28
Row echelon form (ref)

Pivot: First nonzero entry in a row is called a pivot (leading entry).

Pivot column: A column containing a pivot (leading entry) is

called a pivot column.

Definition: An m × n matrix A is in row echelon form provided:

15 / 28
Row echelon form (ref)

Pivot: First nonzero entry in a row is called a pivot (leading entry).

Pivot column: A column containing a pivot (leading entry) is

called a pivot column.

Definition: An m × n matrix A is in row echelon form provided:

All zero rows appear at the bottom.

15 / 28
Row echelon form (ref)

Pivot: First nonzero entry in a row is called a pivot (leading entry).

Pivot column: A column containing a pivot (leading entry) is

called a pivot column.

Definition: An m × n matrix A is in row echelon form provided:

All zero rows appear at the bottom.
The pivot (leading entry) in a row is always to the right of the
pivot of the row above it.

Notation: ref(A) = row echelon form of A.

Remark: Row echelon form of a matrix is not unique.

15 / 28
Row echelon form (ref)

Pivot: First nonzero entry in a row is called a pivot (leading entry).

Pivot column: A column containing a pivot (leading entry) is

called a pivot column.

Definition: An m × n matrix A is in row echelon form provided:

All zero rows appear at the bottom.
The pivot (leading entry) in a row is always to the right of the
pivot of the row above it.

Notation: ref(A) = row echelon form of A.

Remark: Row echelon form of a matrix is not unique.

Convention: We refer to row echelon form simply by echelon form.

15 / 28
Examples
Matrices in echelon form:
p ∗ ∗ ∗ ∗ ∗ ∗ ∗
 
0 0 p ∗ ∗ ∗ ∗ ∗ 
 p ∗ ∗ ∗ ∗
  
 p ∗ ∗
0 0 0 p ∗ ∗ ∗ ∗ 

 , 0 0 p ∗ ∗ ,  0 p ∗  .
0 0 0 0 0 0 p ∗ 0 0 0 0 0

0 0 0 0 p
0 0 0 0 0 0
0 0 0 0 0 0 0 0

Here p stands for pivot and ∗ stands for arbitrary (zero or

nonzero) entry.

Matrices not in echelon form:

 
2 3 4 2    
0 0 1 0 0 1 0 0 0 2
0 0 3 1 , 0 1 2 , 0 2 3 .
     
0 0 1 0 0 0
0 0 0 2
16 / 28
Elementary row operations
Multiply a row by nonzero scalar: rowi (A) −→ α rowi (A).
Add a row with another row: rowi (A) + rowj (A) −→ rowj (A).
Interchange rows: rowi (A) ↔ rowj (A)

Exercise: Describe the inverse operations.

17 / 28
Elementary row operations
Multiply a row by nonzero scalar: rowi (A) −→ α rowi (A).
Add a row with another row: rowi (A) + rowj (A) −→ rowj (A).
Interchange rows: rowi (A) ↔ rowj (A)

Exercise: Describe the inverse operations.

The process of applying elementary row operations to reduce a
matrix to row echelon form is called row reduction.

Definition: Matrices A and B are said to be row equivalent if there

is a sequence of elementary row operations that converts A into B.

17 / 28
Elementary row operations
Multiply a row by nonzero scalar: rowi (A) −→ α rowi (A).
Add a row with another row: rowi (A) + rowj (A) −→ rowj (A).
Interchange rows: rowi (A) ↔ rowj (A)

Exercise: Describe the inverse operations.

The process of applying elementary row operations to reduce a
matrix to row echelon form is called row reduction.

Definition: Matrices A and B are said to be row equivalent if there

is a sequence of elementary row operations that converts A into B.

Example: The augmented matrices (from the previous example)

   
1 −1 −1 2 1 −1 −1 2
 3 −3 2 16  and  0 1 3 5 
2 −1 1 9 0 0 5 10
are row equivalent.
17 / 28
Reduction to echelon form
Gaussian elimination (GE): Use elementary row operations to
reduce a matrix to upper triangular form by introducing zeros
below the diagonals. Here is an algorithm (forward GE).

18 / 28
Reduction to echelon form
Gaussian elimination (GE): Use elementary row operations to
reduce a matrix to upper triangular form by introducing zeros
below the diagonals. Here is an algorithm (forward GE).
1 Search the first column of A to find a nonzero entry and

interchange rows to bring the nonzero entry to (1, 1) position.

Use (1, 1) entry as the pivot and perform elementary row
operations to introduce zeros in the first column below the
(1, 1) entry.

18 / 28
Reduction to echelon form
Gaussian elimination (GE): Use elementary row operations to
reduce a matrix to upper triangular form by introducing zeros
below the diagonals. Here is an algorithm (forward GE).
1 Search the first column of A to find a nonzero entry and

interchange rows to bring the nonzero entry to (1, 1) position.

Use (1, 1) entry as the pivot and perform elementary row
operations to introduce zeros in the first column below the
(1, 1) entry. The reduced matrix would be of the form
 
p11 p12 · · · p1n
 0 p22 · · · p2n 
 
 .. .. .. .. 
 . . . . 
0 pm2 · · · pmn

18 / 28
Reduction to echelon form
Gaussian elimination (GE): Use elementary row operations to
reduce a matrix to upper triangular form by introducing zeros
below the diagonals. Here is an algorithm (forward GE).
1 Search the first column of A to find a nonzero entry and

interchange rows to bring the nonzero entry to (1, 1) position.

Use (1, 1) entry as the pivot and perform elementary row
operations to introduce zeros in the first column below the
(1, 1) entry. The reduced matrix would be of the form
 
p11 p12 · · · p1n
 0 p22 · · · p2n 
 
 .. .. .. .. 
 . . . . 
0 pm2 · · · pmn

2 Repeat step 1 to the (m − 1) × (n − 1) submatrix until the

matrix is reduced to upper triangular form.

18 / 28
Reduction to echelon form
Gaussian elimination (GE): Use elementary row operations to
reduce a matrix to upper triangular form by introducing zeros
below the diagonals. Here is an algorithm (forward GE).
1 Search the first column of A to find a nonzero entry and

interchange rows to bring the nonzero entry to (1, 1) position.

Use (1, 1) entry as the pivot and perform elementary row
operations to introduce zeros in the first column below the
(1, 1) entry. The reduced matrix would be of the form
 
p11 p12 · · · p1n
 0 p22 · · · p2n 
 
 .. .. .. .. 
 . . . . 
0 pm2 · · · pmn

2 Repeat step 1 to the (m − 1) × (n − 1) submatrix until the

matrix is reduced to upper triangular form.
Echelon form: Use further row operations, if necessary, to reduce
the upper triangular matrix to echelon form.
18 / 28
Reduction to echelon form
Gaussian elimination (GE): Use elementary row operations to
reduce a matrix to upper triangular form by introducing zeros
below the diagonals. Here is an algorithm (forward GE).
1 Search the first column of A to find a nonzero entry and

interchange rows to bring the nonzero entry to (1, 1) position.

Use (1, 1) entry as the pivot and perform elementary row
operations to introduce zeros in the first column below the
(1, 1) entry. The reduced matrix would be of the form
 
p11 p12 · · · p1n
 0 p22 · · · p2n 
 
 .. .. .. .. 
 . . . . 
0 pm2 · · · pmn

2 Repeat step 1 to the (m − 1) × (n − 1) submatrix until the

matrix is reduced to upper triangular form.
Echelon form: Use further row operations, if necessary, to reduce
the upper triangular matrix to echelon form.
18 / 28
Example 1

    
2 1 1 x1 2
Square system: −1 2 0
  x2 = 1 
 
1 −1 2 x3 −2

Forward elimination (Forward GE) −→ Upper triangular form:

   
2 1 1 2 1 −1 2 −2
 −1 2 0 1  R3 ↔ R1  −1 2 0 1 
−−−−−−−−→
1 −1 2 −2 2 1 1 2

19 / 28
Example 1

    
2 1 1 x1 2
Square system: −1 2 0
  x2 = 1 
 
1 −1 2 x3 −2

Forward elimination (Forward GE) −→ Upper triangular form:

   
2 1 1 2 1 −1 2 −2
 −1 2 0 1  R3 ↔ R1  −1 2 0 1 
−−−−−−−−→
1 −1 2 −2 2 1 1 2
   
1 −1 2 −2 1 −1 2 −2
R3 = R3 − 2R1  0 1 2 −1  →  0 1 2 −1 
R2 =R1 +R2
−−−−−−−−−−−−−→ 0 3 −3 6 0 0 −9 9

Back substitution: x3 = −1, x2 = 1 and x1 = 1.

19 / 28
Example 2
    
0 1 5 x1 −4
Square System: 1 4 3 x2  = −2
2 7 1 x3 −1
Forward GE:

     
0 1 5 −4 1 4 3 −2 1 4 3 −2
 1 4 3 −2  →  0 1 5 −4  →  0 1 5 −4 
2 7 1 −1 2 7 1 −1 0 −1 −5 3

20 / 28
Example 2
    
0 1 5 x1 −4
Square System: 1 4 3 x2  = −2
2 7 1 x3 −1
Forward GE:

     
0 1 5 −4 1 4 3 −2 1 4 3 −2
 1 4 3 −2  →  0 1 5 −4  →  0 1 5 −4 
2 7 1 −1 2 7 1 −1 0 −1 −5 3

 
1 4 3 −2
→  0 1 5 −4  =⇒ No solution
0 0 0 −1

20 / 28
Example 3

 
  x1  
2 1 1   2
Nonsquare system: x2 =
1 −1 2 −2
x3

Forward GE:

     
2 1 1 2 1 −1 2 −2 1 −1 2 −2
→ →
1 −1 2 −2 2 1 1 2 0 3 −3 6
 
1 −1 2 −2
→
0 1 −1 2

Back substitution: x3 = t, x2 = 2 + t and x1 = −t for t ∈ R.

21 / 28
Echelon form and consistency
Definition: A linear system Ax = b is said to be consistent if it has
a solution. A system is inconsistent if it is NOT consistent.

22 / 28
Echelon form and consistency
Definition: A linear system Ax = b is said to be consistent if it has
a solution. A system is inconsistent if it is NOT consistent.
Theorem: An m × n system Ax = b is consistent ⇐⇒ the last
column of ref([A | b]) is not a pivot column.

Proof: If the last column is a pivot column then all the entries in
the pivot row are zero except the last entry. 

22 / 28
Echelon form and consistency
Definition: A linear system Ax = b is said to be consistent if it has
a solution. A system is inconsistent if it is NOT consistent.
Theorem: An m × n system Ax = b is consistent ⇐⇒ the last
column of ref([A | b]) is not a pivot column.

Proof: If the last column is a pivot column then all the entries in
the pivot row are zero except the last entry. 
Example: Consider the augmented matrix
     
0 1 5 −4 1 4 3 −2 1 4 3 −2
 1 4 3 −2  →  0 1 5 −4  →  0 1 5 −4 
2 7 1 −1 2 7 1 −1 0 −1 −5 3
 
1 4 3 −2
→  0 1 5 −4  = echelon form ⇒ inconsistent
0 0 0 −1

22 / 28
Reduced row echelon form (rref)
An m × n matrix A is in reduced row echelon form provided:

23 / 28
Reduced row echelon form (rref)
An m × n matrix A is in reduced row echelon form provided:
A is in row echelon form.
Each pivot (leading entry) in A is 1.
Pivot is the only nonzero entry in a pivot column.

Notation: rref(A) = reduced row echelon form of A.

23 / 28
Reduced row echelon form (rref)
An m × n matrix A is in reduced row echelon form provided:
A is in row echelon form.
Each pivot (leading entry) in A is 1.
Pivot is the only nonzero entry in a pivot column.

Notation: rref(A) = reduced row echelon form of A.

Matrices in echelon form:

1 ∗ ∗ ∗ ∗
 
0 0 0
0 0 1 0 ∗ ∗ 0 ∗    
  1 ∗ 0 ∗ ∗ 1 0 0
0 0
 0 1 ∗ ∗ 0 ∗
, 0 0 1 ∗ ∗ , 0 1 0 .
0 0 0 0 0 0 1 ∗

0 0
 0 0 0 0 0 0 0 1
0 0 0 0 0 0
0 0 0 0 0 0 0 0

23 / 28
Gauss-Jordan elimination and reduction to rref
Step 1: Forward GE : m × n matrix A −→ ref(A).
Step 2: Backward GE: ref(A) −→ rref(A).

24 / 28
Gauss-Jordan elimination and reduction to rref
Step 1: Forward GE : m × n matrix A −→ ref(A).
Step 2: Backward GE: ref(A) −→ rref(A).
Backward GE: Start GE from the bottom nonzero row to the top
rows and use the pivot in each pivot column for elimination until
the matrix is reduced to rref(A).

24 / 28
Gauss-Jordan elimination and reduction to rref
Step 1: Forward GE : m × n matrix A −→ ref(A).
Step 2: Backward GE: ref(A) −→ rref(A).
Backward GE: Start GE from the bottom nonzero row to the top
rows and use the pivot in each pivot column for elimination until
the matrix is reduced to rref(A).
Example (backward GE):
     
p ∗ ∗ ∗ ∗ p ∗ 0 ∗ ∗ 1 ∗ 0 ∗ ∗
0 0 p ∗ ∗ → 0 0 1 ∗ ∗ → 0 0 1 ∗ ∗
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

24 / 28
Gauss-Jordan elimination and reduction to rref
Step 1: Forward GE : m × n matrix A −→ ref(A).
Step 2: Backward GE: ref(A) −→ rref(A).
Backward GE: Start GE from the bottom nonzero row to the top
rows and use the pivot in each pivot column for elimination until
the matrix is reduced to rref(A).
Example (backward GE):
     
p ∗ ∗ ∗ ∗ p ∗ 0 ∗ ∗ 1 ∗ 0 ∗ ∗
0 0 p ∗ ∗ → 0 0 1 ∗ ∗ → 0 0 1 ∗ ∗
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Gauss-Jordan elimination = Forward GE followed by backward GE.

Gauss-Jordan elimination: m × n matrix A −→ rref(A).
Theorem: Reduced row echelon form of an m × n matrix A is
unique. (to be proved later).
24 / 28
Example: Gauss-Jordan elimination
Forward GE: A → ref(A)
   
0 3 −6 6 4 −5 1 −3 4 −3 2 5
3 −7 8 −5 8 9  → 0 1 −2 2 1 −3
3 −9 12 −9 6 15 0 0 0 0 1 4

25 / 28
Example: Gauss-Jordan elimination
Forward GE: A → ref(A)
   
0 3 −6 6 4 −5 1 −3 4 −3 2 5
3 −7 8 −5 8 9  → 0 1 −2 2 1 −3
3 −9 12 −9 6 15 0 0 0 0 1 4

Backward GE: ref(A) → rref(A)

   
1 −3 4 −3 2 5 1 0 −2 3 5 −4
0 1 −2 2 1 −3 → 0 1 −2 2 1 −3
0 0 0 0 1 4 0 0 0 0 1 4
 
1 0 −2 3 0 −24
→ 0 1 −2 2 0 −7  .
0 0 0 0 1 4

25 / 28
Rank of a matrix

Definition: The rank of an m × n matrix A, denoted by rank(A), is

the number of pivots in rref(A).

26 / 28
Rank of a matrix

Definition: The rank of an m × n matrix A, denoted by rank(A), is

the number of pivots in rref(A).

Example:
   
1 0 3 1 0 3
A := 0 1 −2 → 0 1 −2 ⇒ rank(A) = 2.
2 1 4 0 0 0

26 / 28
Rank of a matrix

Definition: The rank of an m × n matrix A, denoted by rank(A), is

the number of pivots in rref(A).

Example:
   
1 0 3 1 0 3
A := 0 1 −2 → 0 1 −2 ⇒ rank(A) = 2.
2 1 4 0 0 0

Fact:
rank(A) = number of pivot columns in rref(A) = number of
nonzero rows in rref(A).
rank(A) = number of pivot columns in ref(A) = number of
nonzero rows in ref(A).

26 / 28
Leading and free variable:
Free variable: A variable in a system Ax = b is called a free
variable if the system has a solution for every value of that variable.

27 / 28
Leading and free variable:
Free variable: A variable in a system Ax = b is called a free
variable if the system has a solution for every value of that variable.

Example:
   
1 0 3 −1 1 0 3 −1 x1 = −1 − 3x3
 0 1 −2 3  →  0 1 −2 3  ⇒ x2 = 3 + 2x3
2 1 4 1 0 0 0 0 x3 : free

27 / 28
Leading and free variable:
Free variable: A variable in a system Ax = b is called a free
variable if the system has a solution for every value of that variable.

Example:
   
1 0 3 −1 1 0 3 −1 x1 = −1 − 3x3
 0 1 −2 3  →  0 1 −2 3  ⇒ x2 = 3 + 2x3
2 1 4 1 0 0 0 0 x3 : free

Leading variables: Let [A | b] −→ rref([A | b]) =: [R | d]. Then

the variables corresponding to the pivot columns of R are called
leading variable.

Theorem: The number of free variables in a consistent m × n

system Ax = b is given by n − rank(A).

27 / 28
Leading and free variable:
Free variable: A variable in a system Ax = b is called a free
variable if the system has a solution for every value of that variable.

Example:
   
1 0 3 −1 1 0 3 −1 x1 = −1 − 3x3
 0 1 −2 3  →  0 1 −2 3  ⇒ x2 = 3 + 2x3
2 1 4 1 0 0 0 0 x3 : free

Leading variables: Let [A | b] −→ rref([A | b]) =: [R | d]. Then

the variables corresponding to the pivot columns of R are called
leading variable.

Theorem: The number of free variables in a consistent m × n

system Ax = b is given by n − rank(A).

Proof: # Free variables = # non-pivot columns = n − rank(A).

27 / 28
Rank and consistency
Fact: An m × n homogeneous system Ax = 0 has
infinitely many solutions if rank(A) < n,
unique (trivial) solution if rank(A) = n.

28 / 28
Rank and consistency
Fact: An m × n homogeneous system Ax = 0 has
infinitely many solutions if rank(A) < n,
unique (trivial) solution if rank(A) = n.

Fact: An m × n system Ax = b
is inconsistent if rank(A) 6= rank([A | b]).

28 / 28
Rank and consistency
Fact: An m × n homogeneous system Ax = 0 has
infinitely many solutions if rank(A) < n,
unique (trivial) solution if rank(A) = n.

Fact: An m × n system Ax = b
is inconsistent if rank(A) 6= rank([A | b]).
consistent if rank(A) = rank([A | b]).

28 / 28
Rank and consistency
Fact: An m × n homogeneous system Ax = 0 has
infinitely many solutions if rank(A) < n,
unique (trivial) solution if rank(A) = n.

Fact: An m × n system Ax = b
is inconsistent if rank(A) 6= rank([A | b]).
consistent if rank(A) = rank([A | b]).
has unique solution if rank(A) = rank([A | b]) = n.

28 / 28
Rank and consistency
Fact: An m × n homogeneous system Ax = 0 has
infinitely many solutions if rank(A) < n,
unique (trivial) solution if rank(A) = n.

Fact: An m × n system Ax = b
is inconsistent if rank(A) 6= rank([A | b]).
consistent if rank(A) = rank([A | b]).
has unique solution if rank(A) = rank([A | b]) = n.
infinitely many solutions if rank(A) = rank([A | b]) < n.

28 / 28
Rank and consistency
Fact: An m × n homogeneous system Ax = 0 has
infinitely many solutions if rank(A) < n,
unique (trivial) solution if rank(A) = n.

Fact: An m × n system Ax = b
is inconsistent if rank(A) 6= rank([A | b]).
consistent if rank(A) = rank([A | b]).
has unique solution if rank(A) = rank([A | b]) = n.
infinitely many solutions if rank(A) = rank([A | b]) < n.

   
1 2 1 1 2 1
→ ⇒ inconsistent if k 6= 1.
1 2 k 0 0 k −1

***
28 / 28