System of Linear Equations: Vietnamese-German University

System of linear equations
Tan Do
Vietnamese-German University
Lecture 5
Tan Do (VGU) Introduction to MATLAB Lecture 5 1 / 28

Content
In this lecture
Linear system: square system, underdetermined and overdetermined

system
Echelon form, reduced echelon form and Gaussian-Jordan elimination
method
Invertible matrix, how to nd the inverse of a matrix

Introduction
Linear equation
A linear equation of n variables x1 , x2 , . . . , xn is of the form
a1 x1 + a2 x2 + . . . + an xn = b,
where a1 , a2 , . . . , an and b are constants.

For example,
3x1 − x2 = 7
is a linear equation of 2 variable x1 and x2 ,
x1 − 2x2 + x3 = 0
is a linear equation of 3 variables, etc.

Introduction
System of linear equations
A system of linear equations (or linear system for short) consists of several linear
equations together.
Example
3x1 − x2 = 7
x1 + 2x2 = 10
is a system of linear equations of 2 unknowns,
x1 − 2x2 + x3 = 0
x1 + 2x2 − 8x3 = −15
−4x1 + 5x2 + 9x3 = 25
is a system of linear equations of 3 unknowns x1 , x2 and x3 , etc.

Introduction
A system of linear equations may have less equations than the number of variables. We call such
a system an underdetermined system.
Example
x1 − 5x2 + 8x3 = 1
10x1 + 23x2 − x3 = 12
is an underdetermined system.
A system of linear equations which has more equations than the number of variables is called an
overdetermined system.
Example
x1 + x2 = 3
x1 − 2x2 = 1
5x1 + 9x2 = −8
is an overdetermined system.
A system of linear equations with the number of equations equal to the number of variables is
called a square system.
How to solve a linear system
Forward elimination
To solve a linear system, the idea is to remove as many variables in the equations as we
can.
In particular, suppose we want to solve the system
x1 − 2x2 + x3 = 0 (1)
x1 + 2x2 − 8x3 = −15 (2)
−4x1 + 5x2 + 9x3 = 25 (3)
We rst eliminate x1 in (2) and (3).
For this we do the following.

(2) − (1) gives 4x2 − 9x3 = −15.
(3) + 4 ∗ (1) gives −3x2 + 13x3 = 25.

The original system now becomes
x1 − 2x2 + x3 = 0 (1)
4x2 − 9x3 = −15 (2a)
−3x2 + 13x3 = 25 (3a)
The next step is to eliminate x2 in (3a).
First we 'normalize' the constant of x2 in (2a) by dividing both sides of (2a) by 4.

9 15
x2 − x3 = − (2b)
4 4
Then we eliminate x2 in (3a).
(3a) + 3 ∗ (2b) gives 25
x
4 3
= 55
4
or x3 = 11
5
.

Backward substitution
So our new system is

x1 − 2x2 + x3 = 0 (1)
9 15
x2 − x3 = − (2b)
4 4
11
x3 = (3b)
5
Now that we already have the value of x3 , we will substitute this value back in (2b) to get the
value of x2 .
15 9 11 6
x2 = − + ∗ = .
4 4 5 5
Finally we substitute the values of x2 and x3 in (1) to get the value of x1 .

6 11 1
x1 = 2 ∗ − = .
5 5 5

Thus the solution to our original system
x1 − 2x2 + x3 = 0
x1 + 2x2 − 8x3 = −15
−4x1 + 5x2 + 9x3 = 25
is
1
x1 =
5
6
x2 =
5
11
x3 =
5

Matrix method
Matrix form
Using matrix multiplication, we can write a system of linear equations in a

matrix form.
For example,
" #" # " #
3x1 − x2 = 7 3 −2 x1 7
−→ = ,
x1 + 2x2 = 10 1 2 x2 10
    
x1 − 2x2 + x3 = 0 1 −2 1x1 0
    
x1 + 2x2 − 8x3 = −15 −→ 1
 2 −8 x2  = −15
   
−4x1 + 5x2 + 9x3 = 25 −4 5 9 x3 25
and so on.

Matrix method
Augmented matrix
We can use matrix form of a linear system to solve for its solution.
Suppose we want to solve the system
x1 − 2x2 + x3 = 0
x1 + 2x2 − 8x3 = −15
−4x1 + 5x2 + 9x3 = 25
Recall that this system can be written in the matrix form

    
1 −2 1 x1 0
 1 2 −8 x2  = −15 or Ax = b,
    
−4 5 9 x3 25
where      
1 −2 1 x1 0
A= 1 2 −8 , x =  x2  and b = −15 .
     
−4 5 9 x3 25

Matrix method
In solving for the solution of the system, what really matters are the coecients of the variables,
not the variables themselves. So it is redundant to write x1 , x2 and x3 all the time.
We form a new matrix called the augmented matrix of the system.

 
h i 1 −2 1 0
A b = 1 2 −8 −15
 
−4 5 9 25
Remember that the rst column in this matrix stands for the coecients associated with x1 ,
column 2 with x2 and column 3 with x3 .
Next we carry out the forward elimination and backward substitution as before.
     
1 −2 1 0 1 −2 1 0 1 −2 1 0
 R2 =R2 −R1   R3 =R3 +4R1 
 1 2 −8 −15 −−−−−−−−−→  0 4 −9 −15 −−−−−−−−−−→ 0 4 −9 −15
 
−4 5 9 25 −4 5 9 25 0 −3 13 25
Next we normalize the coecient of x2 .

 
1 −2 1 0
R2 = 1 R
4 2
. . . −−−−−−→ 0 −9 −15 
1

4 4 
0 −3 13 25

Matrix method
Then we eliminate x2 in Row 3 of the augmented matrix and then normalize the coecient of
x3to obtain the value of x3 .
   
1 −2 1 0 1 −2 1 0
R3 =R3 +3R2 R3 =R3 ÷ 25
. . . −−−−−−−−−→ 0 −9 −15 −−−−−− −−4
→ −9 −15 
1 0 1
  
4 4   4 4 
25 55 11
0 0 4 4
0 0 1 5
Up to this step we have completed the forward elimination step.

Next we do backward substitution.
− 11
   
R2 =R2 + 9 R3
1 −2 1 0 1 −2 0 5
4 6  R1 =R1 −R3 6
... −
−−−−−−−−−−
→ 0 1 0 −−−−−−−−−→ 0 1 0
  
5  5 
0 0 1 11 0 0 1 11
5 5
1
 
1 0 0 5
R1 =R1 +2R2  6 
−−−−−−−−−−→ 0 1 0 5 
0 0 1 11
5
We can now `read o' the solution of the system

1
x1 =
5
6
x2 =
5
11
x3 =
5
Matrix method
Row operations on matrices
The steps that we carried out before to obtain the solution of the system are called row
operations.
To get an overview of what we did, we give a summary of the steps carried out before.
In order to make the process of getting solution faster, we combine several row operations in one
step here.
     
1 −2 1 0 R2 =R2 −R1 1 −2 1 0 1 −2 1 0
1
 R3 =R3 +4R1   R2 = 4 R2  −9 −15 
 1 2 −8 −15 −−−−−−−−−−→ 0 4 −9 −15 −−−−−−−→ 0 1

4 4 
−4 5 9 25 0 −3 13 25 0 −3 13 25
−7 − 15 −7 − 15
   
R3 =R3 +3R2 1 0 1 0
R1 =R1 +2R2  2 2 R3 =R3 ÷ 25 2 2
−9 −15  4 −9 −15 
−−−−−−−−−−→ 0 1 −−−−−−−−−→ 0 1

4 4  4 4 
0 0 25 55 0 0 1 11
4 4 5
R2 =R2 + 9 R3
4  1

R1 =R1 + 7 R3
1 0 0 5
2 6 
−
−−−−−−−−−−
→ 0 1 0

5 
0 0 1 11
5

Matrix method
There are 3 types of row operations.

1 Swapping two rows of a matrix.
Example " # " #
3 −2 R1 ↔R2 1 2
−
−−−−−
→
1 2 3 −2
2 Multiplying a row by a nonzero constant.

Example " # " #
3 −2 R1 =3R1 9 −6
−−−−−−→
1 2 1 2
3 Adding a multiple of a row to another row.

Example " # " #
3 −2 R1 =R1 −2R2 1 −6
−−−−−−−−−→
1 2 1 2
Signicance Using row operations, we obtain new systems of linear equations which are
equivalent to the original system.

Matrix method
Another example
2x1 + 4x2 − 6x3 = 10
x1 + 2x2 + 7x3 = 2
7x1 + 14x2 + 9x3 = 26
To solve the system, we rst form the augmented matrix and then perform row operations until
we obtain the solution.
     
2 4 −6 10 1 2 −3 5 R2 =R2 −R1 1 2 −3 5
1
 R1 = 2 R1   R3 =R3 −7R1 
1 2 7 2  −−−−−−−→ 1 2 7 2  −−−−−−−−−−→ 0 0 10 −3
 
7 14 9 26 7 14 9 26 0 0 30 −9
41
   
1 2 −3 5 R1 =R1 +3R2 1 2 0
R2 = 1 R2  R3 =R3 −30R2  10
10 3 3 
−−−−−−−−→ 0 0 1 − 10  −−−−−−−−−−−→ 0 0 1 − 10


0 0 30 −9 0 0 0 0

Matrix method
So the original system is equivalent to

41
x1 + 2x2 =
10
3
x3 = −
10
This means that we have innitely many solutions to the system.
The value of x3 is xed (which is − 10

3
).
Each time we assign a value to x2 , we can get a value for x1 using x1 = 41

10
− 2x2 .
In sum,
41
x1 = − 2t
10
x2 = t, t∈R
3
x3 = −
10

Matrix method
Yet another example
3x1 + x2 − 2x3 = 1
x1 + x2 + x3 = 0
2x1 − 3x3 = 2
We have
     
3 1 −2 1 1 1 1 0 R2 =R2 −3R1 1 1 1 0
h i R1 ↔R2  R3 =R3 −2R1 
A b = 1 1 1 0 −−−−−−→ 3 1 −2 1 −−−−−−−−−−→ 0 −2 −5 1
   
2 0 −3 2 2 0 −3 2 0 −2 −5 2
−3 1
   
1 1 1 0 R1 =R1 −R2 1 0 −2
R2 =− 1 R2 R3 =R3 +2R2 2
2 5 1 − 5 1
−
−−−−−−−−
→ 0 1 −−−−−−−−−→ 0 1
  
2 2 2 2 
0 −2 −5 2 0 0 0 3
3 −1
 
R3 = 1 R3
1 0 −2 2
3 5 1
−−−−−−−→ 0 1
 
2 2 
0 0 0 1

Matrix method
The original system is equivalent to

3 1
x1 − x3 = −
2 2
5 1
x2 + x3 =
2 2
0=1
The last equation tells us that there is no solution in this case.

Matrix method
Leading 1's
Recall the nal forms of the augmented matrices in the above examples.
1 41
− 32 − 12
     
1 0 0 5
1 2 0 10
1 0
6  3  5 1
0 1 0 and 0 0 1 − 10 and 0 1
   
5   2 2 
11
0 0 1 5
0 0 0 0 0 0 0 1
The rst 1's that we see in each row of the matrices (looking from left to right) are called
leading 1's.

Matrix method
Leading and free variables
The variables corresponding to the leading 1's are called leading variables.
The remaining variables are called free variables.

Matrix method
Reduced row echelon form of a matrix
A matrix is said to be in row echelon form if

1. It is in a stair-case form.
2. The rst nonzero entries on each row (looking from left to right) are leading 1's.
If in addition to the above conditions, we also have
3. All entries above or below the leading 1's are 0.
Then the matrix is said to be in reduced row echelon form (rref).
For the matrices
The rst 2 matrices are in reduced row echelon form.

The last matrix is in row echelon form.

Matrix method
Rank of a matrix
The rank of a matrix is the number of leading 1's appearing in the (reduced) row echelon form of that matrix. So
for Example 1,
1
   
1 −2 1 0 1 0 0 5
has rank 3 since its rref is
h i
A b = −8 −15 6 
 1 2 0 1 0
  
5 
−4 5 9 25 0 0 1 11
5
In this case the system has a unique solution and

rank(A) = rank([A|b]) = number of variables.
For Example 2,
41
   
2 4 −6 10 1 2 0 10
has rank 2 since its rref is
h i
A b = 1 2 7 2 3 
− 10
0 0 1
  

7 14 9 26 0 0 0 0
In this case the system has a innitely many solutions and

rank(A) = rank([A|b]) < number of variables.
For Example 3,
3 1
   
3 1 −2 1 1 0 −2 −2
has rank 3 since its row echelon form is
h i
A b = 1 1 1 0 5 1
0 1
   
2 2 
2 0 −3 2 0 0 0 1
In this case the system has no solution and

rank(A) < rank([A|b]).
Matrix method
Number of solutions of a linear system
Based on our experience so far, we deduce the following about the number of solutions of a
linear system.
Let n be the number of variables in a system.

If rank(A) = (rank([A|b]) =) n, then the system has a unique solution.
In this case we also say that A is a full rank matrix.
So a linear system has a unique solution if and only if its coecient matrix has full rank.
If rank(A) = rank([A|b]) < n, then the system has innitely many solution.
If rank(A) < rank([A|b]), then the system has no solution.
Denition
A system which has a solution is said to be consistent.
A system which has no solution is said to be inconsistent.
The method to determine the consistency and the solution set of a linear system using rref of
the corresponding augmented matrix is called the Gaussian-Jordan elimination method.

Inverse of a matrix
Motivation
Given the linear equation

ax = b.
The solution is given by

b
x= (1)
a
provided that a 6= 0.
We have seen that we can write a linear system in the form
Ax = b, (2)
where A is a matrix, x and b are column vectors.

Note the similarity between (1) and (2).
Question Can we obtain the solution of (2) using a similar method as (1)?
This means we are talking about matrix division.
Answer This is unfortunately not possible.
But if we look at things in a dierent way, something is still possible.
Inverse of a matrix
We can rewrite x = b
a
as
x = a−1 ∗ b.
The condition here is that a 6= 0.
a−1 is called the inverse of the number a.
It turns out that it is possible to dene the inverse of a matrix provided some conditions
are met.
This means we can solve for x in Ax = b using
x = A−1 ∗ b
under certain conditions imposed on A.

Inverse of a matrix
Inverse of a matrix
A matrix A has the inverse A−1 = B if

1 it is a square matrix and
2 We can perform the following transformation:
h i row operations h i
A I −−−−−−−−−−−→ I B ,
where I is the identity matrix.

If a matrix A has an inverse, it is said to be invertible.
From the above criterion, we deduce that "
a square#matrix is invertible if and only if it has full rank.
1 2
Example Find the inverse of A = (if it exists).
7 3
Solution
1
" # " # " #
h i 1 2 1 0 R2 =R2 −7R1 1 2 1 0 R2 =− 11 R2 1 2 1 0
A I = −−−−−−−−−−→ −−−−−−−−−→ 7 1
7 3 0 1 0 −11 −7 1 0 1 11
− 11
" #
R1 =R1 −2R2 1 0 3
− 11 2
−−−−−−−−−−→ 11
0 1 7 1
− 11
11
So the inverse exists and " #

3 2
−1 − 11 11
A = 7 1
11
− 11
Inverse of a matrix
Application
Solve the system
x + 2y = 1
7x + 3y = −2
Matrix form " #" # " #

1 2 x 1
=
7 3 y −2
" #
1 2
We know from before that is invertible. So
7 3
" # " #−1 " # " #" # " #
3 2 7
x 1 2 1 − 11 11
1 − 11
= = 7 1
= 9
.
y 7 3 −2 11
− 11 −2 11

System of Linear Equations: Vietnamese-German University

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

System of Linear Equations: Vietnamese-German University

Uploaded by

Copyright:

Available Formats

System of linear equations

Tan Do (VGU) Introduction to MATLAB Lecture 5 1 / 28

Linear system: square system, underdetermined and overdetermined

Tan Do (VGU) Introduction to MATLAB Lecture 5 2 / 28

A linear equation of n variables x1 , x2 , . . . , xn is of the form

where a1 , a2 , . . . , an and b are constants.

is a linear equation of 2 variable x1 and x2 ,

is a linear equation of 3 variables, etc.

Tan Do (VGU) Introduction to MATLAB Lecture 5 3 / 28

System of linear equations

is a system of linear equations of 2 unknowns,

x1 + 2x2 − 8x3 = −15

−4x1 + 5x2 + 9x3 = 25

is a system of linear equations of 3 unknowns x1 , x2 and x3 , etc.

Tan Do (VGU) Introduction to MATLAB Lecture 5 4 / 28

In particular, suppose we want to solve the system

x1 + 2x2 − 8x3 = −15 (2)

−4x1 + 5x2 + 9x3 = 25 (3)

We rst eliminate x1 in (2) and (3).

For this we do the following.

Tan Do (VGU) Introduction to MATLAB Lecture 5 6 / 28

The original system now becomes

4x2 − 9x3 = −15 (2a)

−3x2 + 13x3 = 25 (3a)

The next step is to eliminate x2 in (3a).

First we 'normalize' the constant of x2 in (2a) by dividing both sides of (2a) by 4.

Tan Do (VGU) Introduction to MATLAB Lecture 5 7 / 28

So our new system is

Finally we substitute the values of x2 and x3 in (1) to get the value of x1 .

Tan Do (VGU) Introduction to MATLAB Lecture 5 8 / 28

Thus the solution to our original system

x1 + 2x2 − 8x3 = −15

−4x1 + 5x2 + 9x3 = 25

Tan Do (VGU) Introduction to MATLAB Lecture 5 9 / 28

Using matrix multiplication, we can write a system of linear equations in a

Tan Do (VGU) Introduction to MATLAB Lecture 5 10 / 28

x1 + 2x2 − 8x3 = −15

−4x1 + 5x2 + 9x3 = 25

Recall that this system can be written in the matrix form

Tan Do (VGU) Introduction to MATLAB Lecture 5 11 / 28

We form a new matrix called the augmented matrix of the system.

Next we normalize the coecient of x2 .

Tan Do (VGU) Introduction to MATLAB Lecture 5 12 / 28

Up to this step we have completed the forward elimination step.

We can now `read o' the solution of the system

Row operations on matrices

Tan Do (VGU) Introduction to MATLAB Lecture 5 14 / 28

There are 3 types of row operations.

2 Multiplying a row by a nonzero constant.

3 Adding a multiple of a row to another row.

Tan Do (VGU) Introduction to MATLAB Lecture 5 15 / 28

Suppose we want to solve the system

2x1 + 4x2 − 6x3 = 10

7x1 + 14x2 + 9x3 = 26

Tan Do (VGU) Introduction to MATLAB Lecture 5 16 / 28

So the original system is equivalent to

The value of x3 is xed (which is − 10

Each time we assign a value to x2 , we can get a value for x1 using x1 = 41

Tan Do (VGU) Introduction to MATLAB Lecture 5 17 / 28

Yet another example

Suppose we want to solve the system

Tan Do (VGU) Introduction to MATLAB Lecture 5 18 / 28

We rst eliminate x1 in (2) and (3).

Next we normalize the coecient of x2 .

We can now `read o' the solution of the system

The value of x3 is xed (which is − 10

The rst 2 matrices are in reduced row echelon form.

In this case the system has a innitely many solutions and