APM2613/202/0/2022

Tutorial letter 202/0/2022

APM2613

Year module

Department of Mathematical Sciences

This tutorial letter contains solutions for assignment 02.

Note: This is a fully online module and therefore it is only available on myUnisa.

BARCODE

university
Define tomorrow. of south africa
APM2613 Assignment 01

1 Checklist
By this time you should have received the following tutorial matter:
• APM2613/101/0/2022- General information about the module and the assignments

• APM2613/102/0/2022 - Background material

• APM2613/103/0/2022 - Textbook companion website

• APM2613/201/0/2022 - Tutorial Letter 201: Solutions to Assignment 01

• APM2613/202/0/2022 - This tutorial letter: Solutions to Assignment 02

2 Introduction
PLEASE NOTE:
Assignment 02 covers numerical methods for solving linear and nonlinear systems of equations.
In this assignment you get to practise on solving linear systems using direct and indirect(iterative)
methods. The relevant section of the textbook are Chapter 6, 7 and Section 10.1.

You have been told that in working out the following exercises, you can use Matlab/Octave to com-
pute. The solutions given below are based on computations using Matlab/Octave, and they show
the essential steps in the computations. Most of the values given in the process were obtained
using short format of Matlab/Octave, mostly in line with the 4 decimal place requirement.

3 Learning outcomes
After this assignment you should be be able to, among other things:
1. Work more with finite digit/decimal arithmetic accuracy.

2. Apply direct methods for linear systems:

• Gaussian elimination without pivoting (read the section on partial pivoting to understand
what without pivoting means).
• Gaussian elimination with scaled partial pivoting.
• Basic LU-decomposition.
• Using the inverse matrix method

Note that Gaussian elimination is the main focus of this module in solving linear systems as
opposed to other linear algebra methods, say, for finding the inverse.

3. Apply direct/iterative methods:

• Jacobi method
• Gauss-Seidel method

2
 APM2613/202/0/2022

• SOR method
4. Compare solutions obtained using different methods; e.g. residual vector. You should also
beware of the norm and condition number of a matrix as related to convergence of iterative
schemes.
5. Apply Newton’s method for systems of nonlinear equations.
Please note that in an assignment, the set of exercises given does not cover all the possible
concepts. So you should not read only what is covered in the assignment, rather read according
to the plan given in TL101.

4 Feedback on Solutions
Question 1 [30 marks]
Consider the linear system
√ √ √
πx1 − e x2 + 2x3 − 3x4 = 11
2 2 3
√ π x1 +√e x2 − e x3 +
√ 7 x4 = 0
5x1 − 6x2 +√x3 − 2x4 = √π
π 3 x1 + e2 x2 − 7x3 + 19 x4 = 2
whose actual solution is x = (0.788, −3.12, 0.167, 4.55)T . Carry out the following computations using
4 decimal places with rounding:
(1.1) Write the system as a matrix equation. (2)
(1.2) Solve the system using:
(a) Gaussian elimination without pivoting. (7)
(b) Gaussian elimination with scaled partial pivoting. (7)
(c) Basic LU decomposition. (7)
(1.3) Determine the number of arithmetic operations (multiplication/ division, addition/subtraction)
in (1.2)(a) above. (3)
(1.4) By comparing the residual vectors based on the approximate solutions obtained in (1.2)(a),
(b) and (c), which method gave the best approximation. (4)

Solution

(1.1) First we write the given system such that the coefficients are in decimal form, with four decimal
places values:
3.1416x1 − 2.7183 x2 + 1.4142x3 − 1.7321x4 = 3.3166
9.8696x1 + 2.7183 x2 − 7.3891x3 + 0.4286x4 = 0
2.2361x1 − 2.4495x2 + x3 − 1.4142x4 = 3.1416
31.0063x1 + 7.3891x2 − 2.6458x3 + 0.1111x4 = 1.4142

3
 From the above system we deduce the matrix form
    
3.1416 −2.7183 +1.4142 −1.7321 x1 3.3166
 9.8696 2.7183 −7.3891 0.4286   x2   0 
  x3  =  3.1416
    
 2.2361 −2.4495 1 −1.4142 
31.0063 7.3891 −2.6458 0.1111 x4 1.4142

(1.2) (a) To perform Gaussian elimination we start with the augmented matrix
 
3.1416 −2.7183 1.4142 −1.7321 3.3166
 9.8696 2.7183 7.3891 0.4286 0 
C=  2.2361 −2.4495

1.0000 −1.4142 3.1416 
31.0063 7.3891 −2.6458 0.1111 1.4142

As an example the first reduction, the first set of multipliers is obtained using
for i=2:4
m(i,1)=C(i,1)/C(1,1)
end
after initialising the m-matrix as M = zeros(4).
The underlying manual computations are:

m21 = C(2, 1)/C(1, 1); m31 = C(3, 1)/C(1, 1); m41 = C(4, 1)/C(1, 1)

The first reduction of rows R2 to R4 using the first row is computed using the Mat-
lab/Octave code
for i=2:4
C(i,:)=C(i,:)-m(i,1)*C(1,:)
end
which yields
 
3.1416 −2.7183 1.4142 −1.7321 3.3166
 0 11.2581 2.9463 5.8701 −10.4194 
C (1)  
 0 −0.5147 −0.0066 −0.1813 0.7809 
0 34.2176 −16.6034 17.2062 −31.3193

Comment:
It is very important to note the structure of the reduction formula to reduce Row j using
Row i: Rj = Rj − mji Ri for j = i + 1, . . . , n for (some people mix this up and end up with
wrong values)

The command C(i,:)=C(i,:)-m(i,2)*C(2,:) for i = 3, 4 will perform the next row reduction of
R3 and R4 using R2 to yield
 
3.1416 −2.7183 1.4142 −1.7321 3.3166
 0 11.2581 2.9463 5.8701 −10.4194 
C (2) =  
 0 0 0.1281 0.0870 0.3046 
0 0 −25.5582 −0.6354 0.3492

4
 APM2613/202/0/2022

The last (third) row reduction of R4 using R3 will use m43 = m(4, 3) = C(4, 3)/C(3, 3)
and C(4, :) = C(4, :) − m(4, 3) ∗ C(3, :) which leads to
 
3.1416 −2.7183 1.4142 −1.7321 3.3166
 0 11.2581 2.9463 5.8701 −10.4194 
C (3) =  
 0 0 0.1281 0.0870 0.3046 
0 0 0 16.7268 61.1156

Upon back substituting we have (for bravity let us drop the iterative index in the last
matrix C (3) and just use C):

x4 = C(4, 4)/C(4, 5) = 0.6916

x3 = (C(3, 5) − C(3, 4) ∗ x4 )/C(3, 3) = −2.8033
x2 = (C(2, 5) − C(2, 4) ∗ x4 − C(2, 3) ∗ x3 )/C(2, 2) = −0.1045
x1 = (C(1, 5) − C(1, 4) ∗ x4 − C(1, 3) ∗ x3 − C(1, 2) ∗ x2 )/C(1, 1) = 3.6538

Hence, x = (0.6916 − 2.8033 − 0.10453.6538).

(b) In this exercise the focus is on scaled partial pivoting.
The first thing to compute is the vector of scalers, s.
By definition,
s(i) = max(|A(i, j)), j = 1, 2, . . . , 4
The corresponding computer code in Matlab/Octave is
for i=1:4
s(i)=max(abs(A(i,:)));
end
This gives

s(1) = max{3.1416, | − 2.7183|, 1.4142, | − 1.7321|} = 3.1416

s(2) = max{9.8696, 2.7183, 7.3891, 0.4286} = 9.8696
s(3) = max{2.2361, | − 2.4495|, 1.0000, | − 1.4142|} = 2.2495
s(4) = max{31.0063, 7.3891, | − 2.6458|, 0.1111} = 31.0063

So that the scalers are

s = {3.1416 9.8696 2.4495 31.0063}

NOTE: s is a set of values, not a vector. These values are row specific. They are com-
puted only once and are used through out the computations. When a row is moved, the
value moves with it.

In the following steps we perform Gaussian elimination using the scaling factor si com-
puted above.

Step 1:
Next we compute the ratios to determine whether row interchange is necessary. These

5
 are ratios of the latest matrix entries in the pivot column and the corresponding scalers.

So the first time we compute

|C(i1)|
ri1 = , i = 1, . . . , 4
s(i)

 
|C(1, 1)| |C(2, 1)| |C(3, 1)| |C(4, 1)|
ri1 = , , ,
s1 s2 s3 s4
 
|3.1416| |9.8696| |2.2361| |31.0063|
= , , ,
3.1416 9.8696 2.4495 31.0063
= {1, 1, 0.9130, 1}

Using the notation in the textbook, rp1 = max{ri1 } = max{1, 1, 0.9130, 1} This time we
have a tie between r11 , r21 and r41 . In a case like this, where there is no clear max, we
make an arbitrary row choice among those that qualify. Let’s choose Row 1; i.e p = 1.
In which case there is no row interchange. So we reduce the matrix as it is to get (as
computed in (1.2)(a) above)
 
3.1416 −2.7183 1.4142 −1.7321 3.3166 s1
 0 11.2581 2.9463 5.8701 −10.4194  s2

C (1) 
 0 −0.5147 −0.0066 −0.1813 0.7809  s3
0 34.2176 −16.6034 17.2062 −31.3193 s4

Notice that we have appended an extra column outside the augmented matrix to help
keep track of the rows after interchanging and the corresponding si . (This is optional,
done here for the convenience of visual tracking purposes)

Step 2:
Before we continue with the reduction we need to decide whether the next pivot C(2, 2)
is the most suitable one to pivot on. So we focus on rows Ri ; i = 2, 3, 4 and use them
with their corresponding scalers.

So we compute
|C(i2)|
ri2 = , i = 2, 3, 4
s(i)
to obtain
 
|C(2, 2)| |C(3, 2)| |C(4, 2)|
ri2 = , ,
s2 s3 s4
 
|11.2581| | − 0.5147| |34.2176|
= , ,
9.8696 2.4495 31.0063
= {1.1407, 0.2101, 1.1036}

So that rp2 = r2,2 ; i.e. p = 2. This means that Row 2 is the next pivot row, hence there is
no row interchange.

6
 APM2613/202/0/2022

Again, using previous results, we reduce and get

 
3.1416 −2.7183 1.4142 −1.7321 3.3166 s1
 0 11.2581 2.9463 5.8701 −10.4194  s2

C (2) = 
 0 0 0.1281 0.0870 0.3046  s3
0 0 −25.5582 −0.6354 0.3492 s4

Step 3:
Now we repeat the process of looking for the next best pivot row, focusing on i = 3, 4.

|C(i3)|
ri3 = { , i = 34}
s(i)
|C(3, 3)| |C(4, 3)|
= { , }
s3 s4
| − 25.5582| | − 0.6354|
= { , }
2.4495 31.0063
= {0.0523, 0.8243}

and rp3 = max{ri3 } = 0.8243 which corresponds to p = 4. Hence interchange R4 with

R3 :  
3.1416 −2.7183 1.4142 −1.7321 3.3166 s1
 0 11.2581 2.9463 5.8701 −10.4194  s2

C (3) = 
 0 0 −25.5582 −0.6354 0.3492  s4
0 0 0.1281 0.0870 0.3046 s3
Notice the positions of s3 and s4 after the interchange. reducing on R3 we obtain
 
3.1416 −2.7183 1.4142 −1.7321 3.3166 s1
 0 11.2581 2.9463 5.8701 −10.4194  s2

C (4) = 
 0 0 −25.5582 −0.6354 0.3492  s4
0 0 0 0.0838 0.3064 s3

Step 4-Back substituting:

Upon back substituting, we obtain

x = (0.6918 − 2.8039 − 0.1045 3.6551)

(c) The key to answering this question is found in Theorem 6.19. Part (a) requires solution
of the system by Gaussian elimination without pivoting. This is the same as saying with-
out row interchange for any reason. Thus we can use the results in (a) to answer this
question.

In so doing we need to find the lower and upper triangular matrices after all the reduction,
but before back substitution.
It is useful to keep track of the multipliers for answering this part - LU factorisation/decomposition.
The multipliers obtained before each reduction form the subdiagonal entries of the ma-
trix L and the upper triangular matrix is extracted from the final, row reduced matrix.

7
 The matrix of multipliers is
 
0 0 0 0
 3.1416 0 0 0 
M =
 0.7118 −0.0457

0 0 
9.8696 3.0394 −199.5021 0

So that in LU ,
 
1 0 0 0
 3.1416 1 0 0 
L = M + [ unit diagonal matrix ] = 
 0.7118 −0.0457

1 0 
9.8696 3.0394 −199.5021 1

From the last step of row reduction in Gaussian elimination we can extract the upper tri-
angular matrix required for (c) (exclude the column corresponding to the reduced vector
b) as  
3.1416 −2.7183 1.4142 −1.7321
 0 11.2581 2.9463 5.8701 
U = 
 0 0 0.1281 0.0870 
0 0 0 16.7268
In solving the system Ax = LU x = b, we let U x = y and solve Ly = b, where b is the
original RHS of the system. We will then use the y to solve U x = y.
First we solve Ly = b:
    
1 0 0 0 y1 3.3166
 3.1416 1 0 0    y2  =  0
   
Ly = 
 0.7118 −0.0457

1 0   y3   3.1416 
9.8696 3.0394 −199.5021 1 y4 1.4142

Note that we can take advantage of the lower triangular matrix, and solve by forward
substitution to obtain  
3.3166
 −10.4194 
y= 
 0.3047 
61.1330
Now we can obtain x by solving U x = y by backward substitution:
    
3.1416 −2.7183 1.4142 −1.7321 x1 3.3166
 0 11.2581 2.9463 5.8701   x2   −10.4194
    
 = ,
 0 0 0.1281 0.0870   x3   0.3047 
0 0 0 16.726 x4 61.1330

which yields  
0.6913
 −2.8040 
x= 
 −0.1038 
3.6548

8
 APM2613/202/0/2022

(1.3) The number of arithmetic operations can be computed using the formulas given on p371 of
the textbook:
n3 n n3 n2 5n
Multiplications/divisions: + n2 − ; Additions/subtractions: + −
3 3 3 2 6
which works out to be 62 with n = 4.

(1.4) Here we want to compare the the actual solution x = (0.6916, −2.8033, −0.1045, 3.6538) with
the the solutions obtained using the methods in (1.2). The residual vector is simply the differ-
ence between the true solution and the computed solution.
Let xG = (0.6916, ; −2.8033, −0.1045, ; 3.6538), xP = (0.6918, −2.8039, −0.1045 3.6551) and
xD = (0.6913 − 2.8040 − 0.10383.6548) be respectively the solutions obtained using the Gaus-
sian, scaled partial pivoting and decomposition methods. Then the residual vectors are
    
0.6916 0.6916 0
 −2.8033   −2.8033   0 
r G = x − xG =   −0.1045   −0.1045  =  0 
   

3.6538 3.6538 0
     
0.6916 0.6918 −0.0002
 −2.8033   −2.8039   0.0006 
rP = x − xP =   −0.1045  −  −0.1045  = 
    
0 
3.6538 3.6551 −0.001
     
0.6916 0.6913 0.0003
 −2.8033   −2.8040   0.0007 
r D = x − xD =   −0.1045  −  −0.1038  =  −0.0007 
    

3.6538 3.6548 −0.001

Question 2 [25 marks]

Consider the matrices
     
1 −1 2 −1 6 1
 1 0 −1 1   4   1 
C= ; b1 = 
 −2  ;
 b2 −  
 2 1 3 −4   2 
0 −1 1 −1 5 −1

(2.1) Use Gaussian elimination to compute the inverse C −1 . (8)

(2.2) Use the inverse in (2.1) above to solve the linear systems Cx = b1 and Cx = b2 . (6)

(2.3) Find the solution of the above two systems by multiplying the matrix [b1 b2 ] by the inverse
obtained in (2.1) above. Compare the solution with that obtained in (2.2). (4)

(2.4) Solve the linear systems in (2.2) above by applying Gaussian elimination to the augmented
matrix [C : b1 b2 ]. (6)

Solution
This exercise is about solving a linear system using the inverse method. We still apply Gaussian

9
elimination in finding the inverse - not the other methods you learnt from linear algebra.

The true solutions of Cx = b1 and Cx = b2 are respectively

x1 = (3, −6, −2, −1) and x2 = (1, 1, 1, 1)
(2.1) We use Gaussian elimination to find the inverse as follows:
We form an augmented matrix in which the right hand portion of the matrix is the identity
matrix. We then proceed using Gaussian elimination to reduce this augmented matrix until
the identity matrix shifts to the left hand portion.

So we have let’s call the augmented matrix A.

 
1 −1 2 −1 1 0 0 0
 1 0 −1 1 0 1 0 0 
A= 
 2 1 3 −4 0 0 1 0 
0 −1 1 −1 0 0 0 1

Ri = Ri − A(i, 1)/A(1, 1) ∗ R1 , i = 2, 3, 4:

(Equivalent code: C(i,:)=C(i,:)-A(i,4)/C(4,4)*C(4,:))

 
1 −1 2 −1 1 0 0 0
 0
 1 −3 2 −1 1 0 0 

 0 3 −1 −2 −2 0 1 0 
0 −1 1 −1 0 0 0 1
Ri = Ri − A(i, 2)/A(2, 2) ∗ R2 , i = 3, 4:
 
1 −1 2 −1 1 0 0 0
 0
 1 −3 2 −1 1 0 0 

 0 0 8 −8 1 −3 1 0 
0 0 −2 1 −1 1 0 1
R4 = R4 − A(4, 3)/A(3, 3) ∗ R3
 
1.0000 −1.0000 2.0000 −1.0000 1.0000 0 0 0

 0 1.0000 −3.0000 2.0000 −1.0000 1.0000 0 0 

 0 0 8.0000 −8.0000 1.0000 −3.0000 1.0000 0 
0 0 0 −1.0000 −0.7500 0.2500 0.2500 1.0000
To obtain only 1’s on the diagonal of the left hand portion of the last matrix we can proceed
in two ways:
(i) We can use the method discussed in the textbook and solve by taking each column on the
right hand portion and solve A(3) x = ci , i = 1, . . . 4 by back substitution;
(ii) We can perform further row reduction until we obtain the identity matrix on the left. We
can do this by starting at the top and go downward or start at the bottom and move up.

We shall use option (ii) and start reduction from the bottom up.
R4 / − 1; Ri = Ri − C(i, 4)/C(4, 4) ∗ R4 , i = 1, 2, 3:

10
 APM2613/202/0/2022

[(The equivalent Matlab/Octave code is:

C(4,:)/C(4,4);
for i=1:3
C(i,:)=C(i,:)-C(i,4)/C(4,4)*C(4,:)
end ]

We shall omit this code (which changes for i-range) in subsequent calculations.
 
1.0000 −1.0000 2.0000 0 1.7500 −0.2500 −0.2500 −1.0000

 0 1.0000 −3.0000 0 −2.5000 1.5000 0.5000 2.0000 

 0 0 8.0000 0 7.0000 −5.0000 −1.0000 −8.0000 
0 0 0 1.0000 0.7500 −0.2500 −0.2500 −1.0000

C(i, :) = C(i, :) − C(i, 3)/C(3, 3) ∗ C(3, :):

 
1.0000 −1.0000 0 0 0 1.0000 0 1.0000

 0 1.0000 0 0 0.1250 −0.3750 0.1250 −1.0000 

 0 0 1.0000 0 0.8750 −0.6250 −0.1250 −1.0000 
0 0 0 1.0000 0.7500 −0.2500 −0.2500 −1.0000

C(1, :) = C(1, :) + C(2, :):

 
1.0 0 0 0 0.1250 0.6250 0.1250 0
 0 1.0
 0 0 0.1250 −0.3750 0.1250 −1.0000 

 0 0 1.0 0 0.8750 −0.6250 −0.1250 −1.0000 
0 0 0 1.0 0.7500 −0.2500 −0.2500 −1.0000

Now that we have the identity matrix on the LHS, the RHS is the inverse of the matrix so we
can extract it as  
0.1250 0.6250 0.1250 0
 0.1250 −0.3750 0.1250 −1.0000 
A−1 =   0.8750 −0.6250 −0.1250 −1.0000 


0.7500 −0.2500 −0.2500 −1.0000

(2.2) Having obtained the inverse we can compute the solution of the system Ax = b based on the
fact that
A−1 Ax = x = A−1 b
We multiply b, i = 1, 2 on the left by A−1 , replacing b with b1 and b2 . This yields
    
0.1250 0.6250 0.1250 0 6 3
 0.1250 −0.3750 0.1250 −1.0000    4  =  −6 
   
x1 =   0.8750 −0.6250 −0.1250 −1.0000   −2   −2 
0.7500 −0.2500 −0.2500 −1.0000 5 −1
and    
0.1250 0.6250 0.1250 0 1 1
 0.1250 −0.3750 0.1250 −1.0000   1
    1 
x2 = 
 0.8750 −0.6250 −0.1250 −1.0000   2
= 
  1 
0.7500 −0.2500 −0.2500 −1.0000 −1 1
Thus we have obtained the solutions to the two systems, as expected.

11
(2.3) We could find both solutions at once by multiplying the inverse into the combined RHS vectors
as
    
0.1250 0.6250 0.1250 0 6 1 3 1
 0.1250 −0.3750 0.1250 −1.0000 
A−1 [b1 b2 ] =   4  =  −6 1 
1 
  
 0.8750 −0.6250 −0.1250 −1.0000   −2 2   −2 1 
0.7500 −0.2500 −0.2500 −1.0000 5 −1 −1 1

where now the solutions read from the first column for the first system and the second column
for the second system.

(2.4) Instead of using Gaussian elimination on the separate systems we can simply augment both
RHS vectors to C and apply Gaussian elimination.
 
1 −1 2 −1 6 1
 1 0 −1 1 4 1 
C= ;
 2 1 3 −4 −1 2 
0 −1 1 −1 5 −1

So we apply Gaussian elimination as follows:

R2 = R2 − C(2, 1)/C(1, 1) ∗ R1 ; R3 = R3 − C(3, 1)/C(1, 1) ∗ R1 :
 
1 −1 2 −1 6 1
 0 1 −3 2 −2 0 
C= 
 0 3 −1 −2 −13 0 
0 −1 1 −1 5 −1

R3 = R3 − C(3, 2)/C(2, 2) ∗ R2 ; R4 = R4 − C(4, 2)/C(2, 2):

 
1 −1 2 −1 6 1
 0 1 −3 2 −2 0 
C= 
 0 0 8 −8 −7 0 
0 0 −2 1 3 −1

R4 = R4 − C(4, 3)/C(3, 3) ∗ R3 :
 
1 −1 2 −1 6 1
 0 1 −3 2 −2 0 
C= 
 0 0 8 −8 −7 0 
0 0 0 −1 1.2500 −1

By way of demonstrating an alternative to back substitution, we can reduce the matrix so that
the LHS is an identity matrix, starting from the bottom and working up. R4 /C(4, 4):
 
1 −1 2 −1 6 1
 0 1 −3 2 −2 0 
C= 
 0 0 8 −8 −8 0 
0 0 0 1 −1 1

R3 = R3 − C(3, 4)/C(4, 4) ∗ R4 ; R2 = R2 − C(2, 4)/C(4, 4) ∗ R4 :

12
 APM2613/202/0/2022

 
1 −1 2 0 5 2
 0 1 −3 0 0 −2 
C= 
 0 0 8 0 −16 8 
0 0 0 1 −1 1
R3 = R3 /C(3, 3):  
1 −1 2 −1 5 1
 0 1 −3 2 0 0 
C= 
 0 0 1 0 −2 1 
0 0 0 1 −1 1
R2 = R2 − C(2, 3)/C(3, 3) ∗ R3 ; R1 = R1 − C(1, 3)/C(3, 3) ∗ R3 :
 
1 −1 0 0 9 0
 0 1 0 0 −6 1 
C= 
 0 0 1 0 −2 1 
0 0 0 1 −1 1
R1 = R1 + R2 :  
1 0 0 0 3 1
 0 1 0 0 −6 1 
C= 
 0 0 1 0 −2 1 
0 0 0 1 −1 1
Thus the two columns on the RHS partition of the matrix give the two solutions x1 and x2
respectively.

Question 3 [25 marks]

Consider again the linear system Ax = b used in Question 1. For each of the methods men-
tioned below perform three iterations using 4 decimal place arithmetic with rounding and the initial
approximation x(0) = (0.5, 0, 0, 2)T .
(3.1) By examining the diagonal dominance of the coefficient matrix, A, determine whether the
convergence of iterative methods to solve the system be guaranteed. (3)
(3.2) Solve the system using each of the following methods:
(a) the Jacobi method. (6)
(b) the Gauss-Seidel method (6)
(c) the Successive Over-Relaxation technique with ω = 0.4. (6)
(3.3) Compute the residual for the approximate solutions obtained using each method above and
compare results. (4)

Solution

This question is about iterative/indirect methods of solving linear systems. It will serve you well
to note the relationship between these methods. Unlike direct methods, with indirect methods you
start with a guess of the solution, followed by a series of repetitive steps (the iterations) to obtain
(hopefully) better solutions that, if the system is well poised, get closer to the expected solution.

13
(3.1) Diagonal dominance is a quick way to check if a given system is LIKELY to converge as you
carry on with iterations. A system Ax = b is likely to converge to a solution if
n
X
|aii | > |aij |
i=1,i6=j

i.e. if the magnitude of the diagonal entry in each row of the coefficient matrix is not less that
the sum of the magnitudes of the other entries in the corresponding row. So from the given
system, we have the coefficient matrix
 
3.1416 −2.7183 1.4142 −1.7321
 9.8696 2.7183 7.3891 0.4286 
 
 2.2361 −2.4495 1.0000 −1.4142 
31.0063 7.3891 −2.6458 0.1111

so that

R1 : |3.1416|| − 2.7183| + |1.4142| + | − 1.7321| or 3.1416 < 5.8646

R2 : |2.7183| < |9.8696| + |7.3891| + |0.4286| or 2.7183 < 17.687
R3 : |1| < |2.2361| + |2.4495| + | − 1.4142| or 1 < 6.0998
R4 : |0.1111| < |31.0063| + |7.3891| + | − 2.6458| or 0.1111 < 41.041

In actual fact, the diagonal entries in each row have the smallest magnitude.
We can deduce from these inequalities that the matrix is not diagonally dominant, hence the
iterative methods are NOT likely to converge.

Let’s see how it turns out.

COMMENT:
It is important to understand the notation used in indicating the iteration count as distinguished
from mere superscripts. In the textbook, the k-th iteration of a vector x = (x1 , x2 , . . . ) is
denoted  (k) 
x1
(k)  x(k) 
x = 2 
..
.
Thus the interative index/count is denoted with a bracketed superscript (k).

(3.2) Given
3.1416x1 −2.7183x2 1.4142x3 −1.7321x4 = 3.3166
9.8696x1 2.7183x2 7.3891x3 0.4286x4 = 0
2.2361x1 −2.4495x2 1.0000x3 −1.4142x4 = 3.1416
31.0063x1 7.3891x2 −2.6458x3 0.1111x4 = 1.4142

(a) Although there is a formula for the Jacobi iterative scheme given in the textbook, it is
better to understand how the system is arranged to arrive at the scheme.

14
 APM2613/202/0/2022

The basic procedure is to solve for xi in each Ri :

(k+1) 1 (k) (k) (k)
x1 = 3.1416
[3.3166 +2.7183x2 −1.4142x3 +1.7321x4 ]
(k+1) 1 (k)
x2 = 2.7183
[0 −9.8696x1 −7.3891xk3 −0.4286xk4 ]
(k+1) 1 (k) (k) (k)
x3 = 1.0000
[3.1416 −2.2361x1 +2.4495x2 +1.4142x4 ]
(k+1) 1 (k) (k)
x4 = 0.1111
[1.4142 −31.0063xk1 −7.3891x2 +2.6458x3 ]
(0) (0) (0) (0)
Using (x1 , x2 , x3 , x4 )T = (0.5, 0, 0, 2)T as the initial solution, we get:

Iteration 1:
(1) 1 (0) (0) (0)
x1 = 3.1416
[3.3166 +2.7183x2 −1.4142x3 +1.7321x4 ]
(1) 1 (0)
x2 = 2.7183
[0 −9.8696x1 −7.3891x03 −0.4286x04 ]
(1) 1 (0) (0) (0)
x3 = 1.0000
[3.1416 −2.2361x1 +2.4495x2 +1.4142x4 ]
(1) 1 (0) (0)
x4 = 0.1111
[1.4142 −31.0063x01 −7.3891x2 +2.6458x3 ]
(1) 1
x1 = 3.1416
[3.3166 +2.7183(0) −1.4142(0) +1.7321(2)] = 2.1584
(1) 1
x2 = 2.7183
[0 −9.8696(0.5) −7.3891(0) −0.4286(2)] = −2.1307
(1) 1
x3 = 1.0000
[3.1416 −2.2361(0.5) +2.4495(0) +1.4142(2)] = 4.8520
(1) 1
x4 = 0.1111
[1.4142 −31.0063(0.5) −7.3891(0) +2.6458(0) ] = −126.81
Two more iterations yield

i 0 1 2 3
x1 0.5 2.1584 −72.8897 −99.2563
x2 0 −2.1307 −1.0307 823.3169
x3 0 4.8520 −186.2433 −306.4530
x4 2 −126.8132 −332.3847 15988.3637

(b) The Gauss Seidel method is an modification of the Jacobi method by using the newly
(k+1) (k+1) (k+1) (k+1)
computed values xk .e.g. in computing x3 you will already have x1 and x2
(k)
which you use immediately, but you will still use x4 from the previous iteration. So for
the given system, the iterative scheme is
(k+1) 1 (k) (k) (k)
x1 = 3.1416
[3.3166 +2.7183x2 −1.4142x3 +1.7321x4 ]
(k+1) 1 (k+1)
x2 = 2.7183
[0 −9.8696x1 −7.3891xk3 −0.4286xk4 ]
(k+1) 1 (k+1) (k+1) (k)
x3 = 1.0000
[3.1416 −2.2361x1 +2.4495x2 +1.4142x4 ]
(k+1) (k+1) (k+1)
x4 = 1
0.1111
[1.4142 −31.0063xk+1
1 −7.3891x2 +2.6458x3 ]

Note the iterative indices (k + 1) on the RHS which indicate the values that are used
immediately. This is the only modification needed of the Jacobi iterative scheme.

COMMENT:
It is important to include these indices when you answer questions so that you will also
be reminded as to which method you are using and in which terms you are using old
and new values.

15
 Iteration 1
(1) 1 (0) (0) (0)
x1 = 3.1416
[3.3166 +2.7183x2 −1.4142x3 +1.7321x4 ]
(1) 1 (1)
x2 = 2.7183
[0 −9.8696x1 −7.3891x03 −0.4286x04 ]
(1) 1 (1) (1) (0)
x3 = 1.0000
[3.1416 −2.2361x1 +2.4495x2 +1.4142x4 ]
(1) 1 (1) (1)
x4 = 0.1111
[1.4142 −31.0063x11 −7.3891x2 +2.6458x3 ]

which translates to
(1) 1
x1 = 3.1416 [3.3166 +2.7183(0) −1.4142(0) +1.7321(2)]
= 2.1584

(1) 1
x2 = 2.7183 [0 −9.8696(2.1584) −7.3891(0) −0.4286(2)]
= −8.1520

(1) 1
x3 = 1.0000 [3.1416 −2.2361(2.1584) +2.4495(−8.1520) +1.4142(2)]
= −18.8248

(1) 1
x4 = 0.1111 [1.4142 −31.0063(2.1584) −7.3891(−8.1520) +2.6458(−18.8248) ]
= −495.7690

Iteration 2
(2) 1
x1 = 3.1416
[3.3166 +2.7183(0) −1.4142(0) +1.7321(2)]
= −270.86
(2) 1
x2 = 2.7183
[0 −9.8696(−270.86) −7.3891(0) −0.4286(2)]
= 1112.8
(2) 1
x3 = 1.0000
[3.1416 −2.2361(−270.86) +2.4495(1112.8) +1.4142(2)]
= 2633.5
(2) 1
x4 = 0.1111
[1.4142 −31.0063(−270.86) −7.3891(1112.8) +2.6458(2633.5) ]
= 64312
The summary of outputs is

i 0 1 2
(i)
x1 0.5 2.1584 −270.86
(i)
x2 0 −8.1520 1112.8
(i)
x3 0 −18.825 2633.5
(i)
x4 2 −495.77 64312

We can observe from the above computations that the iterates diverge hopelessly from
the expected solution. This is not surprising since the system did NOT satisfy the diag-
onal dominance test.
(c) The SOR method is a modification of the Gauss-Seidel method by introducing the
weighting ω in the computations. Here we modify the Gauss-Seidel scheme to get SOR

16
 APM2613/202/0/2022

scheme:
(k+1) (k) ω (k) (k) (k)
x1 = (1 − ω)x1 − 3.1416
[3.3166 +2.7183x2 −1.4142x3 +1.7321x4 ]
(k+1) (k) ω (k+1)
x2 = (1 − ω)x2 − 2.7183
[0 −9.8696x1 −7.3891xk3 −0.4286xk4 ]
(k+1) (k) ω (k+1) (k+1) (k)
x3 = (1 − ω)x3 − 1.0000
[3.1416 −2.2361x1 +2.4495x2 +1.4142x4 ]
(k+1) (k) (k+1) (k+1)
x4 = (1 − ω)x4 − ω
0.1111
[1.4142 −31.0063xk+1
1 −7.3891x2 +2.6458x3 ]

Note that what is inside the square brackets is exactly what the Gauss-Seidel terms (to-
(k)
gether with the iterative indices). The only modification is the additional terms (1 − ω)xi
and the numerator outside the square bracket which is now ω instead of 1.

So substituting ω = 0.4 into the scheme, we will be computing with the following iterative
scheme explicitly:
(k+1) (k) 0.4 (k) (k) (k)
x1 = 0.6x1 − 3.1416
[3.3166 +2.7183x2 −1.4142x3 +1.7321x4 ]
(k+1) (k) 0.4 (k+1)
x2 = 0.6x2 − 2.7183
[0 −9.8696x1 −7.3891xk3 −0.4286xk4 ]
(k+1) (k) 0.4 (k+1) (k+1) (k)
x3 = 0.6x3 − 1.0000
[3.1416 −2.2361x1 +2.4495x2 +1.4142x4 ]
(k+1) (k) (k+1) (k+1)
x4 = 0.6x4 − 0.4
0.1111
[1.4142 −31.0063xk+1
1 −7.3891x2 +2.6458x3 ]

Using this scheme,

Iteration 1:

(1) 0.4
x1 = 0.6(0.5) − 3.1416 [3.3166 +2.7183(0) −1.4142(0) +1.7321(2)]
= −0.5634
(1) 0.4
x2 = 0.6(0) − 2.7183 [0 −9.8696(−0.5634) −7.3891(0) −0.4286(0)]
= −0.6920
(1) 0.4 (0)
x3 = 0.6(0) − 1.0000 [3.1416 −2.2361(−0.5634) +2.4495(−0.6920) +1.4142x4 ]
= −2.2138
(1) 0.4
x4 = 0.6(2) − 0.1111 [1.4142 −31.0063(−0.5634) −7.3891(−0.6920) +2.6458(−2.2138)]
= −64.1032

Iteration 2
(2) 0.4
x1 = 0.6(−0.5634) − 3.1416 [3.3166 +2.7183(0) −1.4142(0) +1.7321(2)]
= 13.218
(2) 0.4
x2 = 0.6(0) − 2.7183 [0 −9.8696(13.218) −7.3891(0) −0.4286(0)]
= 12.331
(2) 0.4 (0)
x3 = 0.6(0) − 1.0000 [3.1416 −2.2361(13.218) +2.4495(12.331) +1.4142x4 ]
= 33.417
(2) 0.4
x4 = 0.6(2) − 0.1111 [1.4142 −31.0063(13.218) −7.3891(12.331) +2.6458(33.417)]
= 1441.7

17
 Hence the summary of outputs for two iterations is

i 0 1 2
(i)
x1 0.5 −0.56336 13.218
(i)
x2 0 −0.69204 12.331
(i)
x3 0 −2.2138 33.417
(i)
x4 2 −64.103 1441.7

(3.3) The divergence of the iterations does not justify the computation of the residual vectors.
However, looking at the approximations resulting after two iterations, we can observe that
although divergent, the magnitude of the approximate solution vectors is worse with Jacobi
method and smallest with the SOR method.

Question 4 [20 marks]

Consider the following nonlinear system:

5x21 = x22
x2 − 0.25(sin x1 + cos x2 ) = 0

(4.1) Draw a sketch of the two functions on the same axes. (5)

(4.2) Write down a computer code for Newton’s method for more than one variable to solve the
above system. (8)

(4.3) Use the algorithm in (4.2) to solve the given system of nonlinear equations with x(0) = (0, 1)T
as the initial guess of the solution. (7)

Solution

(4.1) Plotting two variable functions is possible with Matlab/Octave. Because we want the proper
root equations f (x, y) = 0for the system, let

z = 5x21 − x22 = 5x2 − y 2 ; h = x2 − 0.25(sin x1 + cos x2 ) = y − 0.25(sin x + cos y)

To plot these in Matlab/Octave first we define the domains of x andy. Then define the func-
tions z and h. Then plot. For a simple plot the series of commands is

[x,y]=meshgrid(-1:0.1:1,-1:0.1:1) % Define x and y domains

z=5*x.ˆ2-y.ˆ2 % Define function for first equation
% (Note the .ˆ for component-by-component powers)
h=y.-0.25.*(sin(x)+cos(y)) % Define function for second equation
mesh(x,y,z) % Plot of the first function
hold on % To hold plot for super imposition
mesh(x,y,h) % Plot of the second function

18
 APM2613/202/0/2022

Try these on Matlab/Octave to see what the graphs look like. There are only two commands
you need, meshgrid and mesh.
(4.2) In this exercise you need to write a code for the algorithm for solving coupled non-linear equa-
tions and use the code to produce some output.

In the case of a single variable, Newton’s method is applied to an root equation written in the
form f (x) = 0 to find a solution x. In a similar manner, now the system of nonlinear equations
has to be in the form

f1 (x1 , x2 ) = 0   
f1 (x1 , x2 )
= F (x1 , x2 ) =
f2 (x1 , x2 )
f2 (x1 , x2 ) = 0


to find a vector solution (x1 , x2 ).

So now F (x1 , x2 ) is a vector function, whose components are the given nonlinear functions.

Unlike in the case of nonlinear equations involving only one variable, Newton’s method here
replaces the derivative with the Jacobian (Don’t try to solve for one variable in terms of the
other): Instead of
f (x(k) )
x(k+1) = x(k) − 0
f (x(k) )
the iterative scheme is now
" # " #
(k+1) (k)
x1 x1 (k) (k) (k) (k)
(k+1) = (k) − J −1 (x1 , x2 )F (x1 , x2 )
x2 x2
 
f1 (x1 , x2 )
where F (x1 , x2 ) = and J is the Jacobian matrix.
f2 (x1 , x2 )

Now given the nonlinear system,

5x21 = x22
,
x2 − 0.25(sin x1 + cos x2 ) = 0
we need to rearrange the first equation to get the form f1 (x1 , x2 ) = 0. So we will work with
the system,
f1 (x1 , x2 ) = 5x21 − x22 =0
f2 (x1 , x2 ) = x2 − 0.25(sin x1 + cos x2 ) = 0
So that  
5x21 − x22
F (x1 , x2 ) =
x2 − 0.25(sin x1 + cos x2 )
The Jacobian matrix of the vector function F (x1 , x2 ) is then
" #
∂f1 ∂f1
∂x1 ∂x2
J = ∂f2 ∂f2
∂x1 ∂x2
   
10x1 −2x2 10x1 −2x2
= =
−0.25 cos x1 1 − 0.25(− sin x2 ) −0.25 cos x1 1 + sin4x2

19
 Manually, the inverse of the Jacobian can be evaluated as follows:
sin x2 1
det(J) = |J| = 10x1 (1 + ) − x2 cos x1
4 2
so that
sin x2
 
−1 1 1+ 4
2x2
J (x1 , x2 ) = cos x1
det(J) 4
10x1
Once these have been computed we get the iterative scheme
" # " #
(k+1) (k)
x1 x1 (k) (k) (k) (k)
(k+1) = (k) − J −1 (x1 , x2 )F (x1 , x2 ) (1)
x2 ) x2

which is explicitly
" # " (k) #" #
(k+1) sin x (k) 2(k) 2(k)
x1 1 1 + 42 2x2 5x1 − x2
(k+1) = (k) (k) (k) (k) (k) (k)
x2 det(J(x1 , x2 ) cos x1
10x1
(k) x2 − 0.25(sin x1 + cos x2 )
4

where
sin x2 1
det(J) = 10x1 (1 + ) − x2 cos x1
4 2
So in an typical algorithm code you need to define the vector function F (x1 , x2 ), the Jacobian
J(x1 , x2 ) and then compute the inverse J −1 (x1 , x2 ) and then combine these in the iterative
scheme. The computation of the inverse may look complex (which it may), but fortunately the
matrix inverse is one of the built-in functions of Matlab/Octave. So we can take advantage of
this built-in function in our code.

The pseudo code in the textbook is based on rearranging the iterative scheme to the equiva-
lent form " # " #!
(k+1) (k)
(k) (k) x 1 x 1 (k) (k)
J(x1 , x2 ) (k+1) − (k) = −F (x1 , x2 ),
x2 x2
with the notation " #" # " #
(k) (k+1) (k)
y1 x1 x1
y (k) = (k) (k+1) − (k)
y2 x2 x2
or
y (k) = x(k+1) − x(k)
, from which the next iterate is computed as

x(k+1) = x(k) + y (k)

So the steps in the code are:

" #
(0)
x1
(i) Define (0) ; F (x1 , x2 ); and J(x1 , x2 ).
x2
(k) (k)
(ii) Evaluate F and J at (x1 , x2 )

20
 APM2613/202/0/2022

(iii) By letting y (k) = x(k+1) − x(k) , solve and compute solve the equation J −1 y = −F at
(k) (k)
(x1 , x2 )
(iv) Compute the next iterate as x(k+1) = x(k) + y (k)
(v) Repeat steps (ii)-(iv) using x(k+1) instead of x(k) .

Given below is a code for the iterative scheme in (1). The basic code (with no stopping
criterion) is:

% Define F and J and initialise x and max number of iterations N

F=inline(’[5*xˆ2-yˆ2; y-0.25*(sin(x)+cos(y))]’);
% Note the vector function within the quotes of the inline sttatement.
J=inline(’[10*x -2*y;-0.25*cos(x) 1+0.25*sin(y)]’);
x=[0;1]; N=2;
% Iterate
for k=1:N
x(:,k+1)=x(:,k)-inv(J(x(1,k),x(2,k)))*F(x(1,k),x(2,k))’
end
% Output
x

(4.3) The output obtained for the given x(0) = (0, 1)T and vector function F is:

k 0 1 2
(k)
x1 0 1.0390 0.5324
(k)
x2 1 0.50000.3845

A few more iterations (by adjusting N) yielded the following output for N = 9

(k) (k)
k x1 x2
0 0 1
1 1.0390 0.5000
2 0.5324 0.3845
3 0.2835 0.3118
4 0.1697 0.2828
5 0.1287 0.2729
6 0.1215 0.2712
7 0.1212 0.2711
8 0.1212 0.2711
9 0.1212 0.2711
from which we observe that Newton’s method converges to the approximate solution (x1 , x2 )T =
(0.1212, 0.2711)T correct to 4 decimal places after 8 iterations.
Slight modifications to the code to accommodate specifications of a specific accuracy can be
done by including an ’if...then’ or ’while ...’ conditional statements.

21