Chapter II

Numerical Solutions of Nonlinear

System of Equations

2.1 Fixed Points for Functions of Several Variables

A system of nonlinear equations has the form

f1 (x1 , x2 , ..., xn ) = 0,
f2 (x1 , x2 , ..., xn ) = 0,
.. . (2.1)
. = ..
fn (x1 , x2 , ..., xn ) = 0,

where each function fi can be thought of as mapping a vector x = (x1 , x2 , ..., xn )t of the n-dimensional space
Rn into the real line R.
This system of n nonlinear equations in n unknowns can also be represented by defining a function F mapping
Rn into Rn as

F (x1 , x2 , ..., xn ) = (f1 (x1 , x2 , ..., xn ), f2 (x1 , x2 , ..., xn ), ..., fn (x1 , x2 , ..., xn ))t .
If vector notation is used to represent the variables x1 , x2 , ..., xn , then system ( assumes the form If vector
notation is used to represent the variables x1 , x2 , ..., xn , then system 2.1 assumes the form

F (x) = 0. (2.2)

The functions f1 , f2 , ..., fn are called the coordinate functions of F.

Example 2.1.1 Place the 3 ◊ 3 nonlinear system

1
3x1 ≠ cos(x2 x3 ) ≠
=0
2
x21 ≠ 81(x2 + 0.1)2 + sin x3 + 1.06 = 0,
10fi ≠ 3
e≠x1 x2 + 20x3 + =0
3

in the form 2.2.

Define the three coordinate functions f1 , f2 , and f3 from R3 to R as

1
f1 (x1 , x2 , x3 ) = 3x1 ≠ cos(x2 x3 ) ≠ ,
2
f2 (x1 , x2 , x3 ) = x21 ≠ 81(x2 + 0.1)2 + sin x3 + 1.06,
10fi ≠ 3
f3 (x1 , x2 , x3 ) = e≠x1 x2 + 20x3 + .
3

15
16 CHAPTER II. NUMERICAL SOLUTIONS OF NONLINEAR SYSTEM OF EQUATIONS

Then define F from R3 æR3 by.

F (x) = F (x1 , x2 , x3 )
= (f1 (x1 , x2 , x3 ), f2 (x1 , x2 , x3 ), f3 (x1 , x2 , x3 ))t
3 4t
1 10fi ≠ 3
= 3x1 ≠ cos(x2 x3 ) ≠ , x21 ≠ 81(x2 + 0.1)2 + sin x3 + 1.06, e≠x1 x2 + 20x3 +
2 3

Theorem 2.1.1 Let f be a function from D µ Rn into R and x0 œ D. Suppose that all the partial derivatives
of f exist and constants ” > 0 and K > 0 exist so that whenever Îx ≠ x0 Î < ” and x œ D, we have
- -
- ˆf (x) -
- ˆxj - Æ K. for each j = 1, 2, ..., n.
- -

Then f is continuous at x0 .

2.1.1 Fixed Points in Rn

Definition 2.1.1 A function G from D µ Rn into Rn has a fixed point at p œ D if G(p) = p

The following theorem explains the Fixed-Point Theorem to the n-dimensional case. This theorem is a special
case of the Contraction Mapping Theorem.
transpose
Theorem 2.1.2 Let D = {(x1 , x2 , ..., xn )t|ai Æ xi Æ bi , for each i = 1, 2, ..., n} for some collection of constants
a1 , a2 , ..., an and b1 , b2 , ..., bn . Suppose G is a continuous function from D µ Rn into Rn with the property that
G(x) œ D whenever x œ D. Then G has a fixed point in D.
Moreover, suppose that all the component functions of G have continuous partial derivatives and a constant
K < 1 exists with
- -
- ˆgi (x) - K
- ˆxj - Æ n , whenever x œ D,
- -

for each j = 1, 2, ..., n and each component function gi . Then the sequence {x(k) }Œ
k=0 defined by an arbitrarily
selected x(0) in D and generated by

x(k) = G(x(k≠1) ), for each k Ø 1

converges to the unique fixed point p œ D
. . Kk . .
. (k) . . (1) .
.x ≠ p. Æ .x ≠ x(0) . .
Œ 1≠K Œ

Example 2.1.2 Place the nonlinear system

1
3x1 ≠ cos(x2 x3 ) ≠
=0
2
x21 ≠ 81(x2 + 0.1)2 + sin x3 + 1.06 = 0,
10fi ≠ 3
e≠x1 x2 + 20x3 + =0
3

in a fixed-point form x = G(x) by solving the ith equation for xi , show that there is a unique solution on

D = {(x1 , x2 , x3 )t | ≠ 1 Æ xi Æ 1, for each i = 1, 2, 3}.

and iterate starting with x(0) = (0.1, 0.1, ≠0.1)t .

Solution: Solving the ith equation for xi gives the fixed-point problem,
1 1
x1 = cos(x2 x3 ) + ,
3Ò 6
1
x2 = x21 + sin x3 + 1.06 ≠ 0.1,
9
10fi ≠ 3 1
x3 = ≠ ≠ e≠x1 x2 .
60 20
2.1. FIXED POINTS FOR FUNCTIONS OF SEVERAL VARIABLES 17

Let G : R3 æ R3 be defined by G(x) = (g1 (x), g2 (x), g3 (x))t , where

1 1
g1 (x1 , x2 , x3 ) = cos(x2 x3 ) + ,
3Ò 6
1 2
g2 (x1 , x2 , x3 ) = x1 + sin x3 + 1.06 ≠ 0.1,
9
10fi ≠ 3 1
g3 (x1 , x2 , x3 ) = ≠ ≠ e≠x1 x2 .
60 20

Theorems 2.1.1 and 2.1.2 show that there is a unique solution on

D = {(x1 , x2 , x3 )t | ≠ 1 Æ xi Æ 1, for each i = 1, 2, 3}.

Now for x = (x1 , x2 , x3 )t in D,
1 1
|g1 (x1 , x2 , x3 |) Æ | cos(x2 x3 )| + Æ 0.5,
3 6
Ò Ò
1 1
|g2 (x1 , x2 , x3 )| = | x21 + sin x3 + 1.06 ≠ 0.1| Æ x21 + sin 1 + 1.06 ≠ 0.1 < 0.09,
9 9
and
10fi ≠ 3 1 e 10fi ≠ 3
|g3 (x1 , x2 , x3 )| = ≠ e≠x1 x2 Æ + < 0.61.
60 20 20 60
So we have, for each i = 1, 2, 3,
≠1 Æ gi (x1 , x2 , x3 ) Æ 1.
Thus G(x) œ D whenever x œ D. Finding bounds for the partial derivatives on D gives
- - - - - -
- ˆg1 - - - - -
- - = 0, - ˆg2 - = 0, - ˆg3 - = 0,
- ˆx1 - - ˆx2 - - ˆx3 -

as well as,
- - - -
- ˆg1 - 1 1 - ˆg1 - 1 1
- ˆx2 - Æ 3 |x3 || sin x2 x3 | Æ 3 sin 1 < 0.281, - ˆx3 - Æ 3 |x2 || sin x2 x3 | Æ 3 sin1 < 0.281,
- - - -

- -
- ˆg2 - |x1 | 1
- ˆx1 - Æ 9x2 + sin x + 1.06 Æ 9Ô0.218 < 0.238,
- -
1 3
- -
- ˆg2 - | cos x3 | 1
- ˆx3 - Æ 18x2 + sin x + 1.06 Æ 18Ô0.218 < 0.119,
- -
1 3
- - - -
- ˆg3 - |x2 |e 1 2
≠x x
e - ˆg3 - |x1 |e≠x1 x2 e
- -= Æ < 0.14, and -
- -= Æ < 0.14.
- ˆx1 - 20 20 ˆx2 - 20 20
The partial derivatives of g1 , g2 , and g3 are all bounded on D, so Theorem 2.1.1 implies that these functions
are continuous on D. Consequently, G is continuous on D. Moreover, for every x œ D,
- -
- ˆgi -
- ˆxj - Æ 0.281 for each i = 1, 2, 3 and j = 1, 2, 3,
- -

and the condition in the second part of Theorem- 2.1.2- holds with K = 3(0.281) = 0.843.
- ˆgi -
In the same manner it can also be shown that -- - is continuous on D for each i = 1, 2, 3 and j = 1, 2, 3.
ˆxj -
Consequently, G has a unique fixed point in D, and the nonlinear system has a solution in D.

To approximate the fixed point p, we choose x(0) = (0.1, 0.1, ≠0.1)t . The sequence of vectors generated by

1 1 2 1
(k) (k≠1) (k≠1)
x1 = cos x2 x3 + ,
3 6
18 CHAPTER II. NUMERICAL SOLUTIONS OF NONLINEAR SYSTEM OF EQUATIONS
Ò
(k) 1 (k≠1) 2 (k≠1)
x2 = (x1 ) + sin x3 + 1.06 ≠ 0.1,
9
(k) 10fi ≠ 3 1 (k≠1) (k≠1)
x3 = ≠ ≠ e≠x1 x2
.
60 20

converges to the unique

. solution. .The results in following Table were generated until The results in Table 10.1
were generated until .x(k) ≠ x(k1) .Œ < 10≠5

We could use the error bound

. . Kk . .
. (k) . . (1) .
.x ≠ p. Æ .x ≠ x(0) . .
Œ 1≠K Œ

with K = 0.843. This gives

. . 0.8435
. (5) .
.x ≠ p. Æ 0.423 < 1.15, .
Œ 1 ≠ 0.843
which does not indicate the true accuracy of x(5) . The actual solution is
. .
. .
p = (0.5, 0, ≠fi/6)t ¥ (0.5, 0, 0.5235987757)t , so.x(5) ≠ p. Æ 2 ◊ 10≠8 .
Œ

2.1.2 Accelerating Convergence

(k)
One way to accelerate convergence of the fixed-point iteration is to use the latest estimates x1 , ..., xi≠1 (k)
(k≠1) (k≠1) (k)
instead of x1 , ..., xi≠1 to compute xi , as in the Gauss-Seidel method for linear systems. The component
equations for the problem in the example then become

1 1 2 1
(k) (k≠1) (k≠1)
x1 = cos x2 x3 + ,
3Ò 6
(k) 1 (k) 2 (k≠1)
x2 = (x1 ) + sin x3 + 1.06 ≠ 0.1,
9
(k) 10fi ≠ 3 1 (k) (k)
x3 =≠ ≠ e≠x1 x2 .
60 20

With x(0) = (0.1, 0.1, ≠0.1)t , the results of these calculations are listed in the following Table

Practice Exercises 2.1.1 • Show that the function F : R3 æ R3 defined by

F (x1 , x2 , x3 ) = (x1 + 2x3 , x1 cos x2 , x22 + x3 )t

is a continuous at each point of R3 .

2.2. NEWTON’S METHOD 19

• The nonlinear system

x21 ≠ 10x1 + x22 + 8 = 0, x1 x22 + x1 ≠ 10x2 + 8 = 0
can be transformed into the fixed-point problem

x21 + x22 + 8 x1 x22 + x1 + 8

x1 = g1 (x1 , x2 ) = , x2 = g2 (x1 , x2 ) = .
10 10
– Use Theorem 2.1.2 to show that G = (g1 , g2 )t mapping D µ R2 into R2 has a unique fixed point in

D = {(x1 , x2 )t |0 Æ x1 , x2 Æ 1.5}.

– Apply functional iteration to approximate the solution.

– Does the Gauss-Seidel method accelerate convergence?

2.2 Newton’s Method

In the last section we transformed non linear system of equations into a convergent fixed-point problem by
algebraically solving the equations for the each variables x1 , x2 , ...xn . It is, however, unusual to be able to find
an explicit representation for all the variables. In this section, we consider an algorithmic procedure to perform
the transformation in a more general situation.
To construct the algorithm that led to an appropriate fixed-point method in the one- dimensional case, we found
a function „ with the property that
g(x) = x ≠ „(x)f (x)
gives quadratic convergence to the fixed point p of the function g. From this condition Newton’s method evolved
by choosing „(x) = f Õ1(x) , assuming that f Õ (x) ”= 0.
A similar approach in the n-dimensional case involves a matrix.
S T
a11 (x) a12 (x) ... a1k (x)
Wa21 (x) a22 (x) ... a2k (x)X
W X
A(x) = W . .. .. X
U .. . . V
ak1 (x) ak2 (x) ... akk (x)
where each of the entries aij (x) is a function from Rn into R. This requires that A(x) be found so that

G(x) = x ≠ A(x)≠1 F (x)

gives quadratic convergence to the solution of F (x) = 0, assuming that A(x) is nonsingular at the fixed
point p of G.

Theorem 2.2.1 Let p be a solution of G(x) = x. Suppose a number ” > 0 exists with
ˆgi
• is continuous on N” = {x|Îx ≠ pÎ < ”}, for each i = 1, 2,...,n and j = 1,2,...,n;
ˆxj
- 2 -
ˆ 2 gi (x) - ˆ gi (x) -
• is continuous, and - - - Æ M for some constant M , whenever x œ N” , for each i =
ˆxj ˆxk ˆxj k -
1, 2, ..., n, j = 1, 2, ..., n, and k = 1, 2, ..., n;
ˆgi (x)
• = 0 for each i = 1, 2, ..., n and k = 1, 2, ..., n
ˆxk
Then a number ”ˆ Æ ” exists such that
. the sequence
. generated by x(k) = G(x(k≠1) ) converges quadratically to p
(0) (0)
for any choice of x , provided that .x ≠ p. < ”. ˆ Moreover,

. . n2 M . .2
. (k) . . (k≠1) .
.x ≠ p. Æ .x ≠ p. , for each k Ø 1.
Œ 2 Œ

To apply Theorem 2.2.1, suppose that A(x) is an n ◊ n matrix of functions from Rn into R in the form of above
matric form.
Assume, moreover, that A(x) is nonsingular near a solution p of F (x) = 0, and let bij (x) denote the entry of
A(x)≠1 in the ith row and j th column.
20 CHAPTER II. NUMERICAL SOLUTIONS OF NONLINEAR SYSTEM OF EQUATIONS

n
ÿ
For G(x) = x ≠ A(x)≠1 F (x), we have gi (x) = xi ≠ bij (x)fj (x).
j=1
Y n 3 4
_
_
ÿ ˆfj (x) ˆbij (x)
_1 ≠
_ bij (x) + fj (x) if i = k
ˆgi (x) ] ˆxk ˆxk
= j=1
n 3 4
ˆxk _
_ ÿ ˆfj (x) ˆbij (x)
_
_
[≠ b ij (x) + fj (x) if i ”= k
j=1
ˆxk ˆxk
ˆgi (p)
Theorem 2.2.1 implies that we need = 0, for each i = 1, 2, ..., n and k = 1, 2, ..., n. This means that for
ˆxk
i = k,

ˆfj (p)
n
ÿ
1≠ bij (p)
j=1
ˆxi

i.e.
ˆfj (p)
n
ÿ
bij (p) = 1. (2.3)
j=1
ˆxi

when k ”= i,
ˆfj (p)
n
ÿ
≠ bij (p) = 0.
j=1
ˆxk
so,
ˆfj (p)
n
ÿ
bij (p) = 0. (2.4)
j=1
ˆxk

2.2.1 The Jacobian Matrix

Define the matrix J(x) by
S T
ˆf1 (x) ˆf1 (x) ˆf1 (x)
W ...
W ˆx1 ˆx2 ˆxn X
W ˆf2 (x) ˆf2 (x) ˆf2 (x) X
X
W ... X
J(x) = W ˆx1 ˆx2 ˆxn X
W
W .. .. .. X X
W . . . X
U ˆf (x) ˆfn (x) ˆf (x) V
(x)
n n
...
ˆx1 ˆx2 ˆxn
Now by conditions 2.3 and 2.4 require that

A(p)≠1 J(p) = I, the identity matrix, so A(p) = J(p).

An appropriate choice for A(x) is, consequently, A(x) = J(x) since this satisfies condition III in Theorem2.2.1.
The function G is defined by
G(x) = x ≠ J(x)≠1 F (x),
and the functional iteration procedure evolves from selecting x(0) and generating, for kgeq1,

x(k) = G(x(k≠1) ) = x(k≠1) ≠ J(x(k≠1) )≠1 F (x(k≠1) ).

This is called Newton’s method for nonlinear systems, and it is generally expected to give quadratic convergence,
provided that a sufficiently accurate starting value is known and that J(p)≠1 exists. The matrix J(x) is called
the Jacobian matrix and has a number of applications in analysis.

Example 2.2.1 Apply Newton’s method to this problem with x(0) = (0.1, 0.1, ≠0.1)t

1
3x1 ≠ cos(x2 x3 ) ≠
=0
2
x21 ≠ 81(x2 + 0.1)2 + sin x3 + 1.06 = 0,
10fi ≠ 3
e≠x1 x2 + 20x3 + =0
3
2.2. NEWTON’S METHOD 21

Define
F (x) = (f1 (x), f2 (x), f3 (x))t ,
where,
1
f1 (x1 , x2 , x3 ) = 3x1 ≠ cos(x2 x3 ) ≠ ,
2
f2 (x1 , x2 , x3 ) = x21 ≠ 81(x2 + 0.1)2 + sin x3 + 1.06,
10fi ≠ 3
f3 (x1 , x2 , x3 ) = e≠x1 x2 + 20x3 + .
3

The Jacobian matrix J(x) for this system is

S T
3 x3 sin x2 x3 x2 sin x2 x3
J(x1 , x2 , x3 ) = U 2x1 ≠162(x2 + 0.1) cos(x3 ) V
≠x2 e≠x1 x2 ≠x1 e ≠x1 x2
20

Let x(0) = (0.1, 0.1, ≠0.1)t , then F (x0 ) = (≠0.199995, ≠2.269833417, 8.462025346) and,
S T
3 9.999833334 ◊ 10≠4 9.999833334 ◊ 10≠4
J(x(0) ) = U 0.2 ≠32.4 0.9950041653 V
≠0.09900498337 ≠0.09900498337 20

Solving the linear system, J(x(0) )y (0) = F (x(0) ) gives

S T
0.3998696728
y (0) = U≠0.08053315147V
≠0.4215204718

and S T
0.4998696782
x(1) = x(0) + y (0) = U0.01946684853V
0.5215204718
Continuing for k = 2, 3, ..., we have
S (k) T S (k≠1) T S (k≠1) T
x x y
W 1(k) X W 1(k≠1) X W 1(k≠1) X
Ux2 V = Ux2 V + Uy2 V.
(k) (k≠1) (k≠1)
x3 x3 y3
where,
S (k≠1) T
y
W 1(k≠1) X (k≠1) (k≠1) (k≠1) ≠1 (k≠1) (k≠1) (k≠1)
Uy2 V = ≠[J(x1 , x2 , x3 )] F (x1 , x2 , x3 )
(k≠1)
y3

Thus, at the kth step, the linear system J(x(k≠1) )y (k≠1) = F (x(k≠1) ) must be solved, where
S (k≠1) (k≠1) (k≠1) (k≠1) (k≠1) (k≠1)
T
3 x3 sin x2 x3 x2 sin x2 x3
W (k≠1) (k≠1) (k≠1) X
J(x(k≠1) ) = U 2x1 ≠162(x2 + 0.1) cos x3 V
(k≠1) ≠x (k≠1) (k≠1) (k≠1) (k≠1)
≠x2 e 1 x2
≠x1 e≠x1 x2
20
S (k≠1) T
y
W 1 X
y (k≠1) = Uy2(k≠1) V .
(k≠1)
y3
Hence,
S 1 2 T
(k≠1) (k≠1) (k≠1) 1
3x1 ≠ cos x2 x3 ≠
2
W X
F (x(k≠1) ) = W (k≠1) 2
U(x1 ) ≠ 81(x2
(k≠1) (k≠1)
+ 0.1)2 + sin x3
X
+ 1.06,V
(k≠1) (k≠1) (k≠1)
e≠x1 x2
+ 20x3 + 10fi≠3
3
22 CHAPTER II. NUMERICAL SOLUTIONS OF NONLINEAR SYSTEM OF EQUATIONS

The results using this iterative procedure are shown in Table

The previous example illustrates that Newton’s method can converge very rapidly once a good approximation
is obtained that is near the true solution
 x"
(0) [8]
#(2 )
&
%
= =

-(1)
[ -] => = =- 20]
-

al =

%F (ii) <
-

yo =

X y
[ 4)
↑
x
=

~ F-fon?) caly-[ * For . * 29s]

Jin")
Fog z
0127444]
-0 :
=

-oresses

I
free sI
-

y I(n!
GE(x))
)
=
=

⑪
X y) +

[rassics
4958936

I
=
0 .