Ziane Achour University of Djelfa Course: Numerical Methods

Faculty of Science and Technology Instructor: Dr. Ali TETA

Department of Electrical Engineering L02 ELT-AUT

Chapter 01: Root of Equations

1.1 Objectives
1- Understanding fundamental concepts of linear and nonlinear equations including variables,
coefficients, and constants.
2- Applying solution methods.
I.2 Bisection method (Dichotomy)
I.2.1 Principle
We consider a function f continuous in the interval [a,b]. We assume that f admits a single root p in the
interval [a,b], ((a).f(b)<0). We pose:
𝑎+𝑏
𝑐= (1.1)
2
- if f(c)=0, then p=c (root of f)
- if f(c)≠0 :
 if f(a).f(c)<0, then p ∈ [a,c[
 if f(c).f(b)<0, then p ∈ ]c,b]

I.2.2 Stopping criterion

There are two common used errors –absolute error and relative error- as follows:
ε𝑎 = |X𝑛∗1 − X𝑛 | (1.2)
|X𝑛∗1 −X𝑛 |
ε𝑟 = |X𝑛∗1 |
(1.3)

For the value of ‘c’ at the nth iteration to be an approximate value of ‘p’ up to (a tolerance) ε>0, it is
enough for ‘n’ to verify:
𝑏−𝑎
≤ ε𝑎 (1.4)
2𝑛
We have thus:
𝑏−𝑎
|𝑝 − 𝑐𝑛 | ≤ ≤ ε𝑎 (1.3)
2𝑛
This makes it possible to calculate in advance the maximum number of iterations ‘n’ ensuring the
absolute precision ε𝑎
𝑏−𝑎
ln( )
ε𝑎
𝑛≥ (1.5)
ln(2)

1
Ziane Achour University of Djelfa Course: Numerical Methods
Faculty of Science and Technology Instructor: Dr. Ali TETA
Department of Electrical Engineering L02 ELT-AUT

Figure 01: Bisection method

I.3.2 Example 01:
We consider the following continuous function 𝑓(𝑥) = 𝑒 𝑥 + 3𝑥 − 2 in the interval [0, 1].
We use the Bisection method to estimate the root ’p’ performing 5 iterations:
iteration a c b f(a) f(c) f(b)
1 0 0.5 1 -1 1.1487 3.7183
2 0 0.25 0.5 -1 0.0340 1.1487
3 0 0.125 0.25 -1 -0.4919 0.0340
4 0.125 0.1875 0.25 -0.4919 -0.2313 0.0340
5 0.1875 0.2188 0.25 -0.2313 -0.0990 0.0340

1.4 Fixed point method

I.4.1 Principle
The principle of this method consists of transforming the equation f(x)=0 into an equivalent equation
g(x)=x, where 𝑔 is a well-chosen auxiliary function.
The choice of the function 𝑔 is motivated by the requirements of the fixed point theorem. Indeed, it
must be contracting in a neighborhood I of p, which amounts to verifying that |𝑔(𝑥)′| < 1 in this
neighborhood. So, we construct a series (xn) ∈ N defined by:
𝑥0 ∈ [𝑎, 𝑏]
{ 𝑛≥0 (1.7)
𝑥𝑛+1 = 𝑔(𝑥𝑛 )
- Definition: If a function 𝑔 defined in [a,b] and if there exists 𝑥 ∗ ∈ [𝑎, 𝑏] such that 𝑥 ∗ = 𝑔(𝑥 ∗ ) then
𝑥 ∗ is said to be the fixed point of the function 𝑔 and it the root of f(x) .

I.4.2 Example 02:

Consider the equation 𝑓(𝑥) = 𝑥 2 − 2𝑥 − 3 = 0, with 𝑥 ∈ ℝ. There are several ways to transform the
equation f into the form 𝑔(𝑥) = 𝑥.

2
Ziane Achour University of Djelfa Course: Numerical Methods
Faculty of Science and Technology Instructor: Dr. Ali TETA
Department of Electrical Engineering L02 ELT-AUT

For example we can obtain:

3
a- 𝑥 = 𝑔1 (𝑥) = 𝑥−2
b- 𝑥 = 𝑔2 (𝑥) = √2𝑥 + 3
𝑥 2 −3
c- 𝑥 = 𝑔3 (𝑥) = 2

We use the fixed point method to estimate the root of the equation f with an initial point 𝑥0 = 4
𝑥𝑛 (iteration n), 𝑥𝑛+1 = 𝑔1 (𝑥𝑛 ) 𝑥𝑛+1 = 𝑔2 (𝑥𝑛 ) 𝑥𝑛+1 = 𝑔3 (𝑥𝑛 )
𝑥0 4 4 4
𝑥1 1.5 3.316 6.5
𝑥2 -6 3.103 19.625
𝑥3 -0.375 3.034 191.070
𝑥4 -1.263 3.011 1.82 104
𝑥5 -0.919 3.003 1.66 108
Convergent Convergent Divergent

We conclude that the choice of the function g(x) is important for the convergence of the iterative process.

(a) (b)
Figure 02: (a) Attractive fixed point (b) Repulsive fixed point

I.4.3 Convergence study

The convergence of the fixed point method is verified if the following two conditions are satisfied:
Stability: 𝑔(𝑥) ∈ [𝑎, 𝑏] for all 𝑥 ∈ [𝑎, 𝑏]
Contraction: There exists 𝑘 ∈ [0,1[ such that |𝑔(𝑥) − 𝑔(𝑦)| ≤ 𝑘|𝑥 − 𝑦| for all 𝑥, 𝑦 ∈ [𝑎, 𝑏]
Or |𝑔′(𝑥)| ≤ 𝑘 < 1 and 𝑘 = 𝑚𝑎𝑥𝑥∈𝐼 |𝑔′(𝑥)|
Therefore, 𝑔(𝑥) admits a unique fixed point 𝑥 ∗ in [a,b] and the series 𝑥𝑛+1 = 𝑔(𝑥𝑛 ) converges to
𝑥 ∗ for an initial point 𝑥0 ∈ [𝑎, 𝑏].

3
Ziane Achour University of Djelfa Course: Numerical Methods
Faculty of Science and Technology Instructor: Dr. Ali TETA
Department of Electrical Engineering L02 ELT-AUT

I.3.2 Stopping criterion

We can use the following criterion to calculate the number of sufficient iterations:
ε𝑎 (1− 𝑘0 )
𝑙𝑛( |𝑥 |
)
1 −𝑥0
𝑛≥ (1.8)
ln 𝑘0
With, ε𝑎 is the absolute tolerance and 𝑘0 = |𝑔′( 𝑥0 )|

I.3.3 Example 03
We try to choose the suitable auxiliary function 𝑔 for the function f in the previous example whose fixed
point method will be convergent, the considered interval is I=[0, 4].
3
1- The auxiliary function 𝑔1 (𝑥) = 𝑥−2
a- Stability condition: We cannot choose this function because it is not continuous in the
interval I, i.e. the condition of 𝑔1 (𝑥) ∈ 𝐼 for all 𝑥 ∈ 𝐼 is not satisfied. Therefore, 𝑔1 (𝑥) is not
stable.
2- The auxiliary function 𝑔2 (𝑥) = √2𝑥 + 3
a- Stability condition: We have 𝑔2 (0) = √3 = 1.732 and 𝑔2 (4) = √11 = 3.316, and
1
𝑔2′ (𝑥) = > 0 for all 𝑥 ∈ 𝐼. Therefore, 𝑔2 (𝑥) is stable.
√2𝑥+3
1
b- Contraction condition: |𝑔2′ (𝑥)| = | | < 0.577 < 1, for all 𝑥 ∈ 𝐼. Therefore, 𝑔2 (𝑥) is
√2𝑥+3
contractive.
𝑥 2 −3
3- The auxiliary function 𝑔3 (𝑥) = 2
−3 13
a- Stability condition: We have 𝑔3 (0) = 2 = −1.5 ∉ 𝐼 and 𝑔2 (4) = 2 = 6.5 ∉ 𝐼.
Therefore, 𝑔3 (𝑥) is not stable.
We notice that only the function 𝑔2 satisfies the two convergence conditions for the interval I.
Therefore, we choose 𝑥0 = 2
𝑥1 = 𝑔2 ( 𝑥0 ) = 2.645
We use the following stopping criterion to calculate the number of iterations with a tolerance ε =
0.01%
ε(1 − 𝑘0 ) 0.01(1 − 0.37)
𝑙𝑛 ( ) 𝑙𝑛 ( )
|𝑥1 − 𝑥0 | |2.645 − 2|
𝑛≥ ⇔𝑛≥
ln( 𝑘0 ) ln(0.37)
0.01(1 − 0.37)
𝑙𝑛 ( )
|2.645 − 2|
𝑛≥ = 4.65 ≈ 5
ln(0.37)
Therefore, the number of sufficient iteration is 5.

4
Ziane Achour University of Djelfa Course: Numerical Methods
Faculty of Science and Technology Instructor: Dr. Ali TETA
Department of Electrical Engineering L02 ELT-AUT

Iteration 𝑥𝑛 𝑥𝑛+1 = √2𝑥𝑛 + 3 ε = |𝑥𝑛+1 − 𝑥𝑛 |

0 𝑥0 2 /
1 𝑥1 2.6458 0.6458
2 𝑥2 2.8795 0.2337
3 𝑥3 2.9596 0.0801
4 𝑥4 2.9865 0.0269
5 𝑥5 2.9955 0.009

1.5 Newton-Raphson method:

I.4.1 Principle
Let 𝑓 ∈ 𝐶 1 [𝑎, 𝑏], such that 𝑓′(𝑥) ≠ 0 for all 𝑥 ∈ [𝑎, 𝑏]. We assume that the equation 𝑓(𝑥) = 0 admits
a single solution in the interval [𝑎, 𝑏].
Starting from a point 𝑥0 , we trace the tangent of the curve of 𝑓 in (𝑥0 , 𝑓(𝑥0 )). The intersection of this
tangent with the x-axis gives us the point 𝑥1 . The new point 𝑥1 obtained is thus closer to the root of the
function and we repeat the operation until the desired precision is reached.

Figure 03: Newton-Raphson method

The principle of the Newton-Raphson method consists of replacing the nonlinear problem 𝑓(𝑥) = 0by
an affine problem 𝑔(𝑥) = 0, where 𝑔 is the "best affine approximation" of f in the neighborhood of a
given 𝑥0 . We identify the representation of 𝑔 with the tangent to the curve of f at the abscissa point in
the vicinity of 𝑥0 . Using, Taylor's formula we have
(𝑥− 𝑥0 )2
𝑓(𝑥) = 𝑓( 𝑥0 ) + (𝑥 − 𝑥0 )𝑓 ′ ( 𝑥0 ) + 𝑓 ′′ ( 𝑥0 ). (1.9)
2
We suppose that:
𝑔(𝑥) = 𝑓( 𝑥0 ) + (𝑥 − 𝑥0 )𝑓 ′ ( 𝑥0 ) (1.10)
We define 𝑥1 such that 𝑔(𝑥1 )=0.
Note that 𝑥1 is well defined when 𝑓 ′ ( 𝑥0 ) ≠ 0 is given by:
𝑓( 𝑥 )
𝑥1 = 𝑥0 − 𝑓′ ( 𝑥0 ) (1.11)
0
We thus construct, provided that 𝑥𝑛 ∈ [𝑎, 𝑏], the sequence as follows:

5
Ziane Achour University of Djelfa Course: Numerical Methods
Faculty of Science and Technology Instructor: Dr. Ali TETA
Department of Electrical Engineering L02 ELT-AUT

𝑥0 ∈ [𝑎, 𝑏] 𝑔𝑖𝑣𝑒𝑛
{ 𝑓( 𝑥𝑛 ) (1.12)
𝑥𝑛+1 = 𝑥𝑛 − , 𝑛∈ℕ
𝑓 ′ ( 𝑥𝑛 )
I.4.2 Study of convergence:
Consider 𝑓 a function of 𝐶 2 class in the interval [a, b], such that:
1- 𝑓(𝑎). 𝑓(𝑏) < 0;
2- 𝑓 ′ (𝑥) is not null in [a, b].
3- 𝑓( 𝑥0 ). 𝑓′′( 𝑥0 ) ≥ 0 such that 𝑥0 ∈ [𝑎, 𝑏]
Therefore, Newton-Raphson series:
𝑓( 𝑥 )
𝑥𝑛+1 = 𝑥𝑛 − 𝑓′ ( 𝑥𝑛 ) , 𝑛 ∈ ℕ (1.13)
𝑛
Converges for the chosen 𝑥0 toward the unique solution 𝑥 ∗ of 𝑓(𝑥)

I.4.3 Example: We solve the equation 𝑓(𝑥) = 𝑥 3 − 2𝑥 − 5 = 0, 𝐼 = [2,3], using Newton-raphson

method performing 3 iterations

a- We have 𝑓(2) = −1, 𝑓(3) = 16 therefore𝑓(2). 𝑓(3) < 0, there exists 𝑥 ∗ ∈ [2, 3] such that
𝑓(𝑥 ∗ ) = 0
b- In addition, 𝑓 ′ (𝑥) = 3𝑥 2 − 2, for 𝑓 ′ (𝑥) = 0 ⇒ 𝑥 = ±√2/3 ∉ [2, 3] therefore 𝑓 ′ (𝑥) ≠ 0 for all 𝑥 ∈
[2, 3]
and 𝑓 ′′ (𝑥) = 6𝑥 > 0 for all 𝑥 ∈ [2, 3], thus according to the convergence theorem the Newton-
Raphson sequence converges
𝑥0 ∈ [𝑎, 𝑏] 𝑔𝑖𝑣𝑒𝑛 𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡 𝑓( 𝑥0 ). 𝑓 ′ ′( 𝑥0 ) ≥ 0
{ 𝑓( 𝑥 ) (1.14)
𝑥𝑛+1 = 𝑥𝑛 − 𝑓′ ( 𝑥𝑛 ) , 𝑛 ∈ ℕ
𝑛
Given that 𝑓 ′′ (𝑥) > 0 for all 𝑥 ∈ [2, 3], it is enough to choose an 𝑥0 for which 𝑓( 𝑥0 ) > 0, let’s
say 𝑥0 = 3.
We perform 3 iteration using Newton-Raphson method:

iteration 𝑓( 𝑥𝑛 )
𝑥𝑛+1 = 𝑥𝑛 −
𝑓 ′ ( 𝑥𝑛 )
0 𝑥0 = 3
1(n=0) 16
𝑥1 = 3 − = 2.36
25
2(n=1) 𝑥2 = 2.127
3(n=2) 𝑥3 = 2.095