Solving Nonlinear Equations

Root r
 Fixed-point Iteration Method
 Rearrange f(x)=0 into an equivalent form x=g(x)
f(x) = x – g(x) = 0

 If r is a root of f, then f(r) = r-g(r) = 0

r=g(r)

 In iterative form,

Numerical Methods © Wen-Chieh Lin 2

 Fixed-point Iteration Method (Cont.)
 Also called function iteration

 For given equation f(x)=0, there may be many

equivalent fixed-point problems x=g(x) with
different choice of g

 Will the method always converge?

Numerical Methods © Wen-Chieh Lin 3

Example: Fixed-point Iteration

Numerical Methods © Wen-Chieh Lin 4

Example: Fixed-point Iteration (cont.)

e!
v e rg
D i

Numerical Methods © Wen-Chieh Lin 5

 Convergence Rate
 For general iterative methods, define error at
iteration n by
en = x n – R
where xn is an approximate solution and R is
the true solution

 For methods that maintain an interval known

to contain a solution, rather than a specific
approximate value for the solution, take the
error to be length of the interval containing
solution
Numerical Methods © Wen-Chieh Lin 6
 Convergence Rate (cont.)
 Sequence converges with rate r if

for some finite nonzero constant C

https://arnold.hosted.uark.edu/NA/Pages/OrderConv.pdf

 Some cases of interest

 r=1 and C<1: linear
 r>1: superlinear
 r=2: quadratic
Numerical Methods © Wen-Chieh Lin 7
 Convergence Rate of Bisection
 Length of interval containing solution reduced
by half at each iteration

 Linearly convergent
 r=1
 C = 0.5

Numerical Methods © Wen-Chieh Lin 8

 Convergence of Fixed-point Iteration
 If R = g(R) and |g’(R)| < 1, then there is an
interval containing R such that the iteration
xn+1 = g(xn)
converges to R if starting within that interval

 If |g’(R)| > 1, then the iterative scheme diverges

Numerical Methods © Wen-Chieh Lin 9

Proof of Convergent Condition

Mean Value Theorem

Fixed-point iteration is linearly convergent

If <1
Numerical Methods © Wen-Chieh Lin 10
 Convergence of Newton’s Method
 Represent Newton’s method in fixed-point
iteration form

 Condition for convergence |g’(R)|<1, where

Numerical Methods © Wen-Chieh Lin 11

 Let’s first check if |g’(R)| < 1

 Yes, if

Numerical Methods © Wen-Chieh Lin 12

Convergence rate of Newton’s method

The above can be derived as follows

𝜉 ∈ [ 𝑥𝑛 , 𝑅 ]

Recall that
For simple roots, Newton’s method
is quadratically convergent!
Numerical Methods © Wen-Chieh Lin 13
Newton’s Methods for Multiple Roots
 Quadratically convergent for simple roots,

 Linearly convergent for multiple roots as

Cannot be omitted

𝑒 𝑛+1=𝑔 ′ (𝜉 ) 𝑒𝑛
Numerical Methods © Wen-Chieh Lin 14
 Remedies for Multiple Roots with
Newton’s method
 If f(x) has a root of multiplicity k at x=R, we
can factor out (x-R)k from f(x) to get

 With a slightly modified Newton’s method

 It can be proved that and

Newton’s method still converges quadratically

Numerical Methods © Wen-Chieh Lin 15

Proof by yourself that

Numerical Methods © Wen-Chieh Lin 16

 Remedies for Multiple Roots with
Newton’s method (cont.)
 In practice, we don’t know k in advance
 Remedies
 1. Try and error
 2. Deflate f(x)
f(x)/(x-s) where s is an approximate R
 Be warned that an indeterminate form at x=R is created
 3. Acton’s method, see textbook

Numerical Methods © Wen-Chieh Lin 17

 Example: Newton’s Method for Finding
Complex Roots
 f(x) = x3 + 2x2 – x + 5

Numerical Methods © Wen-Chieh Lin 18

 Start Newton’s method with a complex value

Numerical Methods © Wen-Chieh Lin 19

 Systems of Nonlinear Equations
 Solving systems of nonlinear equations is much
more difficult than the scalar case because
 Wide variety of behavior—determining existence and
the number of solutions or good starting guess is
much more complex
 No simple way to guarantee convergence to the
desired solution or to bracket solution to produce a
absolutely safe method
 Computational overhead increases rapidly with the
dimension of the problem
Numerical Methods © Wen-Chieh Lin 20
Example: Systems in Two Dimensions

From Michael T. Heath

Numerical Methods © Wen-Chieh Lin 21
 Newton’s Method
 In n dimensions, Newton’s method has form

where J(x) is the Jacobian matrix of f

 In practice, we do not explicitly invert J(xn),

but instead solve linear system

for Newton step sn, then take as next iterate

Numerical Methods © Wen-Chieh Lin 22

 Side Note: Jacobian

 F(u,v) = [x, y, z]
 x(u,v) = u
 y(u,v) = v
 z(u,v) = f(u,v)

Numerical Methods © Wen-Chieh Lin 23

Example: Newton’s Method

Numerical Methods © Wen-Chieh Lin 24

Example: Newton’s Method

Numerical Methods © Wen-Chieh Lin 25

 Fixed-Point Iteration
 Fixed-point problem for is to find
vector x such that
 Corresponding fixed-point iteration is

 Converges if
 starts close enough to solution


Numerical Methods © Wen-Chieh Lin 26

 Fixed-Point Iteration (cont.)
 Convergence rate is normally linear, with
constant C given by

 If then convergence rate is at least

quadratic, e.g., Newton’s method

Numerical Methods © Wen-Chieh Lin 27

Example: Fixed-point Iteration

1.00000000000000 -1.73205080756888

1.00505253874238 -1.72912388057290

1.00398063482304 -1.72974647995027

1.00420874039761 -1.72961406264780
Numerical Methods © Wen-Chieh Lin 28