ME 310

Numerical Methods

2. Finding Roots of Nonlinear Equations

These presentations are prepared by Dr. Cüneyt Sert and modified by Dr. Sezer Özerinç.

They cannot be used and/or modified without the permission of the authors.

Fall 2015

Mechanical Engineering Department

Middle East Technical University

Ankara, Turkey

csert@metu.edu.tr ozerinc@metu.edu.tr
Finding Roots of Equations
f(x)

f(x) is given

x f(x r) = 0  xr = ?

• Bracketing Methods: Need two initial estimates that bracket the root. Always
converge. (Chapter 5)

• Bisection Method

• False-Position Method

• Open Methods: Need one or two initial estimates. May diverge. (Chapter 6)

• Simple One-Point Iteration

• Newton-Raphson Method (Needs the derivative of the function)

• Secant Method
 General Idea of Bracketing Methods (Chapter 5)
f(x) f(x) If f(x L)*f(x U) < 0
xL xL There are odd number of
xU x xU x
roots.

f(x) f(x)
If (x L)*f(x U) > 0
There are even number
xL xU x xL xU x of roots (or no root).

f(x) f(x) Violations:

xU xL xU i) multiple roots
xL x x ii) discontinuities
 Bracketing Methods: Bisection Method
f(x) 1) Start with two initial guesses:

xL xLOWER and xUPPER .

xU x 2) They should bracket the root, i.e.

f(xL) * f(xU) < 0

3) Estimate the root as the midpoint of this

f(x) interval: x = (x L + x U) / 2
xL
4) Determine the interval which contains the root
x xU x if f(x L) * f(x) < 0 root is between x L and x
else root is between x and x U

5) Estimate a new root as the midpoint of the new

f(x)
interval (repeat step 3)
xL x
6) Continue iterations and compute the error after
xU x
each iteration.

7) Stop when the specified tolerance is reached.

Bisection Method (continued)
Example 1: Find the square root of 11.
x2 = 11  f(x) = x2 – 11 (the exact solution is 3.31662479)
Select initial guesses: f(3) = -2, f(4) = 5  xL = 3, xU = 4.

Iteration
xL xU x f(x) |e t| % |e a| %
number

1 3 4 3.5 1.25 5.53 -----

6
Bisection Method (continued)
Figure from Chapra and Canale, 2010
• True error does not drop
monotonically but oscillates.

• ea > et at each step, which is good. It

means using ea is conservative.

• ea can also be estimated as:

ea = (xU – xL) / (xU + xL)
which can also be used for the 1st
iteration.
• Bisection method always converges to
the intended root but note the
following:

• f(xL) * f(xU) < 0 is true if the interval

has odd number of roots, not
necessarily one root.
Bracketing Methods: False-Position Method
1) Start with initial guesses, xLOWER and xUPPER .
f(x)
xL 2) They should bracket the root, i.e.

xU x f(xL) * f(xU) < 0.

3) Estimate the root using similar triangles.

f ( xU )( xL  xU )
x  xU 
f(x) f ( xL )  f ( xU )
xL 4) Determine the interval which contains the
x xU x root
if f(x L)*f(x) < 0 root is between xL and x
else root is between x and xU
f(x) 5) Estimate a new root in this new interval
xL
6) Iterate and compute the error after each
x xU x
iteration.

7) Stop when the specified tolerance is reached.

False-Position Method (continued)
Example 2: Repeat the previous example: x2 = 11  f(x) = x2 – 11
(the exact solution is 3.31662479)
Select initial guesses: f(3) = –2, f(4) = 5  xL = 3, xU = 4

Iteration
xL xU x f(x) |e t| % |e a| %
no.

1 3 4 3.28571429 -0.1040816 0.932 -----

6
False-Position Method (continued)
Figure from Chapra and Canale, 2010

• Generally converges faster than the

bisection method (see next slide for
an exception).

• ea > et at each step, which is good.

It means ea is conservative.

• False-position method always

converges to the true root but note
the following:

• f(xL) * f(xU) < 0 is true if the

interval has odd number of roots, not
necessarily one root.
False-Position Method (continued)
Figure and Example from Chapra and Canale, 2010

• False-position might converge slower

than Bisection in some cases.

• Example 3: Use false-position method

to locate the root of f(x) = x10 – 1.

xr ε t (%)
Iter.
False- False-
Bisection Bisection
Position Position

1 0.65 0.09430 35 90.6

2 0.975 0.18176 2.5 81.8

3 1.1375 0.26287 13.8 73.7

4 1.05625 0.33811 5.6 66.2

5 1.015625 0.40788 1.6 59.2

 Bracketing Methods – General Comments f ( x )  sin(10 x )  cos(3x )

• A plot of the function is always helpful to determine:

• the number of roots and initial guesses.

• whether the roots are multiple or not.

• whether the method converges to the desired root.

• Incremental search techniques can be used to

determine the initial guesses.

• Start from one end of the region of interest.

• Evaluate the function at specified intervals.

• If the sign of the function changes, then there is a root in

that interval.

• Select your intervals small, otherwise you may miss some

of the roots. But if they are too small, incremental search
might become time consuming.

• Incremental search, just by itself, can be used as a root

finding technique with very small intervals (not efficient).
Figures from Chapra and Canale, 2010
Bracketing Methods – A Fortran Code
PROGRAM BISECTION ! Calculates the root of a function

INTEGER :: iter, maxiter

REAL(8) :: xL, xU, x, fxL, fxU, fx, tol, error
READ(*,*) xL, xU, tol, maxiter

DO iter = 1, maxiter
x = (xL + xU) / 2 ! Only this line changes for the False-Position Method
fx = FUNC (x) ! Call a subroutine to calculate the function.
error = (xU - xL) / (xU + xL)*100 ! See page 121
WRITE(*,*) iter, x, fx, error
IF (error < tol) STOP

fxL = FUNC (xL)

fxU = FUNC (xU) Exercise: Write a MATLAB program for the Bisection
IF(fxL * fx < 0) THEN method and implement the following improvements.
xU = x
ELSE
• Check if the initial guesses bracket the root or not.
xL = x • Read the true value if it is known and calculate e t in
ENDIF
addition to e a.
ENDDO
END PROGRAM BISECTION
Open Methods (Chapter 6)

• Open methods have

the advantage of
converging very
rapidly.
• But they also have
the risk of diverging.

• Remember:
Bracketing methods
always converge.

Figures from Chapra and Canale, 2010

Open Methods: Simple One-Point Iteration
• Put the original formulation of f(x) = 0 into a form of x = g(x).

• There can be different g(x) for the same f(x):

ln(x) – 3x + 5 = 0  x = [ ln(x) + 5 ] / 3 or x = e3x - 5
cos(x) = 0  x = x + cos(x)

1) Start with an initial guess x0 Figure from Chapra and Canale, 2010

2) Calculate a new estimate for the root

using x1 = g(x0)
3) Iterate like this. General formula is
xi+1 = g(xi)

• Converges if |g(x)| < 1 in the

region of interest (easier to see
graphically in the coming slides).
Simple One-Point Iteration (continued)
Example 4: Repeat the previous example: f(x) = x2 – 11.
Start with the initial guess, x0 = 3.

(a) x2 – 11 = 0  x= x + x2 – 11 (b) 11 – x2 = 0  x= (4x + 11 – x2) / 4

i xi x i+1=g(xi) |e t| % i xi x i+1=g(x i) |e t| %

0 3 1 70 0 3 3.5 5.53

1 1

2 2

3 3

• Selection of g(x) is important. Note that we did not use the trivial one, x = sqrt(11).
• If the method converges, convergence is linear. That is, the relative error at each iteration is
roughly proportional to the half of the previous error. This is easier to see for et.
Simple One-Point Iteration (continued)
Monotone Spiral
• Graphical convergence convergence

illustration of the
stability criterion
|g(x)| < 1.

• Exercise: Show
that Example 4(a)
violates the
convergence criteria
Monotone Spiral
|g(x)| < 1. divergence divergence

Figure from Chapra and Canale

Open Methods: Newton-Raphson Method

• Start with initial guess xi.

Use the slope f’(xi) to obtain
a better guess.
f ( x0 )  0
f ( x0 ) 
x1  x0
f ( x0 )
x1  x0 
f ( x0 )
• Continue iterations until the
specified tolerance is reached.

Figure from Chapra and Canale, 2010 f ( xi )

xi 1  xi 
f ( xi )
Newton-Raphson Method (continued) f ( xi )
xi 1  xi 
Example 5: Repeat the previous example: f ( xi )
f(x) = x2 – 11. Start with the initial guess, x0 = 3.
f(x) = x2 – 11 = 0 , f’(x)= 2x  xi+1 = xi – (x2 – 11) / 2x

iteration x f(x) |e t| %

0 3 -2 9.55

Selection of the initial guess affects the convergence a lot.

Example 6: NR method is quite sensitive to the starting point. Try to find the first
positive root of sin(x), which is 3.14159, using the following initial guesses.
a) x0 = 1.5, b) x0 = 1.7, c) x0 = 1.8 , d) x0 = 1.9 They all converge to different roots.
Derivation of NR Method from Taylor Series

Taylor Series of f(xi+1) around xi:

f ( xi 1 )
f ( xi 1 )  f ( xi )  f ( xi )( xi 1  xi )  ( xi 1  xi ) 2  ...
2!
Use first order approximation:

f ( xi 1 )  f ( xi )  f ( xi )( xi 1  xi )

Assume that xi+1 is the root, then f(xi+1) = 0.

0  f ( xi )  f ( xi )( xi 1  xi )

Solve for xi+1:

f ( xi )
xi 1  xi 
f ( xi )
Error Analysis of NR Method from Taylor Series
1st Order Taylor Series of f(xi+1) around xi:
f ( xi 1 )  f ( xi )  f ( xi )( xi 1  xi ) . Assume that xi+1 is the root:

0 = f ( xi )  f ( xi )( xi 1  xi ) (Equation 1)

1st Order Taylor Series of f(xr) around xi where xr is the exact root.
f (x )
f ( xr )  0  f ( xi )  f ( xi )( xr  xi )  ( xr  xi ) 2 (Equation 2)
2!
Subtract Equation 1 from Equation 2:
f (x ) Et ,i  ( xr – xi )
0  f ( xi )( xr  xi 1 )  ( xr  xi ) 2

2 Et ,i 1  ( xr – xi 1 )
f (x ) 2

0  f ( xi ) Et ,i 1  Et ,i where xi  x  xr
2
For convergence xi  xr , therefore x  xr
f ( xr )
Et ,i 1  Et2,i
 r)
2 f (x
Error Analysis of NR Method from Taylor Series (continued)
f ( xr )
Et ,i 1  Et2,i
 r)
2 f (x
• The analysis shows quadratic convergence.
• Therefore, the error at each iteration is roughly proportional to the
square of the previous error.
• See Example 6.4 in Chapra and Canale for a demonstration.

Difficulties with the NR Method

• Need to know the derivative of the function.
• May diverge around inflection points.
• May give oscillations around local minima or maxima.
• Zero or near-zero slope is a problem, because f’ is at the denominator.
• Convergence might be slow if the initial guess is poor.
Difficulties with the NR Method (continued)

• Below examples illustrate some cases where Newton-Raphson

Method does not perform well.

Figures from Chapra and Canale, 2010

NR Method Examples

Example 7: We showed that NR can be derived from a 1st order Taylor

series expansion. Derive a new method using 2nd order Taylor series
expansion. Compare the convergence of this method with NR. Comment on
the practicality of this new method.

Example 8: NR method can be seen as a Simple One-Point Iteration

method with g(x) = x i – f(x i) / f’(x i). Using the convergence criterion of
the Simple One-Point Iteration Method, derive a convergence criteria for the
NR Method.

Example 9: Use the NR method to locate one of the roots of

f(x) = x(x – 5)(x – 6) + 3 starting with x0 = 3.5
(NR will oscillate around the local minimum).
Open Methods: Secant Method
• Secant Method is very similar to Newton-
f(x) Raphson Method.
• Instead of exactly determining f’(xi),
approximate it using two initial guesses xi-1
and xi.
x1 x-1 x0 x f ( x1 )  f ( x0 )
f ( x0 ) 
x1  x0
• Substitute this approximation into the NR:
f(x) f ( xi )
xi 1  xi 
f ( xi )
f ( xi )( xi 1  xi )
xi 1  xi 
x2 x1 x0 x f ( xi 1 )  f ( xi )

• Use x i+1 and x i in the next iteration

for estimating f’.
Secant Method (continued)
Example 10: Repeat the previous example: f(x) = x2 – 11.
Start with x-1 = 2, x0 = 3

Iteration x f(x) |ε t| %

-1 2 -7 -311

0 3 -2 -160

1 3.4 0.56 2.51

Selection of the initial guess affects the convergence.

Example 11: There is no guarantee for the secant method to converge. Try to
calculate the root of ln(x) starting with (a) x-1 = 0.5 and x0 = 4, (b) x-1 = 0.5 and
x0 = 5. Part (a) converges, but part (b) does not.
Secant Method vs. False-Position Method
Secant Method
f ( xi )( xi 1  xi )
xi 1  xi 
f ( xi 1 )  f ( xi )
False-Position Method
f ( xU )( xL  xU )
x  xU 
f ( xL )  f ( xU )
• Formulae of both methods are
the same.
• The algorithms for replacing the
previous estimates with the new
estimate are different.
• False-position Method drops one
of previous estimates so that the
remaining ones bracket the root.
• Secant Method always drops the
oldest estimate, xi-1 .
Special Treatment of Multiple Roots
• In case of multiple roots, bracketing methods cannot be used.

• Open methods still work; however,

• they converge slowly (linear instead of quadratic convergence).

• f’(x) also goes to zero at a multiple root. This creates the risk of division
by zero for the Secant and the NR. f(x) will reach zero faster than f’(x),
therefore use a zero-check for f(x), and stop before f’(x) becomes zero.

Figures from Chapra and Canale, 2010

Special Treatment of Multiple Roots (continued)

• Some modifications can be made for increasing the convergence rate.

i) If you know the multiplicity of the root, the NR can be modified as:

f ( xi ) m = 2 for a double root,

xi 1  xi  m
f ( xi ) m = 3 for a triple root, etc.

ii) Define a new function u(x) = f(x) / f’(x) and solve for the roots of u(x)
instead of f(x). It can be shown that u(x) has the same roots as f(x).
u( xi ) f ( xi ) f ( xi ) f’’(x) needs to
xi 1  xi  
u ( xi ) [ f ( xi )]2  f ( xi ) f ( xi ) be known.

• Similar modifications can be made for the Secant Method.

See the book for details.
Summary of Root Finding Methods
Conver
Method Type Guesses
-gence
Stability Comparison of methods
for f(x) = e-x – x
Direct Analytical - - -

Graphical Visual - - -

Bisection Bracketing 2 Slow Always

False- Slow/
Bracketing 2 Always
position medium

One- May
Open 1 Slow
Point diverge

Newton- May
Open 1 Fast
Raphson diverge

Medium May
Secant Open 2
/fast diverge Figure from Chapra and Canale, 2010
Solving Systems of Nonlinear Equations
y
Simple One-Point Iteration Method
Solve the system of equations: x  y  4 ex  y  1
2 2
x
Put the functions into the form x=g1(x,y), y=g2(x,y)

x  g1 ( y )  ln(1  y ) y  g2 ( x)   4  x 2
Select initial guesses such as x0 = 0.0 and y0 = 0.0. Use g1 and g2 for
calculating the next guesses.
x1= g1(y0) = 0 y1= g2(x1) = -2

x2= g1(y1) = 1.098612289 y2= g2(x2) = -1.67124236

x3= g1(y2) = 0.982543669 y3= g2(x3) = -1.74201261

x4= g1(y3) = 1.00869218 y4= g2(x4) = -1.72700321

The solution is converging to the exact solution of x=1.004169 , y=-1.729637.

Exercise: Solve the same system but rearrange the equations as: x = ln(1 – y),
y = (4 – x2) / y and start from x0 = 1, y0 = –1.7. Remember, this method may diverge.
Solving Systems of Nonlinear Equations: Newton-Raphson
Consider the following general form of a two-equation system.
u ( x, y )  0
v ( x, y )  0
Write 1st order Taylor Series Expansion (TSE) for these equations.
 ui  ui
ui 1  ui  ( xi 1  xi )  ( yi 1  yi )
x y
 vi  vi
vi 1  vi  ( xi 1  xi )  ( yi 1  yi )
x y
To find the solution set u i+1 = 0 and v i+1 = 0. Rearrange

 ui  ui  ui  ui
xi 1  yi 1  ui  xi  yi
x y x y
 vi  vi  vi  vi
xi 1  yi 1   vi  xi  yi
x y x y
Solving Systems of Nonlinear Equations: Newton-Raphson (cont.)
Write in matrix form:
  ui  ui 
  ui  ui   ui  xi  yi 
x   xi 1  x y
y     The 2 x 2 coefficient
    matrix is called the
  vi  vi      Jacobian matrix.
  x  y   v  v
 y  i 1
  vi  i
xi  i
yi 
 x  y 
The method can be generalized for n simultaneous equations.
Solve for x i+1 ,y i+1 using Cramer’s rule (ME 210), and iterate.

 vi  ui  ui  vi
ui  vi vi  ui
y y x x
xi 1  xi  , yi 1  yi 
 ui  vi  ui  vi  ui  vi  ui  vi
 
x y y x x y y x

The denominator is the determinant of the Jacobian matrix, |J|.

Solving Systems of Nonlinear Equations: Newton-Raphson (cont.)
Example 12: Solve the given system using NR with initial guesses: x0 = 1, y0 = 1.

u  x2  y2  4  0 v  ex  y  1  0
u u v v
 2x ,  2y , e ,
x
1  |J|  2x - 2ye x
x y x y
ui  2 yi vi 2 xvi  e xi ui
xi 1  xi  , y i  1  yi 
|J | |J |
i x y u v J
0 1 1 -2 2.718282 -3.436564
1 -1.163953 3.335387 8.479595 2.647636 -4.410851
2
3

The solution is converging to the root in the 2nd quadrant, x  -1.8, y  0.8.

Exercises: a) Can you start with x0 = 0 and y0 = 0 ? b) Try to find initial guesses that will
converge to the solution in the 4th quadrant.