Nurul A. Chowdhury - Numerical Methods

University ofSaskatchewan
ury
Sas�atoon, Canada
•
University of Sasbtdlewan
Saskatoon, Ca2,<1da
Kendall Hunt
pub l ish i ng company
All interior images were created by the nutbor.
Cover image© Shuuerstock, Inc.
Kendall Hunt
pub l ish i no company
www.kendallhunt.com
Send a// inquiries to:
4050 Westmark Drive
Dubuque, IAS2004- 1840
Copyright © 2019 by Kendall Hunt Publis.bing Company
ISBN 978-1-5249-9597-3
All eights reserved. No part of this publkation may be reproduced,

stored in a retrieval system. or transmit1ed. in any form or by any means,
elecLronie. mechanical, photocopying, recording, or otheiwise,
wi1hom the prior wriuen permission of the copyright owner.
Published in the United States of America

To my sons,
Rahid
and
Ariq
_J
There must be an ideal world, a sort of mathematician's paradise where
everything happens as it does in textbooks.
-Bertrand Russell
Preface .............................................................................................................................................. xi
Chapter 1 POLYNOMIAL APPROXIMATIONS AND TAYLOR POLYNOMIALS ....... 1

1.1 Taylor Polynomials ....................................................................................... l
1.2 Taylor's Remainder ....................................................................................... 6
1.3 Taylor Series ................................................................................................. 7
J.4 Maclaurin Series ........................................................................................... 8
Chapter 2 SOLUTIONS OF EQUATIONS IN ONE VARIABLE ........................... 11

2.1 The Bisection Method ................................................................................ 11
2.1.l Error bound in the bisection method ............................................ 14
2.2 Fixed-point Iteration ................................................................................... 16
2.3 Newton's M ethod........................................................................................ 22
2.3.l Error term of Newton's method .................................................... 24
2.3.2 The Secant method ....................................................................... 26
2.3.3 Method of false position ............................................................... 28
2.4 Miiller's Method ......................................................................................... 3 1
0
0 Contents
2.5 Brent's Algorithm ....................................................................................... 33

2.6 Repeated Roots ........................................................................................... 37
2.7 Convergence ............................................................................................... 41
2.8 Accelerating Convergence .......................................................................... 42
2.8.1 Aitkcn's extrapolation for linearly convergent sequences ........... .42
2.8.2 Ste:ffensen·s acceleration .............................................................. 44
Chapter 3 SOLUTIONS OF LINEAR SYSTEMS OF EQUATIONS ....................... 47

3.1 Systems of Linear Equations ...................................................................... 47
3.2 Linear Dependence and Independence ....................................................... 48
3.3 Matrix Representation of Systems of Linear Equations ............................. 49
3.4 Gauss Elimination ....................................................................................... 50
3.4.1 Piv,oting ......................................................................................... 53
3.5 Gauss-Jordan Elimination........................................................................... 56
3.6 Solution by Matrix lnversion ...................................................................... 57
3.7 S.olution by Crruner's RuJe ......................................................................... 62
3.8 Solution by Matrix Factorization ................................................................ 64
3.8.1 Triangular matrices ....................................................................... 65
3.8.2 LU factorization ............................................................................ 65
3.8.3 Elementary matrices and LU factorization ................................... 69
3.8.4 Choleski's factoriz.ation ................................................................ 77
3.8.5 LOU factorization ........................................................................ 79
3.8.6 Solution by QR factorization ........................................................ 82
3.9 Ill-Conditioned Systems ............................................................................. 86
Chapter 4 ITERATIVE TECHNIQUES FOR SOLVING LINEAR AND

NON-LINEAR SYSTEMS ....................••...•.•••............................ 91
4.1 Jacobi Iterative Technique .......................................................................... 91
4.2 Gauss•-Seidel Iterative Technique ............................................................... 94
4.3 Iterative Techniques and the Convergence of Linear Systems ................... 96
4.4 Iterative Techniques for Non-linear Systems .............................................. 97
4.5 Newton's Method........................................................................................ 98
Contents ~
Chapter S INTERPOLATION •...................................•......................•..... 103

5.1 Polynomial Interpolations......................................................................... 103
5.2 Lagrange Interpolating Polynomials ........................................................ 104
5.2.l Error bound of Lagrange interpolating polynomial .................... 107
5.3 Newton's Interpolating Polynomials ........................................................ 109
5.3.l Divided difference and the coefficients ...................................... 110
5.3.2 Error bound of Newlon 's interpolating polynomjal.................... 114
5.4 Hennite Interpolation ............................................................................... 115
5.5 Cubic Spline Interpolation ........................................................................ 1 I 8
5.5.1 Constructing cubic splines from the .second derivative .............. 132
Chapter 6 NUMERICAL DIFFERENTIATION AND INTEGRATION ................. 145

6.1 Numerical Differentiation ......................................................................... 145
6.2 Alternate Approach and Error Estinnates .................................................. 148
6.3 Second-order Derivatives.......................................................................... 151
6.4 Numerical Integration ............................................................................... 152
6.4.1 Trapezoidal rule .......................................................................... 153
6.4.2 Simpson's rule ............................................................................ 154
6.5 Error Analy:sis ........................................................................................... 155
6.5.1 Error of the Trapezoidal rule ...................................................... I 55
6.5.2 Error of Simpson's rule ............................................................... 157
6.6 Composite Numerical Integration ............................................................ 160
6.6.1 Composite Trapezoidal rule ........................................................ 160
6.6.2 Composite Simpson's rule .......................................................... 160
6.7 Gaussian Integration ................................................................................. 16 I
Chapter 7 INITIAL-VALUE PROBLEMS FOR ORDINARY

DIFFERENTIAL EQUATIONS .................................................. 171
7.1 Some Basics of Differential Equations ..................................................... 171
7.2 Forward Euler's Method ........................................................................... 173
7.2.1 Error and convergence of forward Euler's method ..................... 175
~ Contents
7.3 Backward Euler's Method ........................................................................ 178

7.4 Trapezoidal and Modified Euler's Method (Heun's Metbod) ................... 180
7.5 Higher-Order Taylor's Methods ................................................................ 182
7 .6 Runge-Kuna Method of Order Four ......................................................... 183
7 .7 Runge-Kutta-Fehlberg Method ................................................................. 189
7.8 Multistep Methods .................................................................................... 194
7.8.1 4lh order Adams-Bashfortb technique ......................................... 194
7.8.2 4•h order Adams-Moulton technique ........................................... 196
Chapter 8 PERIODIC FUNCTIONS AND FOURIER SERIES .......................... 201

8.1 Periodic Functions .................................................................................... 20 I
8.2 Fourier's Theorem .................................................................................... 202
8.2.1 Dirichlet's conditions .................................................................. 203
8.3 Fourier Coefficients .................................................................................. 204
8.4 Convergence of Fourier Series and Gibb's Phenomenon ......................... 207
8.5 Even and Odd Functions ........................................................................... 208
8.6 Functions Having Arbitrary Period........................................................... 21 1
8.7 Complex Form of Fourier Series .............................................................. 214
8.8 Functions Defined Over a Finite Interval ................................................. 216
Chapter 9 EIGENVALUES AND EIGENVECTORS AND THEIR

APPLICATIONS •.....................................•............................ 221
9.1 Eigenvalues. and Eigenvectors .................................................................. 22 1
9.2 State-Space Models .................................................................................. 225
9.2.1 A series RLC electrical circuit.. .................................................. 225
9.2.2 An automobile suspension system .............................................. 226
9.2.3 A two-tank liquid level system ................................................... 227
9.2.4 State-space representations ......................................................... 228
9.2.5 Transfer function matrices and s tability ..................................... 228
9.2.6 Coordinate transformation and eigenvalues ............................... 23 1
9.3 Solving Systems of Differential Equations............................................... 237
Contents <S>
Chapter10 BOUNDARY-VALUE PROBLEMS AND PARTIAL
DIFFERENTIAL EQUATIONS .................................................... 251
10.1 Existence of Solutions to Boundary-value Problems ............................... 25 l
10.2 Shooting Mcthods ..................................................................................... 252
10.2.1 Linear shooting method .............................................................. 252
10.2.2 Nonlinear shooting method ........................................................ 258
10.3 Partial Differential Equations ................................................................... 260
10.4 Solution to PDEs: Method of Separation of Variables ............................. 262
10.5 Finite Difference Method for the Wave Equation ..................................... 273
10.6 Finite Difference Method for the Heat Equation ...................................... 280
Chapter 11 OPTIMIZATION •.....................................•............................ 287

11.1 Convex Sets .............................................................................................. 287
11.2 The Mean Value Theorem ......................................................................... 288
11.3 Relative Maximum and Minimum ............................................................ 288
11.4 Optimization of Functions of a Single Variable........................................ 289
11.5 Sufficient Conditions for an Optimum Point............................................ 29 I
11.6 Global Maximum (Minimum) of One Variable ........................................ 295
11.7 Op1in1a of Convex and Concave Functions .............................................. 296
11.8 Search Methods for Fw1ctions of One Yariable ........................................ 298
11.8.1 Exhaustive search method .......................................................... 298
11.8.2 Fibonacci method ....................................................................... 300
11.9 Optimization of Functions of Several Variables ....................................... 302
11.10 Multivariate Grid Search Method ............................................................. 306
11.11 Univariate Search Method ........................................................................ 307
11.12 Gradient Methods: Directional Derivatives .............................................. 309
11.13 Direction of the Steepest Descent (Ascent) .............................................. 310
11.13.l Steepest descent (ascent) method: one variable at a time ........... 312
11.13.2 Steepest ascent (descent) mcd1od: multiple variable at a time ....... 3 I 3
11.14 Constrained. Optimization......................................................................... 317
References ...................................................................................................................................... 323

_J
Books related to the broad subject area of numerical methods can be classified into three groups. In
the first group, the books deal with the mathematical analyses of numerical techniques. Rigorous
error analyses, mathematical theories and proofs take readers to the very foundation of numerical
techniques. This type of books are written for students majoring in mad1ematics. The second group
deals with the application of numerical techniques to physical and design problems. These books
utilize the mathematical concepts related to numerical techniques mostly without getting into the
analyses and are aimed at scientists and engineers. Then the third group of books that are also aimed
at scientists and engineers, discuss the implementation of numerical techniques with the help of a
particular computer software, Matlab®or Mathcad®.
This textbook belongs to the second group with an emphasis in providing the basic mathematical
concepts and connecting them to the numerical techniques. Thls book provides explanations of the
numerical techniques in such a way that students should be able to de·velop their own algorithms and
implement them with the help of software of their choice. Straightforward numerical examples have
been provided to help students understand the techniques.
This book is written based on the course notes that I bad developed over two decades of teaching
the subject matter at the University of Saskatchewan. In delivering their courses the instructors may
drop some topics and may emphasize others to suit the needs of the students in particular disciplines.
Depending on the course objectives of a discipline, the chapters can be taught in a different sequence
than the one presented in this book. It is necessary that the students have already taken first-year uni-
versity level courses on elementary calculus, Linear algebra and introductory differential equations.
Taylor polynomials are the backbone of many nwnerical techniques. Chapter l provides a sum-
mary review on Taylor polynomials. Although students in their elementary calculus may have been
ex.posed to the concept of Taylor polynomials, a review in the light oftlJe numerical perspective will
0
~ Preface
be: helpful. Interpolation is another area that has deep collJlection to many numerical teclmiques.
Interpolation techniques have been discussed in Chapter 5. Chapters l and 5 should be part of any
course offerings at the junior Ulldergraduate level.
Chapters 1-7 can be delivered as a one term course at the junior undergraduate level. Chapters 8-1 l
can be delivered as a one term course at the senior undergraduate as well as graduate level. Instructors
may skip some of the topics without losing the continuity of their courses. A companion laboratory
class where students should be able to implement the ideas and algorithms utilizing computer pro-
grams and commercially available software of their choice will be invaluable.
I am grateful to my colleagues and students whose feedback and suggestions have been helpful in
writing this book.
Nurul A. Chowdhury
Saskatoon, Canada
PD ~ MI L R MAil I ONS-+-+---+ ---+---<-t--1
---AV- - L O S
Polynomials appear in a wide vaciety areas of mathematics, physics and basic chemistry. A
polynomial is au expression of sum of terms where each term is a product of a coefficient and var-
iables raised to exponent of non-negat ive integers. A polynomial olften represents a function. The
term with the highest sum of exponents is the leading tenn in a polynomial and the highest sum of
the exponents is the degree of a polynomial.
A polynomial in a single vaciable, x can be written as
(I.I}
where a 0 , a, , a2 , • • • • a,. are coefficients and x is a variable.

The polynomial shown in expression (I.I) can be expressed in a concise form using summation
notation.
(1.2)
A real polynomial is a polynomial with real coefficients, an integer polynomial is a polynomial with
integer coefficients and a complex polynomial is a polynomial with complex coefficients.
1.1 Taylor Polynomials

Taylor polynomials are widely used in mathematics. They provide useful means in developing
various algorithms in numerical analysis. Actually, numerical analysis utilizes results from many
branches of mathematics particularly calculus and linear algebra.
0
0 Chapter 1: Polynomial Approximations and Taylor Polynomials
Taylor polynomials can be used 10 mimic a function in the near vicinity of a given point, say, a.
Tbe degree of a Taylor polynomial increases as we impose more conditions that this polynomial
bas to satisfy.
Let us strut with a linear polynomial P, (x) that has to satisfy the conditions:
P, (a)= f(a) and
P,'(a) = f'(a)
Let P, (x)=h0 +h,x

Therefore.
P, (a)=h0 +h,a= f (a) (1.3)
P,'(x) = h,
P,'(a)=b, =f'(a) (1.4)
From Equations ( 1.3) and ( I .4), we get

h0 =f (a) - h,a =f (a) - af'(a)
Therefore,
P, (x) =h0 +h,x = J(a) - af'(a)+x f'(a) (l.5)
Rearranging Equation (1.5),

P, (x) = J (a)+ (x -a)J' (a) (1.6)
Example 1.1
Let /(x)=sinx anda=L
Then /(a)=0.841 and f'(a) =0.54
Therefore, the function can be approximated at a= I by the Taylor polynomial P, (x) = 0.841 +
0.54(x - J) which could be further simplified as
P, (x)=0.301+0.54x
P, (x) is approximately sinx when xis near a. •
Let us now impose an additional condition that the second derivative of the polynomial should be
equal to the second derivative of the function at a. A quadratic polynomial would be required to meet
this condition.
(1.7)
Chapter 1: Polynomial Approximations and Taylor Polynomials 0
The quadratic polynomial shown in Equation ( 1.7) must, therefore, satisfy the following three
conditions.
P1 (a) = f (a)
P;(a) = f'(a)
P{(a)=f'(a)
To satisfy the conditions, we may write

~(a)= h0 +b1a+b2a 1 = f (a) (I.8)
P; (x) = h1 + 2h2 x
P;(a) = h, +2b1 a =f'(a) (1.9}
P1'(x)=2h1
P;(a) = 2b1 = f'(.a) (1.10)
From Equation ( I. LO), b2 = ½/' (a)

From Equation (1.9), b, +2- ..!__ J'(o) -a= f (a)
2
h1 = f'(a) - a·f'(a)
From Equation ( 1.8), h0 = f(a) - h,a - b1a 2
h0 = f (a) - a{f'(a) - a · f'(a)} - a

1
{½ /'(a)}
h0 = J (a) - a · f'(a)+..!... a2 • f'(a) (LI I}
2
Therefore, the quadratic polynomial ( l.7 ) can be written as
~(x) = h0 + h,x+h2x 1
= f (a) - a -J'(a) + ..!._ -a 1 · f"(a)

2
+ {f' (a) - a · f" (a)}x +..!.../"(a) -x 2
2
2
P2 (x)= f(a) +(x - a) · f'(a)+½ (x - a) • f'(a) ( l.12)
Example 1.2
Let f(x) = sinx and a= I.
Then f(a) =0.841, f'(a)=0.54 and /'(a)=-0.841
Therefore. the function can be approximated at a= I by the quadratic Taylor polynomial P2 (x) =
1
0.841+0.54(x - 1) - 0.4205(x - 1) which could be further simplified as
P2 (x)=-0.4205x1 +1.381x - 0.I 195
Pi (.r) is approximately sinx when ,Y is near a. •

If we continue to impose more conditions that P,, (x) be a polynomial of degree II which should
satisfy P,, (a)= f (a) and
i=0, 1, ... , 11
where P,,l'l (.r) is the ith derivative of P,, (x) and _f'-'l (x) is the ith derivative off (x). Then
(x a)a (x a)"
i:(x)=J(a)+(x - a)J<1>(a) + - f 2
l(a)+···+ - f(,,)(a)
2! n!
P,, (x)= I, (x ~,a)' pi> (a) ( 1.13)

1:-0 I.
The polynomial P,, (x) in ( 1.13) is called the Taylor polynomial of degree n for the function f (x) at a.
Theorem:Taylor'sFormula
Suppose that / is continuously differentiable and its (n + l)th derivative exists on an interval
containing the points a and b. Then
(b - a) 2 (b - a}3 1
J(b)=f(a)+(b - a)f'(a)+ ! f'(a) + )! f >(a)
2
( 1.14)
+···+ (b - a)" ji">(a)+ (b - a)"+' i<•+')(z)
n! (11+ I)!
for some number z between a and b.
If we replace b with x in Equation ( 1.14), we get nth degree Taylor's formula with remainder at
x=a.
(t - a) 2 (x - a) l
f(x)=f(a)+(x - a)f'(a)+ · f (a)+ -'---___.c.._ f <1>(a)
1
2! 3!
(1.15)
( x - a )" An)( ) ( x - a )"+' i<•••l()
+···+ ~ ~ -; a +~ ~ - z
n! (n +l)!
where z is some number between a and x and
(x - a t·' f "'') (z) is the nth degree remainder.
(n+I)! .
Example 1.3
Again let f (x) =sinx and a= L Then a Taylor polynomial of degree 3 to approximate f (x) at a= I
can be written using (1.13) as
3
(x - 1)2 (:c -1)
P1 (x)=0.84l+(x - 1) cos(!)- ! sin(l) · -cos(I) (1.16)
2 31
Table 1.1 Taylor polynomial approximations to sinx
X sinx P, ( x ) P2(X) P3(X) lsinx - P. (x )I lsmx-P2 ( x~ lsinx-P3 ( x~

0.25 0.2474 0.436 0.1995 0.2376 0.1886 0.0479 0.0098
0.50 0.4794 0.571 0.4659 0.4774 0.0916 0.0136 0.0020
0.75 0.6816 0.706 0.6797 0.6815 0.0244 0.0019 0.0001
1.00 0.8415 0.841 0.8410 0.8415 0.0005 0.0005 0.0000
.1.25 0.949 0.976 0.9497 0.9489 0.0270 0.0007 0.0001
I.SO 0.9975 1.111 1.0059 0.9952 0.1135 0.0084 0.0023
1.75 0.984 1.246 1.0095 0.9721 0.262 0.0255 0.0119
Table I. I shows values of sinx, Pi (x), P1 (x), ~ (x) and the corresponding errors of the respective
approximations. For a polynomial of fixed degree, the error increases as x moves away from a= I.
Amd, for a fixed x, the accuracy improves as the degree of the Taylor polynomial increases. ■
Example 1.4
Let f(x)=e' and a=O. Then
j!IJ(x)=e'. Jl'l(O)=lforall i~l.
Therefore.
1
P (x)=l+x+ - I x 2 +-1 x 3 +· ··+-I x • = I,
" -x (1.17)
• Jt
21. . II.I ·1
l =-01.
•
Example 1.5
Let f (x) ""e•· and a is an arbitrary point. Then
f 1l) (x) = e·', J!'l (a) =e• for all i ~ I.
Therefore,
P,,(x)=e"{ I+ (x - a)+, I(x - a) "} =e. £..,

1(x - a)1 + I (x - a)3 +···+, ~ (x -_ a)' ■
2. 1
3. 11. 1:0 I.1
Example 1.6
Let / (x) = In (x) and a= L Then
f 1> ( x} = (- 1)1- 1 ( i - I)!_!,., f'l (1) =(- 1)1- 1 ( i - I)! for al I i ~ 1.
x'
Therefore, utilizing the coJTesponding Taylor polynomial is g.iven by
~ (x) = (x - 1)- .!:..(x - 1)2 +.!.(x - 1)3 - .. ·+ (- 1)"-l .!.ex - I)" •
2 3 n
1.2 Taylor'sRemainder
So far, we have seen that a Taylor polynomial P. (x) can be used to approximate a function f(x)
widun the near vicinity of a given point, x =a. However, with this approximation. we encounter a
difference between f (x) and P. (x) that is measured as
R. (x) = f (x) - P,, (x)
where R. ( x) is called the 11th-degree remainder for f (x) at x =a.

Suppose that d1e (11 + 1) th derivative of the function f (x} exists on an interval a~ x ~ f3 and a belong
to d1at interval. Then die difference R,, (x) which indicates the closeness of the approximation is
ex.pressed as
n+I
(
R. (x) x - a) f"+' (z) (1.18)
(11 + I)!
where z is an unknown point between a and x.
From (1.18), it is obvious that the e rror increases as x moves furd1er from a, and the error decreases
as the degree 11 of the Taylor polynomial increases. For a given error bound, R. ( x) can be utilized
to determine ".
Example 1.7
Find a Taylor polynomial of appropriate degree to approximate sinx at a= 0 so that the error within
[-: , : ] does not exceed 10-s_
( )n+I
R.(x) x /"+1 (z)
(11 +1)!
The point z in the remainder of ( 1.1:8) is unknown, therefore, we will seek the worst possible case so
that the error does not exceed 10- 5_ In the worst case, Icos xi = 1 if n is even and Isin xi = I if II is odd.
(x)"'' n
, ~---1 ~ 10-5 for lxl ~ - . Therefore,
(n+I)! 4
(n/4 )"'' ~ w-s (I.I 9)

(11 + I)!
( LI 9) is satisfied when 11 ~ 7.
f(x)= f(a)+(x - a)f'(a)+ (x - a)2 f'(a)+ (x - a)3 f 3>(a)

2! 31
1
(x - a) Jr<1) ( a)
+ ... + -'-----''-
7!
. . ( ) (x - a}2 .
sin.r = sma + x -a cos a - -'-----'-SIDG --'---'- COsa
(x a}3
2! 3!
4 5
(x - a) (x - a) (x - a) 6
+ -'-----''- sin a+ -'----'-cos a - -'----'-sin a
4! 5! 6!
7
( X a)
-..c-~cosa
7!
Since a=O,
. x3 x5 x1
smx=x - -+ - - - ■
31 5! 7!
1.3 Taylor Series

The exact value of z in Taylor remainder term shown in ( 1.18), in general. is unknown. However, the
remainder approaches zero as n ~ oo. Therefore, the Taylor's formula can be written as an infinite
series without the remainder term. This infinite series is called Taylor series and is expressed as
2 l
f(x)=J(a)+(x - a)J'(a)+ (x - a) J'(a)+ (x - a) f 3>(a)
2! 31
(1.20)
(x - a)" •
+···+-'----'-- f '(a)+···
11!
Ta,ylor series fore' at a= 0 is written as
, 1 , 1 l ,,
e = 1+x +- r +- x 3 +···+ - x +··· ( l.21)
2! 3! 11!
With x=I, Equation (1.21) becomes,
J 1 1 l 1
e=l+ - + - + - + - + - + ··· ( 1.22)
I! 2! 3! 4! 5!
1.4 Maclaurin Series

Maclaurin series of a function is the Taylor series of the function wi.Lh a= 0. Three most important
Maclaurin series arc:
2 3 4 5
., X X .JI" X X
e =l+ - + - + - + - + - + ··· ( 1.23)
I! 2! 3! 4! 5!
X3 XS .X1
sinx=x - -+- - -+··· ( 1.24)
3! 5! 7!
. x 2 x• x 6
cosx= 1- -+ - - -+··· ( 1.25)
2! 41 6!
These series hold true for all values of x and hence are mathematical identities. They, therefore, can
be util izcd to derive other infinite series.
Problem Set 1.1

In Problems l through 7, find Tayfor's formula for the given function fat a= 0. Show the Taylor
polynomial P. (x) of degree II and the corresponding remainder term R,, (.x).
L f(x)=cosx, 11=8
2. f(x)=e-', 11=5
3. /(x)=ln(l+x),11=4
4. J(x)=.JI+x, 11=4
1
5. f(x)= - , 11=4
1-x
6. f(x)=si n-• x, n=3
7. f(x)=tanx, 11=3
In Problems 8 d1rough 14, find the Taylor polynomial and the corresponding remainder for the given
vaJ ues of a and n.
8. f(x)=cosx,a=n:/4,n=&
9. f(x)=e-•,a=l.,n=5
10. /(x)=ln(l+x). a=5 , 11=4
11. J(x)=.JI+x, a=2, 11=4
1
12. f(x)= - ,a=2,n=4
1-x
13. f(x)=si n-•x, a=n:/4,11=3
14. f(x)=tanx, a=1t/3, 11=3
In Problems 15 through 2 1, find the Taylor series of the function at the given value of a. Show at
least the first four non-zero terms.
15. f(x)=cos.x, a=n/4
16. f(x)=e-1 ·', a= l
17. f(x)=ln(l+x). a=5
18. f(x)=lnx, a=2
1
19. f(x)= l ' a=2
(1 - x)
20. f (x) =sin-• (2:r:), a= n/4
21. f (x) =tan(x/2), a= n/3
In Problems 22 through 28, find the Maclaurin series of the given function at. Show at least the first
four non-zero tenns.
22. J(x)=cosx. a=n/4
23. f(x) = e-1x. a= I
24. f(x)=ln(l+.x),a=5
25. f(x)=lnx, a=2
I
26. f(x)= ,a =2
(1 - x) 3
27. f (x) =sin-• (2x), a= n/4
28. f (x) = tan(x/2), a= n/3
In Problems 29 through 35, find the Taylor polynomial of appropriate degree to approximate the
function at the given point a such that the maximum error does not exceed l0-5•
29. f(x)=cosx,a=n/4
30. f(x)=e-•, a=l
31. /(x)=ln(l+x), a=5

32. f(x)=JI+x,a=2
1
33. /(x)= - , a=2
1-x
34. f(x)=sin-'x,a =n/ 4
35. f(x)=tanx,a=n/3
_J
In many branches of applied mathematics, it is required to find the value of x that wiU satisfy the
equation f(x)=0. The value of x that satisfies the equation is caUed the root or the zero of the
equation. An equation based on its degree can have multiple roots. The roots can be real as weU as
complex. In this chapter, some methods to find the roots will be discussed.
2.1 The Bisection Method

Tbe bisection method, also caUed binary search me//,od, is based on the Intermediate Value
Theorem.
Intermediate Vallie Theorem: If/ (x) is a continuous function on [ ll, b], then for every d between
f (a) and/ (b), there exists a point c between a and b such that/ (c) = d.
Now, we will extend the theorem by a little bit. Assume that / (a) is positive and f ( b) is negative.
Therefore, somewhere between a and b, there exists a point p such that / (p) = 0. The extension of
the Intermediate Value Theorem simply tells us that p (by the way, this is a root of the equation,
f (x) = 0 exists, it does not tell us how to find it. Bisection method requires division of subinterval
[a, b] into two halves and locating the half that contains the root. The half containing the root is fur-
ther divided into two halves. At each iteration, the half containing the root is therefore squeezed and
eventuaUy the midpoint of the gradually thinner half approaches to the root.
We begin the process by setting a, = a and b, = b. Let p1 be the midpoint betv;een a, and b,. Therefore,
_ a,+b1
P,- 2
If f (Pt)= 0, then Pt is lhe root of the equation within [a, b ]. For sake of simplicity, we assllOle that
there is only one root within [ a, b ]. The method will, however, work if multiple roots exist in [ a, b].
<E>
0 Chapter 2: Solutions of Equations in One Variable
If f(Pt)-¢0, then f(p,) has the same sign as either f(a,) or f(b, ). When f(p,) and f(a,) have
the same sign. we conclude that the root, p is not between a, and p 1 rather p is between p1 and b,.
Therefore, we pick the subinterval (p1 , b, ] for further hal ving by setting a 2 = p, and b1 = b,.
If f(p,) and f(a 1 ) have opposite signs, we conclude that theroot,p is between a, aud p, aud we pick
Lhe subinterval [a, , p 1 ] for further h.alving by setting l2i =a1 and b2 =p,.
The halving and selecting the subinterval containing the root would continue nntil the boundaries
of the subinterval come very close to the root or the newly fonnd midpoint comes very close to the
root, p. Figure 2. l shows a few iterations of bisection algorithm.
When to stop
The iterative process would stop when the root is found, i.e.. whe,n f(Pn) =O. Sometimes how-
ever, we would be happy with a number that is indeed very close to the root. Because, it could
be a very long process before a midpoint would actually laud on the root due to the finite digit
j(x)
fib,)
.ftp,)
I
./1.P!) I
I
I
I
/11
o2 P1 h,
._I__ _,_
I _ _.,1·
Figure 2.1 Bisection algorithm

Chapter 2: Solutions of Equations in One Variable <S>
nature of our computation. Therefore, select a small positive number, e as a tolerance and generate
midpoints p1 , p2 , p3 , ••• , P. until one of the following conditions is met.
IP. - P..-11<e (2.1)
IP. - p.J <e P. ;t:O (2.2)

IP.I ,
IJ(p.)l<e (2.3)
In most circumstances, expression (2.1) can be used. However, in some other iterative methods
where the difference Ip., -P.-11decreases but the sequence {p,, }.__ ruverges, expression (2.1) could
become difficult to use. Although f ( p.,) "' 0 is an indication that an iterative process is approaching
the root, in many cases P. still could be far away from the actual root. This happens in asymptotic
cases and also where some slow varying function reach their maxima or minima. Expression (2.2)
indicates a relative error and can be nsed where a prior knowledge about the function and its roots
are unavailable. In an iterative process, a loop counter should be used to make sure that the number
of iterations does not exceed a maximum threshold.
The stopping criteria discussed above are also used for other root finding methods discussed in this
Chapter.
Example2.1
The equation e-• + x 3 - 5sin (x) - 13 = 0 has a root in [-6, - 2]. Table 2.1 shows the new subintervals
(new boundaries) and corresponding midpoints obtained using the 'bisection med1od. Our desired
tolerance is 0.0000 I.
Table 2.1 Results of the bisection method
II a. b
• P. Si.g n of Sign of Sign of
f (a.) f (b.) f (P.)
1 -6 -2 -4 + - -
2 -6 -4 -5 + - +
3 -5 -4 - 4.5 + - -
4 -5 -4.5 - 4.75 + - -
5 -5 -4.75 -4.875 + - -
6 -5 -4.875 - 4.9375 + - +
7 - 4.9375 -4.875 - 4.90625 + - -
8 -4.9375 -4.90625 - 4.92 188 + - +
9 -4.92188 -4.90625 - 4.91406 + - -
( Co11ri1111ed )
Table 2.1 Results of the bisection method (Cominued)
II a. b• P. Sign of S ign of Sign of

f( a.) f( b.) f (P.)
JO -4.92188 -4.91406 -4.91797 + - -
11 -4.92188 -4.91797 -4.91993 + - +
12 -4.9 1993 -4.91797 - 4.91895 + - -
13 -4.91993 -4.91895 - 4.91944 + - -
14 -4.91993 -4.91944 - 4.91968 + - -
15 - 4.91993 -4.91968 -4.91980 + - +
16 - 4.91980 -4.91968 - 4.91974 + - -
17 -4.91980 -4.91974 -4.91977 + - -
18 -4.91980 -4.91977 - 4.91979 + - +
19 - 4.91979 -4.91977 - 4.91978 + - +
20 -4.91978 -4.91977 - 4.91978 + - -
After 20 iterations, I.here is no change in the 5'" digit place after the decimal. Therefore, the root of
the equation is -4.9 I978 with a tolerance of 0.0000 I. ■
2.1.1 Error bound in the bisection method

In the bisection method, the range becomes half after every successive iteration. Let us assume that
there is a root of / ( x ) = 0 in [a, b ].
Therefore, starting from b, - a, =b - a
b- a
b? - a,= - -
- • 2
. a +b c
S1111ce p,. = " • ,or ali 11 ~ l and
2
Chapter 2: Solutions of Equations in One Variable 0
After 11 iterations, the error bound is
b- a
<--
- 2"
Every iteration shrinks the zone containing the root into half which means with every iteration the
error bound becomes half of its value in the previous iteration.
The error bound is calculated on the basis of infinite-digit arithmetic. Consideration should be given
to the fact that computers are finite-digit machines.
Example2.2
How many iterations will be necessary to find the root wid1in an accuracy of 10- 5? For th.is part. we
are using the equation e-x + x 3 - Ssin(x) -13 = 0 that has a root in [-6, - 2].
Solutio11
b- a
IP. - Pl~r
:. - 2 - (- 6) < 10-s
2"
or, ~<10-s
2"
or, 2 2 -T" < 10-s
Take 10 base log on both sides.
or, (2 - n) log'° 2 < log111 I o-s
log10 10-5
or, (2 - n) <--'-'---
log,02
or, (2 - n)< - 16.61
or, - 2+11> 16.61
or, 11 > 18.61
or, 11 > 18.61; needs 19 iterations.
It is evident from Table 2.1 that the root is within l o-s

after 19 iterations. The actual root is
-4.91977837633747 with 14 digits after the decimal. The root calculated after 19 iterations is
-4.91978073120117. ■
The bisection method is relatively straightforward. The principle advantage of the method is its
guaranteed convergence. Additionally, the error bound is reduced to one half with each iteration.
However, compared to the od1er metl1ods it converges more slowly.
~ Chapter 2: Solutions of Equations in One Variable
Problem Set 2.1

1. Use the Bisection method to find solutions accurate to within 10-" for x 3 - 5x1 + 16x - 9 =0
on [0, I).
2. Use the Bisection method to find solutions accurate to within 10.... for x 3 + l lx 2 + I6x - 8 = 0
on each interval.
a. [- 10, - 8] b. [4 , - 2) c. [- l, I]
3. Use the Bisection med1od to find solutions accurate to within I o-sfor the following equations.
a. x 2 - 1' =0for - 2~x~2 b. e' +x 2 - 5sinx=0for - l~x~I
c. xsinx - 3x3 + 7 =0 for 0.5 ~x ~2
4. Find an approximation to $ correct to within 10-s using the Bisection mediod.
5. Find an approximation to ifI'f correct to within 10-s using d1e Bisection med1od.
6. Use the bisection method to determine a solution accurate to within I 0 ..... for x 3 - 5x 2 + 8 = 0
on [0.5, 2].
7. Use the bisection method to detem1ine a solution accurate to within 10-1 for
x 3 - 5cos (2x) - 7 = 0 on [I, 2 ).
8. Use the bisection method to determine a solution accurate to within IO.... for 2' - 5x1 + 12 = 0
on [- 2, 0).
9. Use the bisection metllod to find solutions accurate to wi thin I o-sfor the following problems.
a. x 3 - 5x 2 +1 2=0, [- 2,0] b. x 3 +3x+l0=0, (- 3, - 1]
c. e' - 3x - 22=0, (2, 4) d. 5(x - 3)+sin x=0, [O, 4]
e. 3' - e' - l7=0, [3,5) f. ln(x - 3)+ sin(x - 3)=0, (3,5)
2.2 Fixed-point Iteration

LC!! us consider the following two functions.
y(x) = x and g(x) = x 2 - 30.
We are interested to find out where iliese two functions intersect, or in oilier words where dlese
two functions become equal to each other. Botb functions are plotted as shown in Figure 2.2 which
shows where me functions intersect wid1 each other.
The t\Yo fw1ctions are equal to each oilier at x = - 5 and at x = 6. Since we are interested where these
wo functions will be equal to each other, we are inherendy looking for x where x 2 - 30 = x , or, in other
words where x 2 - x - 30 = 0. Th.is means we are looking for die roots of the equation, x 2 - x - 30 = 0
and indeed d1e roots of x 2 - x - 30 =0 are - 5 and 6. lo fact, an equation can be partitioned into two
functions and the intersection of the partitioned functions are the real roots of die equation.
In this case, we are looking for a specific way of partitioning an equation, g(x) - x = 0 into t\YO func-
tions where one will always be y(x) =x and the oilier one will be y(x) =g(x). We, therefore, would be
Figure 2.2 Intersection of two functions
looking for x so that x = g(x). Let us say that at x = p, the condition x =g(x) is satisfied, i.e. p = g(p).
2 2
p is called a fixed-point of g. In g(x)=x2 - 30, g(6)=(6) - 30= 6 and g(- 5) =(- 5) - 30= - 5,
i.e, g(x)=x 2 - 30 has fixed points at x= - 5 and at x =6.
We can use this attribute to solve an equation f(x)=O by converting the problem to a fixed-point
problem. Rewrite /(x) = 0 into x = g(x). Thus, finding the fixed point p becomes essentially finding
the intersection of the two equation.~. y = x and y = g (x) as shown in Figure 2.3.
g(x)
0 p X
Figure 2.3 Fixed-point iteration
Theorems
a. If g e c[a, b] and g (x) e [a, b] for all x e [a. b], then g has a fuced-point in [a, b].
b. If, in addition, g'(x) exists on (a, b) and a positive constant k < l exists with lg'(x)I ~ k , for
all x e (a, b ), then the fixed point in [a, b] is unique.
Am alternate way of explaining Theorem (b) follows.
Let g(x) is continuous in [a, b] and g (x) e [a, b] and a positive constant k exists such that k < 1 with
lg(x) - g(y)l ~k lx - yj for all x,ye(a,b] then the fixed point in (a. h] is unique. This means that
x =g(x) has a unique solution pin [a, b]. Also, the iterates x,, = g(x,~,) 11 '2: l will converge top for
an.y choice of x0 in [a, b].
Proof of Theorem (a)
If g(a)=a or, g(b)=b theng has a fixed point at an end point. However, if g(a);ca or, g(b);cb,
then g(a)>a and g(b)<b. The function q(x)=g(x) - x is continuous on [a,b] and satisfies the
following conditions,
q(a)=g(a) - a>O
q(b)=g(b) - b<O.
The Intermediate Value Theorem implies that there exists p e (a. b) for which q(p) =0. This means
that q(p)= g(p) - p=O. Therefore, g(p)= p i.e .. pis a fixed point for g.
Proof of Theorem (b) (i.e., the fixed point is unique)
If the fixed point is not unique, then. g (p) = p and g (r) = r, i.e., p and. rare two fixed points in [ a , b].
:.p - r=g(p) - g(r)
or. IP- 11= lg(p) - g(r)I (2.4)
Now si nce p ;c r (and lg' (x)I ~ k ~ I) the Mean Value Theorem implies that a number S exists
between p and r such that
g(p) - g(r) g'(,s)

p- r
:. g(p) - g(r) = g'(s)(p - r)
i.e., lg(p) - g(rll ~lg'(s)ll<p- r)I
i.e., lg(p) - g(r)I ~ kl(p - r)I
i.e., lg(p) - g(r)I <l(p- r)I (2.5)
(2..4) and (2.5) are contradictions th.at come from our earlier assumption that p ;c r. Hence, p =rand
the fixed point in [a,b] is unique.
Now to prove that x,, = g (x,,_,) 11 ~ 1 will converge top for any choice of x0 in [a, b].
Since g is in [a, b] for all g (x) for x e [a, b], meaning g maps [a, b] into itself, the sequence {P. }:=0
is defined for all 11 ~ 0, and p,, e [a, b] for all 11.
Let us now assume that p,, is the nth approximation for the fixed point (root), p. Then using the
M-can Value Theorem
g(p.) - g(p) -g'(;) (2.6)

P. - P
where ; is between P. and p.
:. g(p,,) - g(p)=g'(~)(P. - p)
But g(p,,) = p,..,
.-. (P,.., - p) = c'(s)(p,, - p)
:. IP.+, - ~ ~lc'(~)IIP. - ~
--- IP. ~~kip,. - Pl
+1 -
By induction, Ip•+• - P l ~ k IP•-•- pj

2
and IP. +, - Pl~e1P. Pl -2-
Replacing n+l ➔ n
IPn - Pl~k3 1Pn-3 -Pl

:. Ip. - ~ ~k"IPo - Pl
Since O< k < I, we have liin k" = 0
Hence, {P. f:0 converges top.

Io order to find a root off (x) ""0, rewrite the equation in the fonu of x"" g (x) and iterate .x. When
x converges to the fixed point p the condition g(p) = p is satisfied and indeed p is then a root of the
equation f(x) =0.
There are only three steps to foUow. First, rewrite f (x) = 0 in the form of x = g(x). Second, select a
starting value for x (say c) and evaluate g (c). Say g(c) is equal to k. Third, k becomes the next value
of x with which g (x) will be evaluated. Repeat the third step for several iterations until the condition
p =g(p) is satisfied with a tolerance. The stopping criteria discussed for the bisection method also
apply for the fixed-point iteration.
Example2.3
Find the root of the equation, x 1 - 5x1 + 3x +4 = 0 within [ 0, 3] using the fixed-point iteration. Show
5 digits after decimal.
Solution
The equation x 1 - 5x1 +3x+4 =0 can be rewritten in the fonn of x ,= g(x) in several ways. In this
example we will examine the following five fonns.
a. x=
J¥
- x +5x2 - 4
3
5
b. x=
3
3
x + 3x +4
c. x=
5x
d. x= ( 5x-, - 3x - 4 ) 1/l
5x 2 - 3x - 4
e. x= 2
X
Table 2.2 shows the sequences generated for (a). (b), (c), (d) and (e). A starting value of Lis selected
for all five cases. The actual root in [0, 3] is 1.61803399. It can be seen from the table that (a) and
(c) converge to the root with a tolerance of 0.00001. (c) however, took fewer iterations than (a) to
converge. (b) is oscillating between, negative and positive values. (d) and (e) will converge to a root
which is outside of the interval. In many cases. solutions wil I diverge. Convergence de_peods on the
choice of expression. We can utilize the previous discussions to select a fixed-point expression that
wiJJ converge to a solution.
Table 2.2 Results of the fixed-point iteration
II (a) (b) (c) (d) (e)

0 l I I l I
l l.26491 0 1.6 - 1.25992 -2
2 l.40133 - 1.33333 1.612 1.976 I l 5.5
3 l.48026 2.41975 1.61599 2.12508 4.32231
4 l.52868 3.70262 1.61734 2.30237 4.09182
5 l .55938 4 .59542 1.6 I 780 2.49852 4.02792
6 l.57924 1.5146 1.61795 2.70158 4.00865
7 l .59225 1.33184 I .61801 2.89996 4.00270
8 1.60085 0 .83553 1.61902 3.08459 4.00084
9 1.60655 - 0 .36425 1.61903 3.24973 4.00026
10 1.61035 -1.09609 l.61903 3.39279 4.00008
11 l.61289 I. 10797 3.5 1360 4.00003
Chapter 2: Solutions of Equations in One Variable 
II (a ) (b) (c) (d) (e)
12 l.61459 0.25928 3.61359 4.00001
13 1.61572 - 1.22710 3.69503 4.00000
14 1.61649 1.79222 4.00000
15 l .61700 2.10119
16 l.61734 2.93276
17 1.61757 4.5935
18 1.61772 1.52577
19 l.61782 1.36264
20 1.61789 0.91792
21 1.61794 - 0.18684
22 1.61797 - 1.27298
23 1.61799 2.05508
24 l.61801 2.81247
25 1.61802
26 1.61803
27 1.61803
•
In summary, to find a root off (x) =0 in [a, b ], a fixed-point expression should satisfy the following
conditions.
1. g(x) should be continuous and be in [a, b] for all x in [a, b].
2. g' must exist on [ a, b] and that a constant 0 < k < I exists wi th lg'(x)I ~ k, for all x in [a, bl.
Problem Set 2.2

1. For the following functions find I.heir respective fixed-point on [0, L], if any, and show that
the fixed-point satisfies f (x) = 0 on f0, 1) if f (x) = x 3 - 5.? + 16x - 9. Use a tolerance of 10· 5 _
I
a. g,(x)=(5x 2 - 16x+9)3
3
f (x)=x - X 5X 2 + I6 x - 9
- ?
• g6 3x- - LOx+ 16
- J6x+9 - x 3 +9
g. g1 (X ) - ,- - h• g 8 (X )
X" - 5X - 5x+l6
2. 1n Prob. I .• show in each case why did the function converge (or did not converge) to the root
of f(x) =0 on [0, 1).
3. Express x 3 + I Lt 2 + I6x - 8 = 0 in the form of x = g ( x) in six different ways. Select the ones
that will have the fixed-point on the following intervals. Carry out iterations to determine the
fixed points. Use a tolerance of 10-s.
a. [- 10, - 8)
b. [-4, - 2]
c. [- 1, I]
4. Use fixed-point iterations to find solutions accurate to within I 0-5 for the following equations.
a. x 2 - 3' =0 for - 2 :;;x :;;2
b. e' + x 2 - 5 sin x = 0 for - I :;; x :;; I
c. xsinx - 3x 3 + 7 = 0 for 0.5:;; x:;; 2
5. Use fixed-point iteration to detenuine a solution accurate to within I0-1 for x 3 - 5x 2 + 8 = 0
on [0.5, 2].
6. Use fixed-point iteration to determine a solution accurate to within I 0-1 for x 3 - 5 cos (2x) - 7 = 0
on[I, 2).
7. Use fixed-point iteration to determine a solution accurate to within 10.... for 2' - 5x 2 + 12 = 0
on [- 2, 0].
8. Use fixed-point iterations to find solutions accurate to within 1o-sfor the following problems.
a. x 3 - Sx 2 +12=0, [-2,0)
b. x3 +3x + 10=0, (- 3, - 1]
c. e' - 3x - 22=0, [2, 4)
d. 5(x - 3)+sinx =0, [O, 4 )
e. 3' - e' - 17=0,[3, 5]
2.3 Newton's Method

Newton 's method is one of the most powerful methods for finding roots of equations. Consider
Figure 2.4 that shows the curve/ (x) and the tangent line at a trial point, p 0 •
Chapter 2: Solutions of Equations in One Variable <S>
J{x)
slope f'(p,)
Figure 2 .4 Iterations with Newton's method
The tangent line at p0 intersects with the x-axis at p, and p, becomes our next trial point. Now draw
the tangent line at p, and extend it. The slope at p 1 intersects with the .x-axis at p2 which becomes our
next trial point. Consider the trianglle with vertices {p2 , 0), {p1, 0) and (p1, f (p,)).
The slope,
f'(p,)= I (Pi)
P, - P2
A sequence can be generated as:
(2.7)
The stopping criteria discussed for llhe bisection method also apply to Newton's method.
Example2.4
Find the root of lhe equation x 3 - 5x 1 + 3x+4 = 0 in [0, 3) using Newton 's method. Use a tolerance
of 0.00001.
Solution
J(x)=x 3 - 5x 2 + 3x+4 J'(x ) = 3x 2 - 10x+3
We start with an initial guess of Po = I

/(Po) = /(1)=3 /'(p0 )= /'(l)=-4
f (Po) 3
P, =p0 -----'---"-'- l - - =l.75
f'(Po) -4
f (p,) = /(1.75) = - 0.70313 /'(Pt)= f'( l.75)=- 5.3125
1, = - f(p,) 1.75 - 0·70313 1.61765

f_ Pi f'(P,) - 5.3 125
f (p2 ) = f (1.61765) =0.00206 f'(p 1 )= /'(1.61765)= - 5.32612
1 = f(p 2 ) 1.61765 - 0.00206 1.61803

13 Pi f'(A) - 5.32612
f (A) = /( J.61803)=0.00000 f'(p3 ) = f'(I.61803) = - 5.32624
= f(P3 ) 1.61803 - O.OOOOO I.61803

3
p, P f'(p3 ) - 5.32624
The sequence converges to 1.61 803 with a tolerance of0.00001. So, this is the root of the equation in
[O, 3]. Keep in mind that the convergence of Newton's method depends on the initial guess. Unlike
the bisection method, Newton's method does not guarantee convergence. ■
2.3.1 Error term of Newton's method

Let us use the Taylor's series.
f(x) = f(x.)+(x - x.)f'(x.)+ (x - x.)2 J•(t;)
2
where t; is between x and x•. If pis a root, then letting x = p and f( p)=0.
0=f(x,,)+(p - xn)f'(x.)+ (p -t• )2 / 1

(/;)
or O /(x,,) +(p - x )+ (p - x,/ J' (t;)

• f'(x. ) " 2 f'(x.)
or - p= - x + f(x,,) + (p - x,,)2 f l (t;)
' " f'(x.) 2 J'(x.)
or = f(x,,) (p - x.>2 / '(t;)
'p x. J'(x,,) 2 f'(x.)
_ (p - x,,)2 1· ct;) . _
Or, p-Xn+i ---'------"-----'--'--., SlDCC, Xn-+I -X,.
2 J'(x.)
or. p - x = - (p - x.>1 f'(() (2.8)
· .., 2 f'(x.)
or P - x n+J J'(0
'(p - x.)2 2j'(x.)
Assuming x •• , = x. - f (x,.) is a better approximation than x. and t;. between x. and x,..,.
J'(x.)
As II ➔ oo, X,. ➔ p, t;. ➔ p
lim p - x,.., - lim /'((.)

•--(p- x.>1 ..-.zf'(x,.)
f'(p)
(2.9)
2/'(p)
Let us utilise the equation x 3 - 5x1 + 3x+4 = 0 that has a root in [0, 3].
Iteration
I x, = 1.75
2 x2 = 1.6176470588
3 X3 =1.6 180339929
4 X4 = J.6180339887
5 X5 = I .6180339887
f',(x) = 3x2 - 1Ox+ 3

J'(x)=6x - 10
The root, p is I .6 I 80339887

f'(p)
~.0273923238
2/'(p)
Taking 11=2
1.6 180339887 - I .6180339929
(1.6180339887 - 1.6176470588)2
p - x,,.+-J ~.0000000042
(p - x.>2 (0.0003869299) 2
~.0000000042
0.0000001497
p - x..,
(p - x.)2
~.0280561122
•
<8:> Chapter 2: Solutions of Equations in One Variable
2.3.2 The Secant method

Newton's med1od requires the value of the derivative off (x) at each iteration. This is the major
weakness of Newton's method. Because, in many cases f'(x) requires difficult mathematical opera-
tions and in some cases f' (x) is so small that it introduces significant error. The difficult mathemat-
ical operations to find f'(x) can, however, be bypassed by using an approximation to f'(x) in d1e
following manner.
(2. 10)
Using the approximation for f'(P.- ,) in Equation (2.7) we get

f (P.-,)(P-1 - P.-2) (2.11}
P,, = Pn-1- f ( Pn-1 ) - f ( P,,-2)
This technique is called the Secant method.

The Secant method can also be viewed as successive straight line approximations and picking up
the corresponding x-axis intercepts as progressively better estimate of the root. Let us start with two
initial points, (Po, f (p0 )) and (p,, f (P,)). These initial points could also be the boundaries within
which we are seeking the root. The equation of the straight line through d1ese two points can be
written as
y f (p,) - f(Po) x+c (2.12)
p, - Po
where c is the y-axis intercept. Since this straight line passes through the point ( p 0 , f ( p 0 )) , we can
find d1e y-axis intercept as
c= P,f (Po) - Pof(p,)
P, - po
The equation of the straight line in (2.12) can be written as
y= f(p,) - f (Po) x+ PJ(Po) - Pof (p.) (2.13)

P, - Po P, - Po
The x-axis intercept p2 can be determined from (2. I3) as
Pof(p,)- p,J(po)
(2. 14)
f(Pi) - f (Po)
The x-axis intercept p2 can be rewritten as
f (P,)(P, - Po)
(2. 15}
p =p, - f(p,)-f(Po)
2
Note from Equation (2. I 5) that the Secant method requires two iuiti al approximations p0 and p 1 to
begin the iterative process. The iterative process can be shown in Figure 2.5 where p2 is the x-axis
intercept of the straight line between (p0 , f (p0 )) and (p,. f (p,)), and A is the x-axis intercept of
the straight line between (p,. f (p,)) and (p2 , f (pi)).
f{x)
Figure 2.5 The Secant method
Like Newton's method, the Secant method is not guaranteed to converge. The condition for con-
ve·rgence for both methods are almost the same and the error formulas are also s imilar. The main
differences are: Newton's method requires two function evaluations f (x.) and f'(x.) per itera-
tion whereas the Secant method requires only one function evaluation f ( x.) per iteration provided
f {x._,) is retained from the previous iteration, and Newton's method converges more rapidly than
the Secant method. However, like Newton's method. when it converges it does so more rapidly than
the bisection method.
Example2.5
Find the root of the equation x 3 - Sx 2 + 3x+4 = 0 in [O. 3] us.ing the Secant method. Use a tolerance
of0.00001.
Solution.
f(x) =x3 - 5x 2 + 3x+4
We start with an initial guess of p 0 =I and p, = 3

f(p,)(p, - p0 ) (- 5)(3-1)
f(p,)= - 5 3 1.75
f(p,) - f{P0 ) - 5- 3
f(p,) =- 5 f (P2 )=-0.703l3

f(P2)(A - P1) 175 (-0.70313)(1.75 - 3) 1_545·45

J)3 = p, .
- f(pJ - f(Pt) -0.70313- (- 5)
p3=1.54545
p, = p3 f (pJ(p3 - P2) J.54545 - (0.38542)(1.54545- l.75) 1.61788

f (p3 ) - f (A) 0.38542- (-0.70313)
J)3 = 1.54545 p. = J.61788 J (P.) = 0.00083

f(p.)(p4 - PJ) (0.00083)(1.61788 - l.54545)
Ps = P, --''-'-'--"----'~ I.6 1788 -'----'-'--- - - - - - ' - I.6 I 803
J(p.) - J(p3) 0.00083- 0.38542
p, = 1.61788 Ps = 1.61803 f (p.} = 0.00083 f(p5 )=0.00000
_ f (p5 ){p5 - p,} 1. _ (0.00000)(1.61 803- 1.61788) 1. ■

61803 61803
A - Ps f(p5 ) - f(p,) 0.00000 - 0.00083
2.3.2.1 Error of the Secant method

fr,o m Equation (2.8), the error of the Secam method can be written as
(p - x,.)2 f"(t;)
p - x11+1 (2. 16)
2 /'(x,,)
where t; is between x,, and xn+r
f"(t;.)
p- x,,+, "" - (p - x,,_,)(p - x,,) .f'(x.) (2.17)
2
For x. and x••1 near to p
(2. 18)
2.3.3 Method of false position

The method of false position starts by approximating the actual function with a straight line within
Lhe bow1dary. The straight line intercept on the x-axis is considered a candidate forthe root. However,
due to the approximation, the x-axis may not fall on the root, hence the name, method of false posi-
tion. The x-axis intercept divides the initial zone into two zones. Test is performed to keep the zone
that has the root The straight line approximation is then applied in the new zone and a new x-axis
intercept is found. The process is repeated UJ1til the x-axis intercept approaches close to the root with
a given tolerance.
Note that the method of false position generates successive estimates for the root in the same manner
as the Secant method. However, in ltbe method of false position a test is perfonned in each iteration
to ensure that the root is always bracketed in the next iteration.
Let us start with two initial points, (p0 ,/(p0 )) and (Pi,f (p,)). p2 is the x-axis intercept of the
straight line between these two points. Now, bracket the root between either (p0 , p1 ) or (p,, p 2 )
in the following way.
If /(Po)·/ (p2 ) < 0, then bracket tbe root between p 0 and A· Determine x-axis intercept A of the
straight line joining (Po,/ (p0 )) and (p2 , f (pi)).
If on the other hand / (Pt)· /{p2 ) <0, then bracket the root between Pi and p,. Determine x-axis
intercept p3 of the straight line joining (p,, f (p, )) and (p2 , f (Pa)).
PJ is an improvement over p 2• 'TI1e iterative process will continue until the absolute difference
between the successive x-axis intercepts becomes less than or equal to a given tolerance.
Example2.6
Find the root of d1e equation x 3 - 5sin (x)+ 24 = 0 in [-6, 6] using the method of false position. Use
a tolerance of 0.0000 l.
Solutio11
We start wid1 an initial guess of p 0 = -6 and p, = 6.
Iteration I
Calculate Pi-
Po · f(p,) - P, · f(Po) p =-0.

66238
P2 f(p,) - /(Po) ' i
Sincef(p0 )· /(p1 )<0, the root is bracketed in [-6,-0.66238].
Iteration 2
Calculate p3 •
Since f (Po)· f (A)< 0 , the root is bracketed in [-6, -l.31169].
Iteration 2
Calculate p..
Po· f(p3) - P1 · f(Po), p. = - 1.87811

f(PJ) - f(Po)
Since f (p0 ) · f(pJ<O, the root is bracketed in [-6, - 1.8781 I].
The results of the successive iterations arc shown in Table 2.3.
Table 2.3 Results of the false position method example
II Root is bracketed within P.

I -6 6 - 0.66238
2 -6 -0.66238 - 1.31169
3 -6 - 1.31169 - l.8781 1
4 -6 - 1.87811 - 2.30153
5 -6 - 230153 - 2.57648
6 -6 - 2.57648 - 2.73797
7 -6 - 2.73797 - 2.82719
8 -6 - 2.827 19 - 2.87484
9 -6 - 2.87484 - 2.89982
10 -6 - 2.89982 - 2.91279
11 -6 - 2.91279 - 2.91950
12 -6 - 2.91950 - 2.92295
13 -6 - 2.92295 - 2.92473
14 -6 - 2.92473 - 2.92565
15 -6 - 2.92565 - 2.92612
16 -6 - 2.926 12 - 2.92636
17 -6 - 2.92636 - 2.92648
18 -6 - 2.92648 - 2.92655
19 -6 - 2.92655 - 2.92658
20 -6 - 2.92658 - 2.92660
21 -6 - 2.92660 - 2.92660
•
Problem Set 2.3
1. Use Newton's method to find sol utions accurate to within JO..., for x3 - 5x 2 +16x - 9 =0 on
[O, l].
2. Use Newton's method to find solutions accurate to within 10_. for x 3 + llx 2 + 16x - 8 = 0 on
each interval.
a , [- 10, - 8) b. (-4,- 2) c. [- 1, 1)
Chapter 2: Solutions of Equations in One Variable <8>
3. Use the Newton 's method to find solutions accurate to within I 0-5 for the following
equations.
a. x 2 - 3' =0for - 2$x$2
b. e' +x 2 - 5sinx=0for -l $x$1
c. xsinx - 3x 3 + 7 =0 for 0.5 $x $ 2
4. Use the Secant method to find solutions accurate to within IQ--1 for x 3 - 5x 2 +16x - 9=0
on [0, 1] .
5. Use the Secant method 10 find solutions accurate to within JO-ii for x 3 + llx2 + 16.r - 8 = 0
on each interval.
a. [-1 0, - 8] b. (-4, - 2] c. [- 1, J]
6. Use the method of false position to detennine a solution accurate to within 10.... for
x 3 - 5x 1 +8 = 0 on [0.5, 2].
7. Use the method of false position to detem1ine a solution accurate to within 10.... for
x 3 - 5cos(2x) - 7=0 on [l , 2].
8. Use the method of false position to detennine a solution accurate to within 10_. for
2' - 5x 1 + 12 = 0 on [- 2, OJ.
9. Use Newton's method to find solutions accurate to within 10-5 for the following problems.
a. x 3 - 5x 2 + 12 =0, [- 2, O] b. x 3 +3x +JO= 0, [- 3, - 1]
c. e' - 3x - 22=0, (2,4] d. 5(x - 3)+sinx=0, [0, 4]
e. 3' - e' - 17=0, [3, 5] f. ln(x - 3) + sin(x - 3)=0, [3, 5]

10. Repeat all problems in 9. using the Secant method.
11. Repeat all problems in 9. using the method of false position.
2.4 Miiller1sMethod
Miiller's method uses three initial points x0 , x, and x 2 to construct a quadratic polynomial. The poly-
nomial intersects with the x-axis at X:r This point is utilized as an improved csti!llate for the root p
of p (x) = 0. The process is then repeated with x 1, x 2 , x3 and then with x1 , x 3, x4 and so on.
Miiller's method can be used to find both real and complex roots.
Lei us construct a quadratic polynomial g(x) with the three initial points x0 , x, and x1 .
g(x) =a0 +a, (x - x 0 )+a2 (x - x0 )(x - x,) (2. I9)
g(x) should satisfy the following condition

g(x, )= f (x,), i =0, I, 2, ...
Determine the coefficients a 0 , a, and a1 in the following manner.

Set x=x0
g(xo)=ao
Go= f (xo) (2.20)
Set x =x,
g(x,)=a0 +a, (x, - x 0 )
g(x,) - a0
a, =
x, - xo
(2.21)
g(x1 ) =a0 +a, (x2 - x0 )+a1 (x2 - x0 )(x2 - x,)

f(x 2 ) = f (x0 )+a, (x2 - x0 )+£½ (x1 - x0 )(x2 - x,)
f (x2 ) - f (x0 ) a,
a.., =
- (x 2 - x0 )(x2 - x1 ) (x2 - x,)
Using (2.2 1) we get
a, = /(x2 ) - /(x0 ) f(x,) - f(x0 )
(2.22)
- (x2 - x0 )(x2 - x,) (x2 - x,)(x, - x 0 )
Calculate x 3, the point of intersection between g(x) and the x-axis, in the following manner.
0=a0 +a, (x3 - x0 )+a2 (x1 - x0 )(,s - x,)
- {a, - {½ (x, +x0 )}± J{a, - a2 (x, +x0 )} - 41½ (a0 - a,x0 +(½x0 x,)
2
x,= - - - - - - ~ - - - - - - - - - - - - - (2.23)
. 2c'2
Example2.7
Let us utilize the previous equation x 3 - 5.r 2 + 3x + 4 = 0 that has a root in [0, 3].
Solutio11
Choose three initial points.
x0 =0, x, =land x 2 =2.
Calculate x3 and then replace x0 = x,, x, =x1 and -~ = .r3 to do the !llext iteration. Equation (2.23)
su_ggests that there will be two values of x 3• Choose the one that is within [0, 3). The results of the
subsequent iterations are shown in Table 2.4.
Chapter 2: Solutions of Equations in One Variable 
Table 2.4 Results of the example of Miiller's method
Xo Xi X2
0.0 1.0 2.0

l.O 2.0 J.6861406616
2.0 1.6861406616 1.6148647603
1.6861406616 l.6148647603 J.6180185807
1.6148647603 1.6180185807 1.6180339894
l.6 180185807 1.6180339894 1.6 I 80339887
1.6180339894 l.6 180339887 J.6180339887
•
Problem Set 2.4
1. Use MUiler's method to find solutions accurate to within IO_. for 3x3 - 5x 2 + 16x - 9=0
on [0, 1).
2. Use MUiler's method to find solutions accurate to wilhin I()-4 for x 3 + I lx 2 + l 6x - 23 = 0
on each interval.
a. [-10, - 8) b. [-4, - 2) c. [- 1, 2]
3. Use MUller·s method 10 determine a solution accurate to within 10_. for x 3 - 5x2 + 11 = 0
on [I, 2).
4. Use Muller's method to determine a solution accurate to within 10..... for x 3 - 5x 2 + l l = 0
on [- 2, 0].
5. Use MWler's method to determine a solution accurate to within JO"" for x 3 - 5cos(2x)- 7 = 0
on [I, 2).
6. Use MWler's method to de,tennine a solution accurate to within 10..., for 2' - 5x2 + 12=0
on [- 2, 0).
7. Use MW1er's method to find solutions accurate to within I0-5 for the following problems.
a. x 3 - 7x 2 +12=0, [- 2,0] b. x 3 +3x+l0=0, [-3, - 1]
c. e' - 3x - 22=0, (2,4] d. 5(x - 3)+sinx=O, [0, 4)
e. 3' - e' - 17=0, [3, 5]
2.5 Brent's Algorithm

Brent's algorithm uses inverse quadratic interpolation and bisection method to find a root. A test is
done in each iteration to bracket the root. Using three initial points, of which the two extreme ones
denotes the boundary containing the root, an inverse quadratic polynomial is obtained. The point
where the polynomial crosses tbe x-axis is calculated. This point is taken as tbe next estimate of the
root. If the estimate falls outside of the root bracket, bisection method is applied in the next iteration
in order to bracket tbe root. If the estimate falls inside the root bracke1. then inverse quadratic inter-
polation is utilized in the next iteration 10 estimate 1he root.
Take three initial points { a, f (a)}, {b, f (b )} and {c, f (c)} such that f (a)· f (b) < 0 and a< c < b.
The root is bracketed in [a, b]. The inverse quadratic polynomial ([QP) passing through the tbree
initial points is
{y- J(b)}{ y - /(c)} {y - /(a)}{y- /(b}}
x= - ~ - - ~ - - - -a+ - - - - ~ - - ~ ~c
{J(a) - f(b)}{J(a) - f(c)} {f(c) - f(a)}{J(c) - J(b)}
(2.24)
+ {y- J(a)}{ y - f (c)} b
{/(b)- f(a)}{J(b) - /(c)}
Since we are seeking for the x-axis .intercept (i.e., wbere y = 0),
x= J(b).f(c) a+ J(a)f(b) c
{J(a) - J(b)}{J(a) - /(c)} {/(c)- /(a)}{/(c)- J(b)}
(2.25)
+ f(a)J(c) b
{J(b) - f(a)}{J(b) - f(c)}
x is the next estimate of tbe root.
If x < a or x > b, use bisection method to bracket the root. If a < x < b, use the inverse quadratic
polynomial with the following update.
If J(a)· f(x) < 0, then b ➔ x . On the other hand, if f (x) · f(b) <0, then a ➔ x. c can be chosen as
tbe midpoint between a and b.
Example2.8
Take the previous equation used in ·the bisection method.
e-~ +x3 - 5sin (x) - 13=0 has a root in [- 6, - 2} Our desired tolerance is 0.00001.
Solutio11
Jtcralion I
Obtain tbe inverse quadratic polynomial that passes through the points a =-6, c =-4 and
b=- 2. Obviously, for each of these three points we have to evaluate !heir corresponding y value.
Using (2. l8)
r = /(- 2)/(-4) ·(- 6)+ /(-6)/(- 2) ·(-4)

• I {/(-6) - /(- 2)}{/(-6) - /(-4)} {/(-4)- /(-6)}{/(-4) - /(- 2)}
+ {J(-2)-/4~}{;~;)-f( --4)} ·(- 2)

x, = - 1.10651
- 1.10651 is outside of[-6, - 2]. Therefore, apply bisection method.
p= a+b = -6 - 2 =-4
2 2
Check if the root should be bracketed in [a, p] or in (p, b].
/(-6)-/(-4)<0
Therefore, the root should be bracketed in [-6,-4].
Iteration 2
Obtain the inverse quadratic polynomial that passes through the points a =-6, c= - 5 and b =-4.
X - f(-4)/(- 5) . -6 + /(-6)/(-4) . -5
2
- {/ (-6)- /(-4)}{/(-6) - f (- 5)} ( ) {/ (- 5) - /(- 6)}{/(- 5) - /(-4)} ( )
/(-6)/(- 5) -4
+ {/ (-4) - /(- 6)}{/(-4)- f (- 5)} ( )
X2 =-4.84215
-4.84215 is insidef-6,-4].
f (a}· J(x2 ) < 0. Therefore, b ➔ -4.84215, i.e. the root is bracketed in [-6. -4.84215].
Iteration 3
Obtain the inverse quadratic polynomial that passes through the points a =-6. c=- 5.42108 and
b=-4.84215.
/(-4.84215)/(- 5.42108) -6
3
X = {/(-6) - f(-4.842 15)}{/(-6) - /(- 5.42!08)} ·( )
+ /{-6)/(-4.84215) ·(- 5.42!08)

{J(- 5.42fi08) - /{-6)}{/(- 5.42108) - /(-4.84215)}
+ /(-6)/(- 5.42108 ) · (-4.84215)

{/(-4.84215) - /(- 6)}{/(-4.842 15) - /(- 5.42108)}
X3 =-4.90018
-4.90018 is inside [- 6, -4.84215].

f(a)·f(x 3 )<0. Therefore, b ➔ -4.90018,i.e. the root is bracketed in [-6,-4.90018].
Jtcralion 4
Obtain the inverse quadratic polynomial that passes through the points a =-6, c = - 5.45009 and
b=-4.90018.
_ / (-4.90018) / (- 5.45009) -6
x. - {J(- 6)- f (-4.90018)}{/(- 6) - / (- 5.45009)} · ( )
+ f(-6)/ (-4.90018) ·(- 5 45009)
{f(- 5.45009) - f(-6)}{/(- 5.45009) - /(-4.90018)} .
+ f(- 6)/(- 5.45009) ·(-4.90018)
{f(-4.90018) - /(- 6)}{/(-4.90018) - /(- 5.45009)}
x. =-4.91475
Table 2.5 Re.~uJts of Brent's rnerbod
n a. b,. c. x. Comment Next

Iteration
1 -6 -2 -4 - 1.10651 outside bisection
2 -6 -2 -4 -4.84215 inside IQP
3 -6 -4.842 15 - 5.42108 - 4.9001& inside IQP
4 -6 -4.90018 - 5.45009 -4.91475 inside IQP
5 -6 -4.91475 - 5.45738 - 4.9184& inside IQP
6 -6 -4.91848 - 5.45924 - 4.91944 inside IQP
7 -6 - 4.91944 - 5.45972 - 4.91969 inside IQP
8 -6 -4.91969 - 5.45985 - 4.91976 inside IQP
9 -6 -4.91976 - 5.45988 - 4.91977 inside IQP
10 -6 -4.91977 - 5.45989 -4.91978 inside IQP
11 -6 -4.91978 - 5.45989 - 4.9197& converged stop
•
Problem Set 2.5
1. Use Brent's med1od to find solutions accurate to widiin 10..... for x 3 - J4x 1 +11 x + 2 1=0
on [- 2, 0l
2. Use Brent's med10d to find solutions accurate to widiin 10-" for x 3 - l4x 2 + I lx + 2 1=0 on
[l, 3].
3. Use Brent's method to find solutions accurate to widiin IO-" for x 3 +11x 2 + I6x - 8 = 0
on each interval.
a. [- 10, - 8] b. [-4, - 2] c. [- 1, I]
4. Use B rent's method to determine a solution accurate to within 10-1 for x 3 - 5x 1 + 8 = 0 on
[0.5,2).
5. Use Brent's method to detemune a solution accurate to within 10.... for x 3 - 5cos(2x) - 7 = 0
on [I, 2].
6. Use Brent's med1od to determine a solution accurate to within I0 .... for 2' - 5x2 +12 = 0 on
[- 2, O].
7. Use Brent's method to find solutions accurate to within 10-s for d1e following problems.
a. x3 - 5x2 + 12aa0, (- 2, 0) b. x3 + 3x +!0aa0, [- 3, - I]
c. e' - 3x - 22=0,[2.4] d. 5(x - 3)+sinx=0,[0,4]
e. 3' - e' - 17 = 0, (3, 5)
2.6 Repeated Roots

Root finding methods discussed so far, do not work well if an equation has repeated roots. In a
repeated root scenario, we have to find the distinct roots as well as their multiplicity.
Definition: Multiplicity of a root
Assume that f(x) and its derivativesf'(x), f'(x) . .... f (m) (x} are defined and co111i11uo11s in the
11eighbo11rhood of Ao· Then f (x)= 0 has a root of multiplicity mat .x =lo if and 011/y if f (lo) = 0.
/' (A0 )= 0, /"(Ao)= 0, ... , .r-•,
(Ao)=0 tmd f'">(Ao) ¢0.
A root is a simple root if m = I and is a repeared (mu!Tiple) root if m ~ 2.
If J (x) is continuous in the neighbourhood of its simple root, Ao, then it can be factored in the f onn
f (x) = (x - Ao) <p0 (x)
where <p0 (x) is co111in11ous in the neighbourhood of Ao and <p0 {Ao)* 0.

Assume that f (x) has n roots, among them are k distinct roots, denoted by A1, i = I, 2, ... , k and with
multiplicity 111;, i = I, 2, . . . , k where ' =11, then f (x) caa be expressed as

I,m,
l=I
t
f(x)= <p(x)I1(x - A1 )"'
l=l
*
where <p(x) 0 is continuous in each neighbourhood of A1, i = 1, 2, .... k.
f (x) and its first derivative f' (x) have only one greatest common factor
k
fc(x)= Il(x - A,)"'' _,
such that/ (x) = fc (x) / 0 (x) and/' (x) = f. (x)J, (x).

fo (x) has onl y simple roots. k simple roots d1at are exactly the same k distinct roots, ,i,, as those of
f{x).
Consider the following equation.

1
(x - a)(x - p) (x - q)'" =0
The equation bas three distiuct roots, a simple root a and multiple roots p and q with respective
multiplicity of / and m. If a polynomial equation is presented in d1is fashion or can be converted
to this factorized form. then the roots and their multiplicity can be detennined with little difficulty.
However, most often a polynomial equation will be presented in the following manner.
•
I,ax =0 1
1
1:-0
where a1 is the ith coefficient.
We can divide the solution as a two-step process. In d1e first step, the: distinct roots are found and in
Lhe second step their multiplicities are determined.
Step 1
In order to find die distinct roots. we have to collect me distinct factors of the polynomial expression
ignoring d1eir multiplicity.
1
Let/ (x) = (x - a) (x - p) (x - q)'"
The first derivative off (x) is
g(x) = f'(x)=(x - p)' (x - qf' +l(x- a)(x - pf' (x - q)'" +111 (x - a)(x - p)' (x - q)"'-'
Note mat g (x) does not contain me common factor for simple root, however, it contains the common
factors off (x) for multiple roots whose indices have been reduced by one due to differentiation.
This makes it possible to isolate the greatest common divisor of /(x) and g(x) utilizing a suitable
technique. We use Euclidean Remainder to find dle greatest common divisor (GCD).
Euclidean remainder
Definition: Polynomial Division. Let p(x) and d (x) be polynomials with d (x) ;t, 0. Then there exist
unique polynomials, me quotient q ( x) and ilie remainder r (x) such that
p(x)"' d(x)q(x)+ r (x)
wim either r(x) = 0 or the degree of r(x) is less than that of d (x).
TJ,eorem: Euclidean Theorem for Polynomials. Let f (x) and g(x) be nonzero polynomials and r(x)
be the re111ai11derob1ai11ed by dividi11g f (x)by g(x). If GCD (J (x), g (x)) de1101es 1he GCD off (x)
and g(x), then GCD (J (x), g (x)) =GCD (g (x), r (x))
Proof
Let/, (x) be the GCD off (x) and g(x) such mat
f(x)= f, (x)q1 ( ,~) and g(x)= le (x)q, (x)
where q1 (x) and q, (x) are nonzero and have no common divisors. Dividing f (x) by g(x),
f.. (x)q1 (x) = f. (x) q, (x)q(x)+ r(x).
Rearranging the above,
q1 (x) - q, (x)q(x) and q, (x) have no common divisor since there is no common divisor of q1 (x)
and q, (x). Therefore,/, (x) is the GCD of g (x) and r(x).
The Euclidean algorithm for polynomials use successive polynomial division starting by dividing
f (x) by g(x). The divisor and the remainder of any division are arranged respectively as the divi-
dend and the divisor in the subsequent division yielding a sequence of remainders, called Euclidean
remainder sequence. The repetitive divisions are continued until the remainder is zero. Ultimately,
Lhe last nonzero remainder in the sequence is the GCD of/ (x) and g(x).
Example of Euclidean remainder

If ,i is the GCD of a and b, then there are integers II and v such that d =au+ bv. Let us give an exam-
ple with two integers first. We would like to find the GCD of 96 and 40. We can prepare the follow-
ing table of successive divisions and remainders in the process of finding the GCD of 96 and 40.
a=96 40=b
2b=80 32 =2a - 4b
lj = a - 2b = 16 8=- 2a+5b =r2
-4a+l0b= 16
r3 = 5a -1 2b=0
The last nonzero remainder is 8 which is the GCD of 96 and 40. 8 can be expressed as 8 =- 2a + 5b,
and, therefore, 11 = - 2 and v = 5.
Poly11omia/ extended Euclidean remainder

Now we find the GCD of two polynomials. The procedure is the same as above, except we would
be dealing with polynomials. If d(x) is the GCD of a(x) and b(x), then there are polynomials u(x)
and 1•(x) such that d(x)=a(x)u(x)+b(x)v(x).
Let a(x)=xs +x• - 5x3 - x 2 +&x - 4 and b(x) = .l - 2x3 - 3x 2 + 8x -4
t1(x)=xs + x• - 5x 3 - x 2 +8x-4 x• - 2x 3 - 3x 1 +8x - 4=b(x)
(x+3)b(x) = xs +x• - 9x3 - x 1 +20x - 12 x• - 2x 3 - 3x 2 +8x - 4
= (x - 2).!._{ a(x) - (x 2 - x)b(x)}

4
lj = a(x) - (x +3)b(x)=4x 3 -12x+8
l
- {a(x) - (x+3)b(x)}=x 3 - 3x+2
4
Note that in determining the polynomial extended Euclidean remainders, we convert the polynomi-
als into manic at each stage if necessary. The last nonzero remainder is
x 3 - 3x+2
which is the GCD of a(x) and b(x).

The greatest common divisor (GCD) of f(x) and g(x) is
J,,(x) = (x - pt' (x - qf-'
We can, therefore, write
/(x) = J,, (x)s (x) and g (x) = Jc (x)t (x)
Dividing /(x) by h(x) gives us the just the distinct factors.
s(x)= f(x) (.r - a)(x - p)'(x- qr (x - a)(x - )(x - )

1- 1 ( ),._ , p q
f,, ( x. ) ( x - p) x - q
Utilize any conventional technique to find the roots of s (x) = 0 which are d1e distinct roots off (x) = 0.
Step 2
Note mat the indices of the distinct factors off (x) appear as multipliers in g(x).
Divide g (x) by h (x).
t(x) = g(x) =(x - p)(x - q)+l(x - a)(x - q)+m(x - a)(x - p)

h(x)
t ( i) (x - p)(x - q)+I (x - a) (x - q)+m(x - a) (x - p) th

Let v (x) = - · - -'--'--'--''----'-'--'----'--'---'-'----'----'--'--'-'-., en
s'(x) ~ - ~(x - q)+(x - a)~ - q)+(x - a)~ - p)
(x - p)(x - q)+l(x - a)(.x - q)+m(x - a)(x - p)

v(x) I.'"' = (x - p)(x - q)+(x - a)(x - q)+(x - a)(x - p) ,.,,
=I, multipl icity of root a
(x - p)(x - q)+l(x - a)(x - q)+m(x - a)(x - p) . . . f
v(x)l,=p ( x - p)( x - q ) + ( x - a)( x - q ) + ( x - a )( x - p) ,=p = 1, mulupllc1ty o root p
and
V(x
·)I _(x - p)(x - q)+l(x - a)(.x - q)+m(x - a)(x - p)
= 111, multiplicity of root q.
· •=q (x - p)(x - q)+(x - a)(x - q)+(x - a)(x - p) , =q
Example2.9
Fillld the roots and the multiplicity of the roots of the following polynomial equation.
x 1 +5x6 +3x5 - l7x4 - 16x3 + 24x 2 + 16x - 16=0
Chapter 2: Solutions of Equations in One Variable ~
Solution
Let /(x)=x 7 +5x6 +3x 5 - 17x4 - 16x3 + 24x 2 + 16x - l6
Obtain the first derivative of /(x).
g(x) = f'(x) =1x6 +30x 5 + 15x' - 68x 3 - 48x 2 + 48x+ 16
Convert g(x) to a manic only to find the GCD.

6 30 3 15 4 68 3 48 2 48 16
c.,(x)=x + x + x - x - x + x+
1 1 1 1 1 1
The GCD off(x) and g(x) is
5 3
h (x) =x +4x' + -t - 10x 2 - 4x+8
s(x) = f(x) =x2 +x - 2

h(x)
The distinct roots are determined from s(x) =x1 +x - 2= 0, which a.e I and - 2.
r(x) = g(x)=7x+2
h(x)
v(x) = r(x) 1x+ 2

s'(x) 2x+ 1
v(x )I 2
= ?x + 1 = 3 i.e., the root at J has a multiplicity of 3, and
,=, 2x + I .,=,
v(x)I =7x + 21 =4 i.e., the root at - 2 has a multiplicity of 4.
= -i 2x + I , =-l
2.7 Convergence
Convergence analysis is performed to find if a solution process will be stable and lead to an answer.
In addition, the analysis can be utilised to ascertain the rate at which a solution process would
converge to an answer.
Rate of Convergence
Rate of convergence indicates the speed at which a so lution process would converge to an answer.
Several sol ution processes or techni ques can be compared with respect to their rate of convergence
to ascertain computational efficiency.
LiJ1ear Co11verge11.ce
A sequence {x,} is said to converge linearly to x· if there is a constant 0<c<J such that
xt+i - x • ~ ext - .i/ for all k > N for some natural number N > 0.
We may, lherefore, say tha1 this sequence converges linearly to x·
Where O< c < I, c is called the rate of convergence.
Con vergence with order of q

A sequence converges with order of q to x • if
If q = 2, then it is called quadratic convergence

q =3, then it is called cubic convergence etc.
2.8 Accelerating Convergence
In most cases we do not have the luxury of quadratic convergence. Some modifications can be done
in the iteration algorithms that are linearly convergent to converge at an accelerated rate.
2.8.1 Aitken's extrapolation for linearly convergent sequences

Let {p. }:=o is a linearly convergent sequence with limit p. In many cases the sequence converges
ve,ry slowly top. If p - p,,, p - P••, and p - p..2 have the same sign or have a panem of alternating
signs, and
p - p,,. , "'A-<I
p - p.
wlnere A is a constant and 11 ~ 0,
then Aitken 's extrapolation technique commonly known as Aidcen's I:!. 2 method can be used to mod-
ify the sequence so that it converges at a faster rate.
Assume that in an iterative process the approximation improves as II increases. We can write
p - p,,.,
p-' -2p· Pn+I + (P.,+I )2 "'p-, - p· P..2 - P· Pn + Pn · Pn+2

2
P(P.+2 - 2p.,., + P.)"' P••1 • P. - (P•• , )
1
P"' Pn+2 · Pn - (P.+1)
P.+2 - 2p,,., + P.
Chapter 2: Solutions of Equations in One Variable <e>
Add and subtract the terms P. 2 and 2p. · P.+i in the numerator.
_ p,. (Pn+2 - 2p,.., + p,. )- (p,.2 - 2pn · Pn+I + Pn+.2)

p- 2 +
Pn+1 - P11+1 Pn
(2.26)
Denote the Aitken 's extrapolation sequence as

(P.+1 - P.)2 (2.27)
P••2- 2JJ,,.1+P.
to rustinguish it from the original sequence.
The ratio A can be computed in the following manner.
Pn+i - Pn+I (P - Pn+1) - (P - Pn+2)

Pn+I - pn (p - P.) - (p - pn+,)
l - p - JJ..2
P,,+2 - Pn+I p - p,,.._,
P,,+, - p,, p - p,,
p - p •• ,
P,,+i - Pn+I ._ ~- A
P,1+1 - p,, -- 1
l
P,,+i - Pn+I "'A
P,1+1 - 1',,
Example 2.10
Consider the equation x 3 - 7./ +4e';nc,J +8=0 that has a root in [l, 3]. We can utilize a fixed-point
iteration by using x = g (x)
x 3 + 4e"•(•) + 8
where g (x) = .1- - - - -.
7
Table 2.6 shows the results using the fixed-point iteration and Aitken 's extrapolation. For a tolerance
w-
of 1 , fixed-pointrcquires 14 iteration whereas Aitkcn's extrapolation requires 5 iteration.
<B:> Chapter 2 : Solutions of Equations in One Variable
Table 2.6 Results of a fixed-point iteration and Aitke11's extrapolation
n P. 'I.
0 1.0 l.9 119457
I 1.6 159521 l.8946464
2 1.8 158739 l.8943805
3 1.8723815 l.8944179
4 1.8882 160 1.8944223
5 1.8926724 J.8944227
6 1.8939289
7 1.8942834
8 1.8943834
9 1.8944116
10 1.8944196
11 1.8944218
12 1.8944225
13 1.8944226
14 l.8944227
•
2.8.2 Steffensen's acceleration
Steffensen's acceleration is a slight modification of Aitken's t!,.2 method as applied to fixed-point
iterations. In Aitken 's t!,.2 method as applied to a fixed-poi nt iteration the sequence proceeds as
Po, P1 =c(Po), P? =g(P1), %
Pt, P2, p3 =g(P2) , q,
P2, PJ, P, = g(PJ ), q2
where q. is obtained using (2.27).
In Steffensen's method, l/o, q1 are used as initial values for the fixed-point iteration in the following
manner.
p/l, Pt(OJ =C(Po(O) ), p /OJ =g(pt> ), l/o(O)
p/> =q/>, Pt(I) =g(p/>),p/l =g(p.<1>), q/l
), p}2l = 8 (P/21), q/ 1
Po(2) = q/1, p/ 2>=8 (Po<2>
Amd soon.
Chapter 2: Solutions of Equations in One Variable <§'.>
Example 2.11
Consider the fixed-point problem shown in Section 2.7. I. Table 2. 7 shows the results of fixed-point
iteration with Steffenscn's acceleration.
Table 2.7 Results of a fixed-poim iteration with Steffensen's acceleration
k p/t) P 1(t) p/ll
0 I l.615952055 l .815873924
1 1.911945689 1.899376500 l.895821228
2 J.8944 I8955 l.894421644 l.894422402
3 1.8944227 1.8944227
•
_J
Systems of linear equalions arise in solving real-world problems. These type of equations also arise
in solvi ng systems of nonlinear eqlllations as well as boundary value and initial value problems.
In this chapter. direct methods of solving systems of linear egualions will be presented.
3.1 Systems of linear Equations

A system of linear equations can be expressed as
a,,x, +a11 x 2 +···+a,.x. =b,

0 21.\".1 +a22··"'.?: +···+a2n·.rn = b~
(3.1)
where x,, x 2 , •••, x. are the unknowns and a 11 , a,2' ..., a,,,, are the respective coefficients.
We can solve the system shown in (3. I) by reducing it. The steps taken to reduce a system of linear
equations are a combination of the following three operations.
I. Interchange of two equations
Il. Multiply an equation by a nonzero constant
m. Multiply an equation by a nonzero constant and add that to another equation
~ CHAPTER 3: Solutions of Linear Systems of Equations
Let us consider the following system of linear equations.

x, - 2x2 - 3x3 = -4
3x, - 5x2 +4x3 = 25 (3.2)
4x1 - 6x2 - 3x3 = 7
At this stage we will assume that tl!le three equations in (3.2) are independent with respect to each
oth er and, therefore. a unique solution exists for the three unknowns.
Multiply the first equation of (3.2) by - 3 and add that to the second equation. The resulting equation
bcccomes the new second equation. Similarly, multiply the first equation of (3.2) by -4 and add that
to the third equation. The resulting equation becomes lhe new third equation. The modified system
of linear equations is shown in (3.3). TI1e modified system of equations will have the same solution
as the original system.
X1- 2X2 - 3x.3 =-4
X2 + J3x3 = 37 (3.3)
2x2 + 9x3 =23
Next, multiply the second equation of (3.3) by - 2 and add that to the third equation of (3.3). The result-
ing equation becomes the new third equation. The modified system of linear equations is shown in (3.4).
Again, the modified system of equations will have the same solution as that of the original system.
x, - 2x 2 - 3x3 = -4
X2 + l3x3 =37 (3.4)
- 17x3 = - 51
The system of linear equations is now reduced to the point that we can easily solve for the unknowns
if we start from the third equation. We get x 3 =3. Now we move to the second equation and solve
for x 2 , we get x2 = - 2. Wi th x 3 and x 2 become known, we now move to the first equation and solve
for x,. We get x, = I. This method is commonly known as backward-substitution med1od. Note that
we started the backward-substitution when the system of linear equations is reduced to a triangular
form as shown in (3.4).
3.2 Linear Dependence and Independence

A.l!l equation in a set of equations is linearly independent if it cannot be generated by any linear
combination of the other equations. If an equation in a set of equations can be generated by a linear
combination of the od1er equations then it is called a dependent equation.
In order to have a unique solution for a set of unknowns, the otllllber of independent equations must
be at least equal to the number of unknowns. Therefore, to solve for three unknowns in a unique way,
we need at least three independent equations.
Let us consider the following set of equations.
2x1 - 4x2 + 5x3 = 36
- 3x1 +5x2 + 7x3 = 7 (3.5)
CHAPTER 3: Solutions of Linear Systems of Equations 0
It can be easily noticed that the third equation in (3.5) can be generated by multiplying the first
equation in (3.5) with 2. Therefore, the third equation in (3.5) is a linearly dependent equation.
2x1 - 4x2 + 5x3 = 36
- 3x,+5x2 + 7x3 =7 (3.6)
- x 1+ X2 + J2x3 = 43
In (3.6), d1e third equation can be generated by adding the first equation 10 the second equation.
Therefore, the third equation in (3.6) is a dependent equation.
2x1 - 4x2+ 5.rJ. =36
-6.r, + 8x2+43xJ. = 136 (3.7)
- 3x, + 5x2+ 7.rJ. =7
In (3.7), the second equation can be generated by adding three times of the first equations to four
times of d1e third equation. Therefore, dle second equation in (3.7) is a dependent equation.
Dependent equations do not add any new information to the solution process and. therefore, lack
unique solutions. However, dependent equations may result in multiple arbitrary solutions.
2x1 - 4x2 +Sx3 =36
- 3x1 +5x2 + 7x3 =7 (3.8)
5x 1 +3x-i - 8x3 = - 31
All three equations in (3.8) are linearly independem with respect to each oilier. These three equa-
tions provide a unique solution for the three unknowns: x, y and ;:. Using the backward-substitution
medlod we get x, =2, x2 = - 3 and x 3 =4.
A system of Linear equations is independent if
k , (eqn. l)+k1 (eqn. 2)+···+k., (eqn. m) ,;eQ (3.9)
for all values of k,, k"l, ... , km except k, = k2 = · · · = k.,. = 0.
3.3 Matrix Representation of Systems of Linear Equations

Systems of linear equations can be .represented by matrices and these representations are very
convenient for applying matrix algebra not only to solve for the unknowns but also to analyze the
characteristics of the physical systems modelled by these systems of Linear equations.
The system of linear equations shown in (3.1 ) can be written as
Ax= b (3.10)
where A is an m x 11 matrix, x is an 11-dimensional colwno vector and b is an 111-din1ensioual colwno

vector.
a" a,1 a,.

0 21 022 a2,,
A= (3. 11)
a.,, a,_,,1 a.""'
x,
X2
x= (3. 12)
x.
and
b,
b=
b2 (3. 13)
b,.
A is a square matrix if 111 = 11 , otherwise it is a rectangular matrix. In most cases, we will be dealing
with square matrices.
3.4 Gauss Elimination

Gauss elimination reduces a syste m of linear equations into a triangular form by utilizing a combi-
nation of tbree row operations and then uses backward-snbstin1tion to obtain tbe solution. Note that
this what we have essentially done in Section 3.1. However, in Gauss elimination we represent the
linear system with the he lp of an augmented matrix.
Am augmented matrix is an array that assembles both A and b together.
a" a,2 a,,. b,

a2, a22 t½. b2
[A, b]= (3.14)
a,,-' a n2 a.,, b.
where A is considered a square matrix of dimension II x II and b is an 11-dimensional column vector.

The three row operations that are used to reduce the linear systems are:
L Interchange of two rows,
Il. Multiply a row by a nonzero constant and
m Multiply a row by a nonzero constant and then add that to another row.
CHAPTER 3: Solutions of Linear Systems of Equations 
The augmented matrix is reduced in such a way that the first elements of the second to the last rows
are eliminated, then the second elements of the third to the last rows arc eliminated and then the third
elements of the fourth to the last rows are eliminated, until we reach the last row.
Use a 11 as the pivot element to eliminate a 21, a 3 1, ••• , a.,, from the respective rows using the following
row operations.
R' ➔ R +(-~)R
l 3
Gu
I
AJthough the row operations above will modify the elements of the augmented matrix, we will keep
the same notations for the sake of convenience. The reduced augmented matrix is shown below.
a,. a;2 a,n b,

0 a:22 a2.,, h2
[A,b)=
0 0 n2 ann bn
Use a12 as the pivot element to eliminate a 32 , a 41 , .• •, a. 2 from the respective rows using the following
row operations.
and soon.
[A,b]=
0 0
Using backward-substitution, we get

b,.
x. = -
a..
b, - a,nx,J - al.(n-l)xn- 1 · ··- a1-1X4 - al3x3 - a,2X2

x, =
a,,
Example3.1
l
Consider the system of linear equations in (3.8). The augmented matrix for this system is
2 -4 5 36
[A,b]= - 3 5 7 7 (3.15)
[ 5 3 - 8 - 31
Perform the following row operations on [ A, b] .
The resulting augmented matrix after the above row operations is
2 -4
[A,b]= 0 - 1 14.5
[
5 36
61
0 13 - 20.5 - 12 1
l (3. 16)
In essence, after the above row operations, the linear system in (3.8) is now reduced to the system
shown in (3.17)
2x1 - 4x 2 + 5x3 = 36
- x~ + l4.5x3 =61 (3.17)
l3x2 - 20.5x3 = - 12 1
Perform the following row operation on the augmented matrix in (3. 16).
R; ➔ R 3 + 1 3R 2
CHAPTER 3: Solutions of Linear Systems of Equations <&>
l
The resulting augmented matrix after the above row operation is
[A,b)=[ ~~
0 0
14.~ 36
61
168 672
(3.18)
and the linear system is reduced to

2x, - 4x~ + 5x3 = 36
- x 2 + 14.5x3 = 61 (3. I9)
168x3 = 672
Using backward-substitution, we get x 3 = 4, x 2 = - 3 and x, = 2. •

3.4.1 Pivoting
So far, we did not have to perform interchange of two rows. Interchan,g e of two rows becomes neces-
sary when the pivot element is zero. In addition, interchange of rows is belpfuJ in reducing round-off
error when the pivot element is small compared to the corresponding elements in other rows.
Let us consider the multiplier that originates during a row operation in reducing the augmented
matrix.
R'
"'
➔ R,. +(-a,.i,)R
a
l:t
t
(3.20)
The multiplier is
If the pivot e lement au « a,,.., the multiplier M,.. becomes very large and, therefore. the round-off
errors in the elements of the kth row get magnified. To prevent this, before a row operation, the row
(column) with the current pivot position and all rows below it (columns on the right-hand side of it)
can be searched to select the element with the largest absolute value and necessary row (column)
ex.changes can be done to bring the element with rl1e largest absolute value in the pivot position, This
is generally termed as partial pivoting.
Example3.2
Consider the following system of linear equations shown in (3.2 1). We will first solve the system
using Gauss elimination with no pivoting. We will use four-digit decimal floating point arithmetic
with rounding.
0.02x1 - 6x2 + 3x3 = 24.306
Sx, + 4.56x1 + 0.563x3 = - 7.018 (3.21)
0.367.r, + 4.23x2 + l3x3 = 35.881
We write the augmented matrix as
 CHAPTER 3: Solutions of Linear Systems of Equations
(A, b]=
0.02 -6 3 24.306
5 4.56 0.563 - 7.018
[ 0.367 4.23 13 35.881
l
l
The reduced augmented matrix becomes
0.02 -6 3 24.306
[A,b)= 0 1505 - 749.4 -6084
[ 0 0 14.9 52.28
Using backward-substitution, the solution is x3 =3.509, x 2 =-2.295 and x, =0.45. The exact solu-
tion is x 3 =3.5, x 2 = - 2.3 and x, = 0.3. Note that the solutions for x 1 and x2 are close to the exact
solution. However, the solution for Xi is 1.5 times the exact solution and clearly UJ1acceptable for any
practical use. The rounding error has impacted the solutions of both x3 and x 2 • But the rounding error
became even further magnified when we solved for x 1 due to a relatively small pivot element 0.02.
Let us now apply partial pivoting in reducing the system (3.2 1). We will again use four-digit decimal
floating point arithmetic with roUJ1d ing.
Scan the elements in U1e first column of the augmented matrix. The element with the largest absolute
vaJue is in the second row. Let us, therefore, interchange rows I and 2.
The augmented matrix after the row interchange is shown below.
[ A, b) =
5 4.56 0.563 - 7.018
0.02 -6
[ 0.367 4.23
3 24.306
J3 35.881
l
l
Eliminate the coefficient of Xi in rows 2 and 3 using a 11 = 5 as the pivot elemeut. The reduced aug-
mented matrix becomes
5 4.56 0.563 - 7.018

[A,b)= 0 -6.018 2.998 24.33
[ 0 3.895 12.96 36.4
Examining the elements in the second colwnn, from row 2 to 3, we find that row 2 has U1e element
with the largest magnitude in the second colwnn position. Therefore, no row interchange is neces-
sary. We use a 22 =-6.018 as the pivot element to eliminate U1e coefficient of x 2 in row 3.
After elinlinating the coefficient of x, in rows 2 and 3 and further el.inninating the coefficient of x 2 in
row 3. the reduced augmented matrix becomes as following.
5 4.56 0.563 - 7.018

[A,b)= 0 -6.018 2.998 24.33
[ 0 0 14.9 52.15
l
Using backward-substitution, the solution is x 3 = 3.5, x2 =-2.299 and x, = 0.299 which is close to
U1e exact solution.
CHAPTER 3: Solutions of Linear Systems of Equations <8)
Let us now examine the impact of using fixed number of digits after decimal on Gauss elimination.
Consider the following system of linear equations. We will use three digits after decimal
0.2x, + 670.245x2 + 3x3 = -4211.984

58x, + 4.56x2 + 0.563x3 = - 9.358
0.367x, + 4.23x2 + I 30x3 = 428.461
The augmented matrix is shown in
[A,b) = 58
[ 0.367
0.2 670.245
4.56
4.23
3 -4211.984
0.563
I 30
- 9.358
428.46 I
l
The reduced augmented matrix with no pivoting is shown below.
[A,b)=
0.2
[ 0
670.245 3
0 - 194366.49 - 869.437
0 129.712
-4211.984
1221466.002
828.656
l
Using bachvard-substitution, x3 =6.388, x 2 =-6.313 and x, =0.543. The exact solution is x 3 =3.5,
~ = - 6.3andx1 =0.3.
Now, use partial pivoting. Scan the first column of the augmented mattrix. The elemem with the larg-
[ A, b] = 0.2
[ 0.367
4.56 0 .563
670.245
4.23
- 9.358
3 -421 l. 984
130 428.461
l
est absolute value is in row 2. Therefore, interchange rows l and 2. The resulting augmented matrix is
58
[ A, b] =
[ 0
0
4.56
670.231
4.203
0.563 - 9.358
2.998 -4211.956
129.997 428.517
l
Eliminate the coefficient of x, in rows 2 and 3. The reduced augmenued matrix becomes
58
Examining the elements of the second column, from row 2 to row 3 , we notice that row 2 has the
element with the largest magnitude. Therefore, no row interchange is necessary. We now eliminate
the coefficient of x 2 in row 3 using ,a22 as the pivot clement.
l
After eliminating the coefficient of x , in rows 2 and 3 and further eliminating the coefficient of x 2 in
row 3, the reduced augmented matrix becomes as following.
58 4.56 0.563 - 9.358

[ A, b) = 0 670.23 I 2. 998 -421 1.956
[ 0 0 129.979 453.789
 CHAPTER 3: Solutions of Linear Systems of Equations
Using backward-substimtion, lhe solution is x 3 = 3.49 1, x 2 =-6.3 and x , = 0.3. The solution obtained
using partial pivoting is close to the exact solution of x 3 = 3.5, x 2 = -6.3 and x, = 0.3. ■
Both rows and columns can be searched for the element with the largest absolute value and necessary
row and column interchanges can be done to bring that element to the pivot position. This approach
is called complete pivoting. It should be noted that although neither row interchanges nor column
interchanges change the solution to the linear system, column interchanges do change the placement
of the solution to the unknowns. For example, if you interchange -columns I and 2 in the linear
system shown in (3.2), the order of the solution for the unknowns will be - 2, I and 3 instead of I ,
- 2 and 3.
3.5 Gauss-Jordan Elimination

Gauss-Jordan elimination can be considered a rwo-stage process. First, we apply Gauss elimination
as discussed in Section 3.4 to reduce a linear system of equations to a triangular form. Then in the
second stage, we further reduce the triangular form of equations to a diagonal form by eliminating
off-diagonal elements of the coefficient matrix segment of the augmented matrix starting from the
bottom and working our way to the top.
Let us start the second stage from the reduced augmented matrix shown in (3. I8).
Perform the row operation R; ➔ - 1- R3 on the augmented matrix.

168
The system reduces to
2 -4 536]
[A,b]= 0 - 1 14.5 61 (3.22)
[ 0 0 I 4
Perform the following row operation on (3.22).
R; ➔ R, +(- 5)R3
l
2 -4 0 16
[A,h]• [ 0 -1 0 3 (3.23)
0 0 I 4
Now, perfonn the following row operation.
R; ➔ R, + (-4) ~
l
2 0 0 4
[A, b]•[ 0 - ) 0 3 (3.24)
0 0 I 4
The off-diagonal elements of the coefficient matrix segment of the augmented matrix become zero.
Frnm (3.24), the reduced equivalent linear system of equations can be thought of as following.
2x1 =4
- X2 =3 (3.25)
X3 =4
The unknowns can be solved directly from the reduced equations as .x1 = 2. x2 = - 3 and x 3 = 4.
3.6 Solution by Matrix Inversion

Let us first discuss matrix inversion by Gauss elimination.
If A is a square nonsingular matrix then its inverse A- 1 exists and that
AA-' = I (3.26)
where I is an identity matrix of same dimension of A.

A square matrix is nonsingular if its determinant is nonzero. This is also related to the fact that a
linear system is independent if its determinant is no11Zero.
The inverse of a square nonsingular matrix can be determined by Gauss elimination in the fol lowing
manner.
Assume A is a 4 x 4 nonsingular matrix as shown below.
all a,2 0 13 a,.

a2, a-n On 01,1
A=
ll31 ~2 ~ 3 0 14
a., G 41 0 43 a,.
A has its inverse A_, that can be written as
8 11 812 813 8,.

A-I= 8 11 811 813 82•
8 31 8n 8 33 8:w
8•1 8.2 8 43 g.,.
To find A- i , we have to find its 16 elements.
We may, therefore. write
QII all a,3 a,. g,, 812 813 8,. I 0 0 0

Q"ll G12 ~ 3 024 8 21 822 821 8u 0 l 0 0
a3, a .12 ll33 a,. 831 8n 833 g'.1,1 = 0 0 1 0
a., a,2 ll43 a.. g., 8,2 80 g.. 0 0 0
Performing the matrix multiplication and equating the corresponding elements. we can write the
folJowing systems of linear equations.
0 11811 +a,181, +au83, +o,.8., = 1

0 2,811 + 0 1182, +a238J1 +c11,8.. = 0
(3.27)
G31811 +On811+0338J1+tl.14841 =0
a.,8,, +a.282, +a438J1 +a,.8., = 0
a118,2 +0 ,2822 +a,Jgn +a1•8•2 =0

a2 ,8,2 +022822 +an832 +a24g.2 = I
(3.28)
a31812 +On822 +tL33g32 +o'.!48•1 =0
0 .,812 + 0 ,2822 +a.13832 + 0 .. 8.2 =0
a,,8u +a,2823 +a13g33 + 0 ,. 8.., =0

a2 ,813 +Cln823 +t2iJg33 +a2•8•3 =0
(3.29)
0 31813+ 0 n823+ ~3833 + 0'.348•3 = 1
o.,g,3+a428 23 +a43g 33 +a44 g 43 =0
a11g,. +a1282, +a13g34 +a,.g.., =0

a2,8,. +a22824 +11z1g34 +a2•8•• =0
(3.30)
a3,g,. +a32g24 +<'-n8J• +~g.. =0
a.,g,4 +a42 g24 +a43 g3' +o,.g,. = 1
We have four systems of linear equations. Each system has four unknowns. Ald1ough die tmknowns
are different, the coefficient matrices of the four systems are the same. We can dierefore apply die
same row operations in doing Gauss elimination to reduce die four systems into the same triangular
form. Only the right-hand constant lerms are diffcrenL We can write an extended augmented matrix
as shown in {3.31) to apply Gauss elimination.
a" a,2 au a,-11 I 0 0 0

0 21 a22 023 a24 0 1 0 0
(3.31)
a1, 032 033 G:34 0 0 1 0
a., a 42 043 a., 0 0 0 l
After the reduced triangular form oif die extended augmented matrix , we can determine die elements
of the inverse in the following manner.
(3.32)
1> _ a.l5 - a34 g41

03J -
(3.33)
G33
CHAPTER 3: Solutions of Linear Systems of Equations <S>
(3.34)
(3.35)
(3.36)
C.u (3.37)
(3.3 8)
(3.39)
0 .1
g.3 = - (3.40)
0 .,
0-n - o3,g,3
(3.41}
G33
a21 - ang33 - a.1,Co (3.42)
(3.43)
(3.44)
(3.45)
a:u
au - 0 23834 - ~g.. (3.46)
{122
g = a,s - a.1182, - a,3g34 - a,,g.,. (3.47)

" G ,a
Cons ider the following system of linear equations.

a 11 x , +o, 2x 2 +a, 3x 3 +a.. x, =b1
a 21 x 1 +o12 x 2 +o23 x 3 +a 24 x4 =b1
(3.48)
a.3,x, +a12X1 +<7:i3X3 +~x, =b3
a, 1x 1 + a42 x2 + o43 x 3 + a.,.x, = b,
As discussed before, the system of linear equations shown in (3.48) can be written as
Ax=b (3.49)
<e> CHAPTER 3: Solutions of Linear Systems of Equations
where A is a 4x4 matrix, x is a 4-dimensional column vector and b is a 4-dimensional column

vector.
au an a,3 a,.
a2, G22 ~ a24
A= (3.50)
G31 1½2 Ct33 G34
a.. a,2 a,u a.,,
x,
X2
x= (3.51)
X3
x,
and
b,
bl
b= (3.52)
b3
b,
The unknown column vector x can be determined as

x =A- 1 b
A unique solution exists provided A_, exists.
Example 3.3
Consider the system of linear equations shown in (3.53).
2x, - 3x2 + 5x3 + 2x. = 3

x, + 4x2 - 2x3 + x, = 5
(3.53)
- 3.r, + x 2 +2x3 + 3x4 = - 2
4x1 - 2x 2 - x 3 - x4 = 9
For this system the square coefficient matrix is
2 -3 5 2
1 4 -2 1
A= _,
-3 l 2 ~
4 - 2 - .I - 1
the constant term column vector is
3
5
b=
-2
9
The extended augmented matrix is
2 -3 5 2 1 0 0 0
1 4 -2 1 0 I 0 0
-3 I 2 3 0 0 I 0
4 -2 - I -1 0 0 0
Do the following three row operations to eliminate the first element from rows 2 to 4.
R; ➔ R+(-½)R,
2
R; ➔ R +(-~ )R,
3
R~ ➔ R. +(-1)R,
We will use 4 digits after decimal for our calculations. The extended augmented matrix becomes
2 -3 5 2 L O O 0
0 5.5 -4.5 0 ~.5 1 0 0
0 - 3.5 9.5 6 1.5 0 I 0
0 4 - 11 - 5 -2 0 0 I
Do the following two row operations to eliminate the second element from rows 3 and 4.
The extended augmented matrix bocomes
2 -3 s 2 I 0 0 0
0 5.5 -4.S 0 ~.5 I O 0
0 0 6.6364 6 1.1818 0.6364 1 0
0 0 - 7.7273 -5 - 1.6364 ~-7273 0
<e) CHAPTER 3: Solutions of Linear Systems of Equations
Do the following row operation to eliminate the third element from row 4.
R' ➔ R
• •
+( - 7.7273)R
6.6364 3
The extended augmented matrix bocomes
2 -3 5 2 I 0 0 0
0 5.5 --4.5 0 -0.5 I 0 0
0 0 6.6364 6 1.1 818 0.6364 I 0
0 0 0 1.9863 - 0.2603 0.0137 1.1644 1
Using the expressions from (3.32) to (3.47) we can find the elements of the inverse of the coefficient
matrix.
0.1172 0.1517 -0.1034 0.0759

A-'= 0.1517 0.2552 -0.3 I 03 -0.3724
0.2966 0.0897 -0.3793 -0.4552
-0.1310 0.0069 0.5862 0.5034
The unknown column vector is

x=A-'b
0.1172 0.1517 -0.1034 0.0759 3

0.1517 0.2552 -0.3103 -0.3724 5
x= X
0.2966 0.0897 -0.3793 - 0.4552 -2
-0. 1310 0.0069 0.5862 0.5034 9
2
- )
x=
-2
3
•
3.7 Solution by Cramer's Rule
Consider the following set of linear equations.
a 11 x1 +a 12x2 +a0 x3 = b1
a11 x1 + a11 x1 +a2lx3 = b1
0 J1xt + 0 J2x1 + 0 u·':., = bl
CHAPTER 3: Solutions of Linear Systems of Equations <§>
The system of equations above can be written in matrix form as:
Ax= B
l
where
a., 12 3
A= a,, aa.22 a
a~
[ a,, a32 a 31
*
··[ :l~ •·[:l
If D 0, then the system has a unique solution as shown below.
D, X _D
_ 1 t _D
_ 3
,x,= - , 1- and + 3-
D D D
where
a., a,2 au b, a., a,,

D= 0 21 an a" DI = b, 0 22
a"
a,, au a ll IJ, an a,,
a,, b, a,, a,, all b,

D2 = 0.21 b, all D = a2, all b,
all b_, a ~J
' a}I 0 12 b,
Example3.4
Let us consider the following set of linear equations.
2x1 -4x2 +5x3 =36

- 3x 1 +5x2 +7x3 =7
Sx1 +3x2 - 8x3 = - 31
The set of linear equations can be written in matrix form as

Ax= B
<e::> CHAPTER 3: Solutions of Linear Systems of Equations
where
A·[-l : j l
· [ : lMd •·[_:;]
2 -4 5
D= -3 5 7 =-336
5 3 -8
36 -4 5
D, = 7 5 7 = - 672
- 31 3 -8
2 36 5
D, = -3 7 7 = 1008
5 - 3 1 -8
2 -4 36
D, = -3 5 7 = -1 344
5 3 - 31
D, -fJ72
r =-=--= 2
.' D - 336
D, !008
r2 = - = - - = -3
• D - 336
D, -1344
r =-=--= 4
.' D - 336 •
3.8 Solution by Matrix Factorization

In some cases, systems of linear equations can be solved by factorizing the coefficient matrices
into triangular forms. The unknowns then can be solved by relatively simple steps of backward and
forward substitution. T he number of arithmetic operations in such cases, specially for higher order
systems, could be significantly less than solutions by Gauss elimination and matrix inversion.
CHAPTER 3: Solutions of Linear Systems of Equations <e)
3.8.1 Triangular matrices
Triangular matrices provides for relatively simple steps in solving systems of linear equations.
A square matrix L is lower triangular if Ly = 0 for j > i. A square matrix U is upper triangular if
L_. = 0 for j < i.
Some properties of triangular matrices are:
a triangular matrix with nonzero diagonal elements is nonsingular,
inverse of a lower triangular matrix is lower triangular and
inverse of an upper triangular matrix is upper triangular.
3.8.2 LU factorization
Assume A can be factorized as A= LU. A system of linear equations tben can be expressed as
LUx= b (3.54)
Equation (3.54) can be solved in two stages as following.

1. Solve for y from Ly = b usi:ng forward-substitution.
2. Then solve for x from Ux =y using backward-substitution.
But first, let us find L and U. Assume
[,, L,, 0 0 u ,, u ,2
l
a l2 au ljll
A= '½1
a ll
a-n tin
a 32 a 33
} L•[ ½, ½1 ½2
½2 ½3
lmd U•[
0 0
0
U22
0
v~,
U 33
Therefore. we can write
]·[ ~
. ][ ~ 0 0
l
011 a,2 a,J U,2 U13
[ Oi,
a3, a.,.1
a"ll 0 23
0 33 L 31
L22
L32
0
L 33
U21 U23
0 U33
[,, Oi, Oi2

0 31 032
a,2 al 3
0 23
0 33
l[ =
L,,u,,
Li, V11
L 3,U11
L, 1U,2
L11U,1 + LnV22
L 31U ,2 + L32U22
½1Un +lnUn
L,,Ul.l
L3,U,3 + L32U 23 + L 33U J1

l
Equating the elements of tbe first row, we get
L, ,U 11 = 011,
L 11U 12 =a 12 and
L11U 13 = a 13 .
<e:> CHAPTER 3: Solutions of Linear Systems of Equations
Equating the elements of tbe second row, we get
Equating the elements of tbe third row, we get
In the above, we bave 9 equations but 12 unknowns. We can solve for 9 unknowns if we can assign
some convenient values to 3 elements. In Crout's method, the diagonal elements of the upper trian-
gular matrix U are set to land in Dolinle's method, the diagonal elements of the lower triangular
matrix L are set to 1.
Crout's method:
U" =! , U22 =I , 1/33 =l

LII -- a II ' L21 -- a2P L31 -- o3 1' U12 --L
an
II
Ln = an - L21U12, L31 = on - L3,U,2
Do little's method:
L11=I. ½2 =I, £-i3 =l

a.,,
U" = a U =a, V =a,.,. L =u·
11 , 12 2, 13 2,
II
U22 = 0 22 - Li,U,1, V n = On - £i,U,3

0 32 - L3,V12
U22
Example3.5
Use LU factors to solve tbe system of linear equations shown in (3.55).
2x1 + x 2 + 3x3 = 13
X 1 + X2 - 2X3 =7 (3.55)
3x1 - 2x2 +4x3 = - 5
Solution
The system of equation can be written as
Ax=b
where
A=LU
Let us first solve the system using Crout's method.
U11 =I, U 21 =I, 1/33 =I
L., =2, ½, =1, ~, =3, U, 2 =0.5

L 22 =0.5, ~ = - 3.5
Therefore,
0
0.5 ~
- 3.5 - 25
] and U = [ 0
0
1 0.5
] -7
0
1.5 l
LUx= b
Ly= b
wlnere Ux=y.
Using forward-substiturion,
u:l·l .:n
<e:> CHAPTER 3: Solutions of Linear Systems of Equations
Now, using Ux = y
Using backward-substitution,
Now solve the system using Dolittle's method.
L 11 = L, L22 = I, ½ l =I
ull =2, U,2 =I, Uu =3, L2, =0.5

U 22 =0.5, U 23 = - 3.5
½, =1.5, L32 = - 7
l
Therefore,
0 0
L=[ O.~ 1 0 0.5I - 3.53
1.5 - 7 L 0 - 25
LUx=b
Ly=b
where Ux = y.
Using forward-substitution,
Now, using Ux = y
u,-[
2 I 3
0 0.5 - 3.5
0 0 - 25
Xi
l[ J·[
X2
X3 in
•
3.8.3 Elementary matrices and LU factorization
EJ.ementary matrices can be utilized to perform elementary row operations (ERO) on a matrix.
Defi11itio11: Any matrix obtained by performing a single ERO on the identity mairix is called an
elementary malrix.
Corresponding to the three ERO, th.ere are three types of elementary matrices.
Type I: PIJ - permute rows i and j in /,..
Type 2: M 1 ( k) - multiply row i of/,. by the nonzero scalar k.
Type 3: Ag (k ) - add k times row i of I. to row J of I,,.
For the 3 x 3 identity matrix, we can write
P, 2 =
[
0
I
0 0
l
0
n-,.-u ~ !J~.·[ ; : il
M,(k)•[ 0k O
1 O
0 0 I
0 l [ : ;J i : ~ l
, M 2 (k)= 0I
0
M,(k)•[
A,,(k) •[ k l
I O 0
0
0 0 1
lA,,(k)•[ i i :n
0 0
I
k
0
I
Pr,e-multiplying a matrix A by an elementary matrix E has the effect of performing the corresponding
ERO on A.
Talce an example.
Permute rows 1 and 2 in A, the matrix becomes
We achieve lhe same result if we pre-multiply A by P12•
The most common ERO to reduce a coefficient matrix is adding k times row i of A to row j of A .
Let us multiply row I of A by -0.5 and add that to row 2 of A. The Iiesulting matrix will become
[ ~ -~3.: ]·
-) 3 4
The same result can be achieved if we pre-multiply A by A12 (-0.5).
Silllce any ERO is reversible, its corresponding elementary matrix is non-singular and, therefore,
invertible. The inverse of the lhree 3 x 3 elementary matrices can be obtained in the following manner.
l O O
0 I 0
0 0 1
l
CHAPTER 3: Solutions of Linear Systems of Equations <8>
Equating the corresponding coefficients,
P11 =0, P12 = I, P13 =0
Therefore.
P,,-' • [ ! I 0
0 0
0 I
l
P,t =P,2
Pn ?.i/ = l 3
[
I O O ][ P11
1
0 l 0
0 0 P21
P3o
P,1 P11 ] - [ I O O
Pri P2J - 0 I 0
Pn p 33 0 0 I
l
[~:~ ~ ]·U!n
P21 = 0, P12 = 0, Pn = I
JJ31 = 0, Pn = l , p 33 = 0
l
Therefore,
0 0ll
0
l 0
<fJ> CHAPTER 3: Solutions of Linear Systems of Equations
Similarly,
Therefore,
0 0
M,(kf' =
M, (kf'
[ 'i'~
=M, (1/k)
I 0
0 l
l
M2 (k)M 2 (kf' =15
CHAPTER 3: Solutions of Linear Systems of Equations <8)
m11 =l, m 12 =0, m 13 =0
"'21 =0, m22 =1/k, m23 =0

11131 = 0, 11132 = 0, 11133 = l
Therefore,
l
0 0
M,(kf' =[: ll/k 0
0
1
M 1 (kf =M 2 (1/ k)
Similarly,
M, (kr' =[ i 0
]
0 1/ k
0
0
l
M3 (kf' =M3 (1/ k)
1
A12 (k)A12 (kf =13
(: ! ~ l[ ~: :: :: ]=(i ! ~ l
~ ]=l : ! ~ l
[ .. · :: ,~,+~ ,~. +~

a 11 = I, a 11 =0, a 13 =0
<$> CHAPTER 3: Solutions of Linear Systems of Equations
Therefore,
~.1,r' ·(-i !n
A12 (kf' = A12 (-k)
A23 (k)A23 (kf1 =1 3
[:: ~l[~: ~ ~H:: ;J

[ h; • • ~: kan .:: kan .~ ]·[ : : :
Equating the corresponding coefficient~,

l
Therefore,
Aul'r' •[ i _;:]
An{kf' =A13(- k)
Similarly,
For a generalized II x II elementary matrices,
Alil ERO can be performed on a matrix by pre-multiplying the matrix by a corresponding elementary
matrix. Therefore, all EROs that are performed in Gaussian elimination to reduce the coefficient
matrix to its row echelon fonn (REF) can be achieved by pre-multiplying A by a sequence of
elementary matrices.
where R denotes an REF of A.
Since the unique reduced row echelon fonn (RREF) of a non-singular matrix is the identity matrix,
E,£1E3· ··Ek A= 1.
(3.56)
A-I =E,E1E3·-- Ekl .
Equation (3.56) shows that A- 1 can be obtained by applying the same sequence of ERO to I . that
reduces the non-singular A to its unique RREF.
LU Factorization
EJ.emcntary matrices can be utilized to reduce a non-s.i ngular matrix to an upper triangular matrix
l
much the same way a REF of a matrix is obtained utilizing elementary matrices.
4 -2 3
Let us reduce A = 2 7 5 to an upper triangular form using elementary matrices.
[ -I 3 4
The given matrix can be reduced to an upper triangularfonn using the following sequence ofEROs.
7/23
19/4
l
➔ An( - 5/16) ➔ [ l
04 - 28 7/ 23
0 0 117/32
In this section. we limit our discussion to non-singular matrices that can be reduced to triangular form
withom needing any row permutation. For such cases, a matrix can be reduced to upper triangular form
utilizing only Type 3 row operations. In addition, we restrict the Type 3 row operations in such a way
that we utilize ith row to obtain a zero in a row beneath ith row. With this restriction. all elementary
matrices corresponding 10 the Type 3 row operations that we will use will be lower triangular matrices.
Matrix A, therefore, can be reduced to an upper triangular matrix in the following way.
E,E1E3···EkA=U
where £,, £ 1 , · ·· Ek are lower triangular Type 3 elementary matrices.

Since elementary matrices are invertible
A-E-' Ek- 1-• ... £- 1£ · 1u
- k ~ I
<S:> CHAPTER 3: Solutions of Linear Systems of Equations
The inverse of a lower triangular matrix is another lower triangular matrix, and the product of two
lower triangular matrices is also another lower triangular matrix. Therefore,
A=LU
wlnere L = E,_, £,_,-•··•Et£,-• is a lower triangular matrix.

Example3.6
Determine the LU factorization of the matrix A.
Solution
Using the previous results,
7/23
117/32
l
l
where
ol o0
- 5/16
-I 17/32
7/23 l
The inverse of the elementary matrices E,. E 2, £ 3 are
I 0
E,-• = A12 (1/2) = 1/ 2 l
[
0 0
E/ =A13 (- l/4)=[ ~
- 1/ 4
E/ = A13 (5/16)=
( I
0
0
I
0 5/16
n
nl l
0 l l 0 0 l 0 0
L = £,-•£;';;,-•= [ 1/ 2 1 0 1 0
)[ 0 I 0
0 0 -1/ 4 0 l 0 5/16
n
I 0
L=[ - 1/ 4 1/ 2
5/ 16
A=[ - 1/I/;4 5/ 16
0
1
nl ,l4 -2
0
0
8 7/ 2
0 117/ 32
•
By observing the lower triangular matrix, we notice that the elements beneath the leading diagonal
are. the multipliers by which a specific row was multipl.ied and the result was subtracted from row i
to obtain a zero at ij location in developing the upper triangular matrix. For this example,
m21 = 1/ 2, m3 1 = - 1/ 4, m 32 = 5/ 16.
We may, therefore, save these muluipliers when the upper triangular matrix is developed and then
obtain the lower triangular matrix by directly using the multipliers instead of mnltiplying the inverse
of the elementary matrices.
3.8.4 Choleski's factorization

Definition: A matrix A is positive definite if it is symmetric and if x' A" > 0 for every n-dimens ional
ve.ctor x ~ 0.
If the coefficient matrix A is positive definite, then A can be factorized as LL'. where L is a lower
triangular matrix.
Consider a 3 x 3 positive definite matrix A.
[,, ]-[~. 0 0 ½1
l
0 12 a, 3 L" L 3,
a 2l
~ I
{111
a 32
{113
an
- ½1 ½2
½1 4- ½J
0
][ 0
0
L21
0
½2
L 33
<£9 CHAPTER 3: Solutions of Linear Systems of Equations
Since A is symmeLric,
a3, =au
a32 = 0-n
L ll -- vuu
~a t L 21 -- -
"n , - 0 13
, "-'ll - -
L,. L,,
ll11 - Li,"31
L 22
We can write a generalized expression for the diagonal elements as
. Ja - 1
L }.} = }.}
1-,
k=I ( L J.k)
2
and the off-diagonal elements as
L.IJ = -LI ( aiJ - I,1-1L

k=I l .k
)
L } .k .
J.J
Example3.7
Solve the system of linear equations using Cholesk:i's factorization.
4x1 +2x2 + 3x3 =5
2x, +6x2 + x3 = 13
3x1 + x2 + 7x3 = - 2
Solution
The system can be arranged in mabrix fonn as
Ax=b
wlnere
A=[~3 !I ~iand b =( I~]

7 -2
The coefficient matrix is symmetric positive definite. Therefore, Choleski's factorization will work.
L11 =2, L 21 = l, 4 1 = 1.5
4 = 2.236, ½2 =-0.224
Jl
0
L=( '. 2.236
1.5 - 0.224 2
LLrx=b
Ly= b
where LTx = y.
,J ][ l·l ~
n
2 0 Y,
Ly=( I 2.236
1.5 -0.224
Y2
.V3
Using forward-substitution,
l
2.5
[
Y1
Yi
)'3
]=( 4.696
- 2.168
Now, using Lr x = y
L',=[
2 I
1,
0 2.236 -0.224
0 0 2.168
l[ ]=[
X1
X2
X3
2.5
4.696
- 2. 168 l
XI
[ ]=( _i]
"~
X3
•
3.8.S LDP factorization
A symmetric positive definite matrix can be factorized into the form LDLT, where L is a lower
triangular matrix with diagonal elements being I, D is a diagonal matr ix and LT is the transpose of L.
A=LDLT
<e) CHAPTER 3: Solutions of Linear Systems of Equations
[,, d11Li,
l
a,2 0 13 0 0 dll d11½1
a,, 0 23 ½, l 0 0 dn d2242
]·[
a "l.1
a ll 0 12 a ll ½1 4i ][ 0 0 d,1
Equating the elements,

d" = a .. , d" Li, =a,2, d" ½ 1= a11
The elements of the lower triangular matrix L and the diagonal matrix D are:
d 11 =a 11
L, = a, 2
_, i
l II
Solve Ax= b by using the following operations.

Ax= b can be expressed as LDLTx = b where A= LDLT.
LDLTx =b can be expressed as Ly= b where y =DLTx.
y = DLTx can be expressed as y = Dz where z = LTx.
The solution for x can be found by .solving the following.

Solve for y from Ly = b,
solve for z from Dz= y, and then
solve for x from LTx =z.
Example3.8
Solve the system of linear equations using LDLTfactorization.
4x1 + 2x2 + 3x3 =5
2x1 +6x2 + x3 =13
3x1 + x2 + 7x3 = - 2
Solutio11
The system can be arranged in matrix fonn as
Ax=b
where
A=[ ~3 ! ~ ) b=[
I 7
and I~ ]
-2
Note that A is a symmetric positive definite matrix.

Lu =I, 42=l, ½.1 =I
d II -- aIt -- 4 ' L,_J -_ -a12 -_ -2 -_ o.J~

, L.JI -_ -a13 -_ -3 -_ 07::
. J
du 4 du 4
-, ., 2 2
d33 =an - du½i- - d22 L 31 - =7 -4 x(0.75) - 5x(- 0.1) =4.7
LDLTx= b
l( i l·( ~n
0 0 4 0 0 0.5 0.75 X1
( 0.5I I 0
0.75 --0.J 1 ][ 0 5 0
0 0 4.7 0
l --0.l
l l[ X2
X3
Ly= b
l·( ~n
0 0 Y,
( 0.5I I 0 Y2
0.75 --0. J 1 ]( )'3
<e> CHAPTER 3: Solutions of Linear Systems of Equations
Using forward-substitution.
Dz=y
[:: Jl[ : ]·[ ~l l

[~ ]·( '; l
•
3.8.6 Solution by QR factorization
The QR factorization, also called QR decomposition, is a factorization of a matrix into an orthogonal
matrix and a triangular matrix. If A is a real square matrix then it cani be factored as A= QR. where
Q is an orthogonal matrix and R is an upper triangular matrix.
A real square matrix witb ortbonormal colwnns is called an orthogonal matrix. The columns (rows)
are mutually perpendicular witb respect to each other. An orthogonal matrix is invertible with the
property that
Therefore,
wlnere I is the identity matrix.

CHAPTER 3: Solutions of Linear Systems of Equations <e>
The product of two orthogonal maitrices is an orthogonal matrix aind the inverse of ain orthogonal
matrix is ain orthogonal matrix. The determinant of ain orthogonal matrix is either I or - 1. The
columns (rows) of ain orthogonal matrix are an orthonormal basis.
Moreover, if Ais an orthogonal matrix then the linear function f (x) = Ax
preserves inner products: (Ax)T (Ay) = xTATAy = xTy
IIAxll=((Ax)T (Ax)} 'ri =(xTx)'12 =llxll
1
preserves norms:
preserves distances: IIAx-Ayll=IIAII llx-yll=llx-ylI

preserves angles: L(Ax,Ay)= arccos((Axr (Ay))= arccos( xTy11)= L(x,y)
IIAx IIIAYII llxllllY
where x aind y are any vectors in R 0 •
Let us write A and Q in the foUowing manner.
A=[ a la2 la3 I··· la.] where
1 a,,a1 ,a3 .. • ,a,, arecolumnsofAand
0 Rn R23
0 0
0 0 0 0 R,,,,
Using the Gram-Schmidt procedure we obtain the following.

ii, =a,
iii =0i - (a/q,)q,

ii?
q1 = 1~:11
ii. =a. - (a/q,)q, - (a/q1)q1 - (a.1 q3)q3 ··•- (a/q._, )q,,_,

q
_ ii.
•- lki. ll
<B::> CHAPTER 3: Solutions of Linear Systems of Equations
For i = l, 2, 3, ... , 11
a,rq, a/q, 03
T
q, a Tq
• I
T T
0 a/q2 aJ q2 a. q2
0 0
0 0 0 0 a" Tqn
Example3.9
Consider the system of linear equarion,
2x1 - 3x 2 + 5x3 = - 3
+ 4x2 - 2x3 = 2
X1
- 3x1 + X 2 + 2x3 = - JI
The system can be arranged in matrix form as
Ax=b
where
A=[ - ~3 -:I -~
2
land b=[ -; ]
- 1l
QRx = b where A = QR
Q=
0.5345 -0.4645 0.706 1
0.2673 0.8855 0.3802
[ -0.8018 -0.0145 0.5974
l
R=
[
3.7417 - l.3363 0.5345
0
0
4.9208 -4.1224
0 3.9648
l
Qz = b where Rx = z
z =QTb since Q is orthogonal.

0.5345 -0.4645 0.7061
z= QTb = 0.2673 0.8855 0.3802
l
[ -0.8018 -0.0145 0.5974
7.7506
z= 33241
[ - 7.9296
l ll l
Now Rx =z
3.7417 - 1.3363
0.5345
0 4.9208 -4.1224 x,
X1 = 7.7506
3.3241
[ 0 0 3.9648 x - 7.9296
3
•
Problem Set 3.1
1. Solve the following sets of l.inear equations by Gaussian elimination, Gauss-Jordan elim.ina-
tion, Cramer's rule and mattix inversion.
a. 4x 1 - x2 + x 3 = 8 b. x, - x2 + 3x3 = 2
2x, +5x2 + 2x3 = 3 3x, - 3x2 + X3 =- I
x , +2x 2 +4x3 = 11 =3
1 l I
C. - X
1
+ - X, + - X3 =9 d. 2x, - 3x1 + x3 = 6
4 5 - 6
X1 + X1 - X3 = 4
1 I I
- x, +- x, + - x3 = 8 - X 1+ X2 - 3x3 =5
3 4 - 5
I
X1 + X 2 + 2X3 = 8
2
e. 2x, + 3x1 - x3 = 4 f. 2x, - x 2 + x'.l = - I
X 1 - 2x2 + X 3 = 6 x 1 + Xi + 3x3 = 0
x, - 12.xl + 5X3 = IO 3x, + 3x2 + 5x3 = 4
2. Solve tbe sets of linear equations in P. L by LU, Choleski, LDLTand QR factorization.

<8:> CHAPTER 3: Solutions of Linear Systems of Equations
3.9 Ill-Conditioned Systems

The solutions of some linear systems are more sensitive to round-off error than others. For some
linear systems a small change in one of the values of the coefficient matrix or the right-hand side
ve·c tor causes a large change in the solution vector.
Consider the following system of two linear equations in two unknowns.
400 - 201
[ - 800 401
][.x']=[
.X2
200]
- 200
The system can be solved by using previously covered methods and the solution is x 1 = - 100 and
X-i = - 200.
Now, let us make a slight change in one of the elements of the coefficient mate.ix. Change A11 from
400 to 40 I and see bow this small change affects the solution of the following.
401 - 201
[ - 800 401
][x•]=[
.X2
200]
- 200
This time the solution is x, = 40000 and x 2 = 79800.

With a modest change in one of the coefficients one would expect only a small change in the solu-
tion. However, in this case the change in solution is quite significant. It is obvious that in this case
the solution is very sensitive to the values of the coefficient matrix A.
When the solution is highly sensitive to the values of the coefficient matrix A or the right-hand side
constant vector b, the equations are called to be ill-conditioned. 111-conditioned systems pose par-
ticular problems where the coefficients or constants are estimated from experimental results or from
a mad1ematical model. Therefore, we cannot rely on the solutions coming out of an ill-conditioned
system. The problem is then how do we know when a system of linear equations is ill-conditioned.
To do that we have to first define infinity nonns of vec10r and 111a1rix.
Vector and Matrix Infinity Norms

A nonn of a vector is a measure of its length or magnitude. There are, however, several ways to
define a vector nom1. For the purpose of tllis discussion we will use. a computationally simple for-
mulation of a vector norm in the following manner.
(3. 57)
The notation max { } denotes the maximum element of the set. The fonnulation shown by Equation
(3.57) is also called the infi nity norm of the vector x. Note the following properties of infinity nonn.
llxll>0 for x >= 0

llxll=0 for x = 0 (3.58)
llaxll=l~ll·llxll
llx +YII~ llxll+I~II
CHAPTER 3: Solutions of Linear Systems of Equations <€>
The last property in (3.58) is called the rriangle inequality.
Now we need to consider lhc notion of a matrix norm. A matrix norm can be defined in terms of a
vector norm in the following manner.
IIAII ! ~q,x{fIAk/1} (3.59)
Note that the expression for IIAII involves summing the absolute values of elements in the rows of A .
Example 3.10
Consider the following Linear algebraic system.
We have llbll=max{3,7,ll}=l 1 and IIAll =max{l2.ll,16}=16.

The mattix norm satisfies all the properties of a vector norm and, in addition. a matrix norm has the
following important property.
(3.60)
Condition Number
Let us investigate first, how a small change in the b vector changes the solution vector. x is the solu-
tion of the original system and let x + l!.x is the solution when b changes from b to b + l!.b.
Then we can write
A(x+ I!. x) = b+ l!.b
or, Ax + AA x = b + Ab
But because Ax= b. it follows that Al!. x = l!.b.

i.e., I!. x = A-'!!.b
By using the relationship shown in Equation (3.60) we can write that
11A-'I!.~ 1~IIA-'11·I ~~ I
i.e., II!!. xii~IIA-'11·II!!.~ I (3.61)
Again using Equation (3.60) to the original system, Ax = b we can write that
IIAxll~IIAll·l~II
i.e., l~ll~IIAll·l~II
or, IIAII ·llxll ~ 11h11 (3.62)
Divide Equation (3.61) by Equation (3.62)
ll&xll <IIA-'ll·11&~1
lvtll·l~II- 11h11
. 1~ 11 lvtll·IIA-'ll·ll&bll
J.C. , ~ ~ 11~1
or ll&xll ~IIAll·l~-,11·1~~1
. 1~11 llbll
or ll&xll <K(A)- l~bll (3.63)
' llxll - llbll
where K (A) is called the condition number of the matrix:A and is defined as
K(A) ~ IIAll·IIA-'11 provided A is nonsinglltlar.

K (A) is a measure of the relative sensitivity of the solution to changes in the right-hand side vector b.
Equation (3 .63) gives the upper bou111d of the relative change
Now, let us investigate what happens if a small change is made in the coefficient matrix A. Consider
A is changed to A+ M and the solution changes from x to x + &x.
Then (A+M)(x+ &x)= b .
Ax =(A+M)(x+&x )
Ax= Ax+A&x+&Ax+M&x
(3.64)
Applying the properties of infinity norm to (3.64), we can write,

(3.65)
P~emultiply both s ides of Equation (3.6) by IIAII

IIAII ll&xll ~ IIAIIIV'--'II IIMII IKx+ &x)II
11.1.x11 ::;IIAII 1~-•11 IIMII
1Kx+6.x~I lvtll
116.xll <K(A) IIMII (3.66)
llx+ 6-xll- IIAII
When the condition number K{A) becomes large, the system is regarded as being ill-co11ditio11ed.
Matrices with condition nwnbers near I are said to be well-conditioned.
Example 3.11
In Example 3.10,
A =[ 400 - 201 ] and th e correspon ding .rnverse A-' =[ - l.002

_2 -0.502 ]
- 800 401 -)
Condition number K(A)i IIAll·IIA-'11=3603 •

Hilbert matrix, H. is a well-known example of an ill-conditioned matrix.
I I I
I - -
2 3 II
I I I l
- -
H"= 2 3 4 n+l
1 I
II II+ 1 11+2 211 - J
Ill-conditioned matrices including Hilbert matrices arise in applications involving least squares
techniques and in discrete solutions of some partial differential equations.
Scaling
Large condition nwnbers can also arise from equations that are in need of scaling. Consider the
following coefficient matrix which corresponds to one 'regular' equation and one ' large' equation.
A= [ I -1]
1000 1000
In this case the inverse of the matrix is:
A_, = [ 0.5 0.0005 ]

-0.5 0.0005
~ CHAPTER 3 : Solutions of Linear Systems of Equations
I AI=max{2,2000}= 2000 and IIA-'11=max{ 0.5005.0.5005}. The condition number is: K ( A)=
2000x0.5005 = 1001
Scaling (called Equilibration) can be used to reduce the condition number for a system that is poorly
scaled. If each row of A is scaled by its largest element, then the new A and its inverse become
A=[ : -: ] A-'=[~:~ ~:~ ]

The condition number of the scaled system is 2.
We have just mentioned that when the condition number K (A ) becomes large, the system is
regarded as being ill-conditioned. But, how large does K ( A) have to be before a system is regarded
as ill-co11ditio11ed? There is no clear threshold. However, to assess the effects of ill-conditioning,
a rule of thumb can be used. For a system with condition number K (A) , expect a loss of roughly
log, 0 K (A) decimal places in the accuracy of the solution. Therefore for the system with
400 - 201]
A= _ whose condi tion number K{A) is 3603 expect to lose 3 decimal places in
[ 800 401
accuracy.
IEEE standard double precision numbers have about 16 decimal digits of accuracy, so if a matrix bas
a condition number of 1010• you can expect only six digits to be accurate in the answer. An important
aspect of conditioning is that. in general, residuals R = Ax - b are reliable indicators of accuracy
only if the problem is well-conditioned.
Problem Set 3.2
l
1. Compute the condition number of the following matrices.
a.[!;!] b.[~ ~!]

7 8 9 3 4 5
c.
I
2
I
1
3 d.
0.1 0.2
0.3
0.4 0.5 0.6
[ 0.7 0.8 0.9
2 3 4
I I I
3 4 5
11 2 3
4 15 6
7 8 19
21 31
51 61
29 49
l
R TECH I
8 tlN
N-- -E-A ifsv~
Although it may appear that direct methods provide exact solutions, rounding and truncation errors
accumulate quickly due to finite digit arithmetic operations and the solutions obtained become far
from exact. Instead, iterative techniques could be used to obtain solutions that are not exact but
are very close to lhe exact solutions. Iterative techniques are relatively straight forward in tenns of
computation and its implementation.
In this chapter, we will first discuss two iterative techniques that can be applied to linear as well
as non-Linear systems of equations. An iterative technique to solve an n x n Linear system Ax= b
starts with an initial approximation x<0l to the solution x and generates sequence of vectors x(Kl that
converges 10 the solution, x.
4.1 Jacobi Iterative Technique

Consider the system of Linear equations.
a,,x, +a,2 x 2 +a,yx3 + ··· +a,. x. = b,
a~,x, +a22 x~ +a23 x 3 +···+a2. x. = b2
12:i,x, +"32x 2 +a.33x 3 +···+a3.x,. =b3 (4.1)
aIt ,x,+a ,x, +a . ,x3 +···+CI M x =bII

,i _ - ti .~ ff
~ Chapter 4: Iterative Techniques for Solving Linear and Non-Linear Systems
Convert the set Ax = b in the form of x = Tx +C. Each equation in (4.1) can be rewritten in tenns
of one of the unknowns in the following manner.
_ a, a 13
x 1 - - -2 x2 - -x -
a,,.
···- - X +-b,
3 11
a11 a11 ° 11 °11
ci., 1
X 2 = - --- .X 1
Gu
- -x
~. b,
-· · ·- -X. +--
3
a22 an °22 a22
x, -- -G-x
31 a ,2 a,,. b3
(4.2)
1 - - x 2 -· ·· - -x. +-
a1l an a11 a 3J
A generalized expression for the ith unknown , therefore, can be written as

~ a0 b,
1 +-
x1 = - ~ -x i= 1, 2,3, ... , 11 (4.3)
J=• au au
J,J
The kth update of the unknown vector x<•>can be obtained using x1•-•>
.
x/kl = - I, ayx/ 1-1)+!!I_ j= 1, 2, 3... . , II (4.4)
i-=-• au au
J,J
Example4.1
Consider the following set of linear equations.
12x1 - x 1 +2x3 +x, =9
- 2x1 + l lx 2 + x 3 - 3x, = 15
(4.5)
x, + x 2 +9x3 +x, =- 5
2X1 + x2 - 2X3 +8x4 = 16
Convert the set Ax = b in the fom1 of x = Tx + C.

1 l l 3
x1 = - x 2 -- x 3 - -x 4 +-
12 6 12 4
1 3 15
- -x+- 3
x +-
11 11 ' 11
--1 x s
9 4 9
+2
Chapter 4: Iterative Techniques for Solving Linear and Non-Linear Systems 
Start with an initial approx.imation of x<0l = (1 , I, I, 1]7. Then xl1l is given by
x,<o = _Ix (ol - ~ x co) - _Ix (o) + ~

2 3
12 6 12 • 4
,(l) _ ~X(O)
-''"2 - J
I (o) +-3 x (O)
- -x 3 4
+-JS
Jl II II II
l • (0) _ -LX (0)
,. (1) _ _ _
"'3 -9"• 92
I I I 3
- x(l) -- x(l) - -(J) + - =0.58333
12 6 12 4
2 I 3 IS
x/> Jl = - x(l) - - x(I)+ - x( I) + - = 1.72727
II LI II
I l
x 3() = -- x(I) - - x(l)
L I s
-- x(l) -- =--0.&8889
9 9 9 9
+ 2= 1.87500
After the first iteration, x(•) = [0.58333, 1.72727, --0.88889, 1.87500)7. With x<•>. we now proceed to
find xl2l in the following manner.
• (2) -
.Al -
_!_ x (ll _ _!,x (I) _ _
I x<•> +~
2 3
12 6 12 • 4
X4
12> _ I (1) E (1)
- - - X1 - - X 2 + -I x,(1) +2
4 8 4
(2) - I l I 3
XI -
- x (l.72727) -- x(--0.88889)- - (1.87500) + - = 0.88584
12 6 12 4
2 I 3 JS
x2 (2 l = - x (0.58333) - - x(--0.88889)+ - x (1.87500) + - = 2.06187
II II II 11
x}2> = - ¼x(0.5&333) - ¼ x(l.72727) - ¼x ( 1.87500) -%= - l.02062

I I I
x,c2l =-- x(0.58333) - - x(l .72727) + - x (--0.88889) +2 =1.41604
4 8 4
Similarly, xt2>, x l3l etc. are generated and presented in Table 4.1.
Table 4 .1 Iterates using Jacobi iterative technique
k X (kl X / kl X (kl X Ck )
I l 3 4
0 1.00000 l.00000 1.00000 J.00000

J 0.58333 l.72727 - 0.88889 1.87500
2 0.88584 2.06 187 - 1.02062 1.41604
3 0.97392 2.00367 - 1.04042 1.26565
4 0.98490 l.98047 - 1.02703 1.24596
5 0.98238 1.97588 - 1.02348 l.24946
6 0.98112 l.97606 - J.02308 1.25155
7 0.98089 l .97636 - J.02319 J.25194
8 0.98090 l.97644 - 1.02324 1.25193
9 0.98092 l.97644 - 1.02325 1.25191
The decision to stop may be based on one of the following:
Ilx1•> - x<•-•>11 _ < e

Ili'l - x<•-•>11 _
llxc•111_ <£
where e is a small positive number.

The solution is X = [0.98092, 1.97644, - 1.02325, 1.25 I90]7accurate within 10· 5• The method used
in this example is called Jacobi iterative rnclhod. The Jacobi iterative sequence a~ applied to our
ex.ample problem can be expressed in the following manner.
x, (•l = - x,
I (k- 1) I (k- 1) l (k- 1)
--x3 - -x4 +-3
12 - 6 12 4
I
X
2
(I) _
- -
2
] ]
X
I
(k- 1)
- -
(1- 1)
X3 +-3 .r4(l- 1) +-15
11 11 ll (4.6)
(k) _ ] (k- 1) ] (k-1)
x, - - - x, - - x,
· 9 9 ·
(k) - I (k- 1) l (k- 1) l (k- 1) +2
~ - --~ --~ +- ~
4 8 4
4.2 Gauss-Seidel Iterative Technique

In Jacobi iterative method, all elements of x<1l are calculated using ex.elusively the elements of x<•-•>.
However, a possible improvement in terms of the speed of convergence can be made if instead
of using all e lements of x<•·•I i.e., .r/•·•>,
xt· 'l, x/· 1f and x/· 11 we use x,<tl, x/1 and x}'1 as they
Chapter 4: Iterative Techniques for Solving Linear and Non-Linear Systems 
become available to find x<'1. In the example of Jacobi technique, x?) would be available right
after lhe first expression which then can be used in the 2nd , 3rd and 4 th expressions. Similarly, 1 x/
rd
would be available right after the second expression and then can be immediately used in the 3
and 4 th expression and so on. This modification is called the Gauss-Seidel iterative technique. The
Gauss-Seidel iterative seq uence as. applied to the example problem (4.6) can be expressed in the
following manner.
• (k) -
-'"I -
• (k1 _ 2_X (l)

--x
1 (l- 1)
+- x 4
3 (l-1)
+-J5
-'\,2 - I
11 11 3 11 ll
(4.7)
I (k- 1)
-- i
5
--
9· • 9
Again, with an initial approximation of x<01 = [I, I, l, Ir

we can generate iterates using the Gauss-
Seidel iterative technique. The iterates are shown in Table 4.2.
•• (I) -
-'"I - -1 x,(o) --x
l co1
3
l
- -x 4
(oJ
+-3
12 - 6 12 4
I to1 +-3 x (OJ +-15
- -x 3 4
11 11 11
--I x (ol 5
--
9 • 9
1 1 I 3
xi'1= x( IJ - x(J) - (1) + = 0.58333
4
12 6 12
x/>= 2.. x(0.58333) -.!.x(L) +2- x(I) +~=l.65 152
11 11 11 II
x/1=-¼x (0.58333) - ¼x(J.65l52) - ¼x (l) - %=--0.91498
x/> = -.!._ x (0.58333) -.!. x(1.65152) + .!.x(--0.91498) +2=1.41898

4 8 4
1 (1) 1 (1) 1 (1)3
, -
.....• (2) - - x 2 -- x 3 - - x 4 +-
12 6 12 4
L (I) 3 (I) 15
- -x +- x 4 +-
11 3 11 11
(2) - I I I 3
XI x(I .65I52) - x(-0.9I498)- (1.41898) + =0.92188
-
12 6 12 4
2 I 3 15
xt>= - x(0.92188) - -x (-0.91498)+- x (I.41898) +- =2.00143
11 II II II
xt> = - .!.x(0.92188) - .!.x(2.00143) - .!.x(I.41898) - ~=- 1.03803
9 9 9 9
x/> = - ¾x (0.92I88) - ½x(2.00I43)+¾x(- J.03803) + 2 = 1.25985
Table 4.2 Iterates using the Gauss-Seidel iterative technique
xt.t' X (k l X (k) X ( k)
k I 2 3 4
0 1.00000 1.00000 1.00000 1.00000

I 0.58333 1.65152 -0.91498 1.41898
2 0.92188 2.00143 - 1.03803 1.25985
3 0.98480 I.98065 - 1.02503 1.24996
4 0.98173 1.97622 - 1.02310 1.25177
5 0.98089 l .97638 - 1.02323 l .25192
6 0.98091 1.97644 - 1.02325 1.25190
7 0.98092 1.97644 - 1.02325 1 .25190
Taible 4.2 clearly shows the advantage of the Gauss-Seidel iterative technique over Jacobi iterative
technique.
4.3 Iterative Techniques and the Convergence of Linear Systems

Let us first define a diagonally dominant matrix. A square matrix is said to be diagonally dominant
if the magnitude of the diagonal element of every row is larger or equal to the sum of the magnitudes
of the off-diagonal elements in the corresponding row. For a matrix A that is diagonally dominant,
la1,I ~ I.laul for all i

f ,/
where a!J is an element of A in the fth row and jth column.

For a strictly diagonally dominant matrix,
Ia1,I> I.laul for all i

f ,/
A strictly diagonally dominant mairix is nonsingular. A symmetric diagonally dominant real matrix
with nonnegative diagonal entries is positive semidefinite.
Chapter 4: Iterative Techniques for Solving Linear and Non-Linear Systems 0
If the coefficient matrix A of a Linear system is strictly diagonally dominant, then for any choice of
x<0>both Jacobi and Gauss-Seidel it,erative techniques produce sequences { X (k)
the unique solution of Ax = b. •=<>
that converges to r
4.4 Iterative Techniques for Non-linear Systems
At this point we would like to utilize the two techniques to solve non-Linear systems of equations.
Consider the following system of non-Linear equations.
l5x 1 +x; - 4x3 =Il4
x~+ l 0x2 - x 3 = - ll (4.8)
3x, + x; - 25x, = - 19
We will first utilize an iterative technique similar to the Jacobi iterative technique (fixed-point
iteration for multiple urumowns) to solve the system. The first step is to rewrite the system in the
folJowing manner. There is no rule: as to which equation should be used for x, and which one for
Xi· The choice is made as a matter of convenience and to circumvent the situations where an equa-
tion may not have an unknown Linked to it.
(k) -
X1 - --
l ( .x,
. (k- 1) )2 + -4 X (k- 1) + -I 4
]5 - 15 3 15
2
(k) _ I ( (t- 1)) l . (t - 1) 11 (4.9)
X, - - - Xi + - .X3 --
- IO JO JO
X3<•>-_ - 3 X 1(t- 1) +- l ( X,<•-•> )3 +-l 9
25 25 - 25
We take an initial approximation of x<l = [ I, 1, l

0
f and the corresponding iterates are shown in
Table4.3.
Table 4.3 Iterates using fixed-point iterative technique
k X Ctl X CtJ X / kl
j 2 J
0 1.00000 1.00000 l.00000

l 1.13333 - 1.10000 0.92000
2 J.09800 - 1.13644 0.84276
3 1.07197 - L.13628 0.83305
4 J.06940 - 1.13161 0.82995
5 J.06928 - 1.13137 0.83037
6 1.06943 - 1.13130 0.83039
7 1.0694S - J.13L33 0.83042
8 1.06945 - 1.13133 0.83041
The solution is x = (!.06945, - L 13133, 0.83041f which is accurate within 10· 5 •

We will now solve the set of equations (4.8) using the Gauss-Seidel iterative technique.
(k) - l ( (k - 1) ) 2 4 X (! - 1) +-14
X1 - - - X, +-
15 - 15 3 J5
X
(I) -
- - -
l ( X. (I) )2 + - I X. (k- 1) - -11 (4.10)
2 IO I 10 3 10
x,(k) -- -3 x1<•> + - L ( x2(t) )3 +-I 9

. 25 2:.5 25
We take an initial approximation of x<0> = [ 1, I, 1 f and the corresponding iterates are shown in
Table4.4.
Table 4 .4 Iterates using the Gauss-Seidel iterative technique
k X (kJ X (l J X ( kJ
I 1 3
0 1.00000 1.00000 l.00000

1 1.13333 - 1. 12844 0.83852
2 1.07205 -1.13 108 0.83076
3 1.06958 -1.13 132 0.83043
4 1.06946 - 1. 13133 0.83041
5 1.06945 - l.13133 0.8304)
In general, the Gauss-Seidel iterative technique requires less number of iterations compared to the
Jacobi iterative technique.
4.5 Newton's Method

Consider the following set of non-linear equations with multiple number of unknowns.
f, ( x,. x~, ... , x,,) =0
J;(x,, x2 , ... ,x,,)=0
(4.11)
.f.(x,, x 2 , .••• x.)=0

Following the process that we used for single unknown, we can write a sequence for multiple
number of unknowns.
(4.12)
Chapter 4: Iterative Techn iques for Solving Linear and Non-Linear Systems <S>
where
x,(k- 1)
x. (k- 1)
and
• ( (k- 1) (k- 1) , (k-l))

} 11 .x. ,X1 ,·· · ,..t,.
cJf, aJ. cJf,

ax, dX2 ax.
a/2 cJJ; cJfi
J (X) = cJx, dx1 ax.
cJf. cJ.f. cJf,,

ax, axl ax.
The matrix J ( X) is called the Jacobian matrix.
Consider the following set of non-linear equations.
x,3 +x,x
2 ,c
1 - x,x3 =-u
e'' +e'' - x 3 =0 (4.13}

Xi - 2x1x 3 =4
Rewrite the equations to conform with the set of equations shown in (4.13).
x; +x:x1 - x,x3 +6=0
e'' + e'' - x 3 = 0 (4.14)
x: - zx,x 3
- 4 =0
Note that the right hand side of all equations in (4.14) is zero. The corresponding three functions are
shown in (4.15)
f, (x, ,x1,x3)= x; + x?x2- x,x3+ 6
/2 (x,, Xi, X3) = tr' + e·'' - X3 (4.15)
h (x,,x1 , x3 ) = x: - 2x,x3 - 4
<@> Chapter 4: Iterative Techniques for Solving Linear and Non-Linear Systems
F(X)= e·• +ex"' - x3

Xi - 2x,x3 - 4
,
[ ,,: + ,,,,, - x, x ·I - x,
l (X)= e''
- 2x1
e-'1
2x1
- I
- 2x1
l
Make an initial guess as:
4.00000 1.00000 - l.00000

J(X(O))= 2.7]828 2.71828 - l.00000
[ - 2.00000 2.00000 - 2.00000
l
J(X(O
)r' =
0. 20000 0.00000 -0.10000
-0.43279 0.58198 -0.07459
[ -0.63279 0.58198 -0.47459
l
I .00000
x<•I = 1.00000 -
[ 1.00000
l[ 0.20000 0.00000 -0.10000
-0.43279 0.58198 -0.07459
-0.63279 0.58198 -0.47459
l[ 7.00000
4.43656
- 5.00000
l
x<•>=
-0.90000
1.07459
[ 0.47459
l
Chapter 4: Iterative Techniques for Solving Linear and Non-Linear Systems ~
F( x(I>) =
6.56855
2.86078
[ - L.99098
l
1(x(I))=
0.02114 0.81000 0.90000
0.40657 2.9288 - L.00000
[ -0.94919 2. 149 19 1.80000
l
2.04925 0. 13152 -0.95156
1(xC•J r = 0.06002 0.24641 0.10688
[ J.00896 -0.22485 -0.07384
l
xl 2
>=
-0.90000
1.01459
[ 0.47459
l[ -
2.04925 0. 13 I 52 -0.95 156
0.06002 0.24641 -0.10688
I.00896 -0.22485 -0.07384
l[ 6.56855
2.86078
l
l
- I.99098
-16.63141
x<2> = 0. I 8822
[ - 5.65656
l
Similarly,
- 10.21778
x<3>= - 2.49499
[ - 2.0318
x<•) =
-7.80536
L.65635
[ 0.42509
l
X= -0. 13180
[ 1.01790
l
Continue until the stopping criterion is satisfied. After IO iterations, the unknowns are:
- 1.95630
accurate within 10-5 _
•
<0> Chapter 4: Iterative Techniques for Solving Linear and Non-Linear Systems
Problem Set 4.1

1. Solve the following sets of linear equations by Jacobi iterative technique and Gauss-Seidel
iterative technique.
a. 4x, - x 2 + x 3 =8 b. x, - X2 +3X3 = 2
2x, +5x 1 +2x3 = 3 3x1 - 3.x2 +x 3 = - l
x,+2x2 + 4x3 = 11 =3
I l I
c. - x, + - x,+ - x 3 = 9 d. 2x, - 3.x2 + X3 = 6
4 5 · 6
x 1 +x 2- x 3 = 4
I J I
- X 1+ -X, +- X = 8
3 4 - 5 3 - X 1 + X2 - 3x3 = 5
I
- x, +x 2 +2x3 = 8
2
e. 2x, + 3x2 - x 3 =4 f. 2x 1 - x 2 +x3 =- I

=6
X1 - 2X2 +x., X1 +X2 + 3x3 =0
x, - 12x2 +5X3 = 10 3x1 + 3x 2 + 5x3 = 4
2. Solve the following sets of non-linear equations by fixed-point iterative technique, the
Gauss-Seidel iterative technique and Newton's technique.
a. x~ - 3x1x 2+xJ + 4 = 0 b. x~ - 3x2 + x 3 = 9.5
2x1 + Xi - X1X 3 = 7 2x, +xJ - 5x 3 - 1.25= 0
x1x2 + x3' = 3 3x, +x 1 + Xi =5.5
c. x~ + 2x2 + x3 = 0 d. X1 Sin _.\Oz +x 3 + 2.994 = 0

2x1 +x ,x~ - 5x 3+ 10.5 = 0 5x 1.x2 + 3x 2 - Xi+ 23.657 = 0
3.x1 + 4x2 + x 2x3 +J 2 = 0 3x1 + X1 + X2X3 =5.76
In many applications, it becomes necessary to find mathematical expressions of functions based on
their values at certain discrete points. The values at certain discrete points may come from some
experimental procedures, sampling of analog signals or from some empirical relationships. The
question is then why do we need a mathematical expression of a function whose values at certain
points are already known to us? The answer is simple; we want to use the mathematical expression
to estimate its values at some intermediate points that were not given to us.
Interpolation produces a function that matches the given data points exactly. Tbe function should
provide a good approximation to the (unknown) data values at intermediate points.
Interpolation may also be used to produce a smooth graph of a function for which values are known
only at discrete points, either from measurements or calculations.
In many cases, polynomial expressions of appropriate degrees can represent functions that pass
Lhrough their respective data points. We will discuss two standard forms of polynomial, Lagrange
and Newton ·s, that can match given data points exactly.
In some cases, it is necessary to interpolate derivative values in addition to function values. Hermite
interpolation can be used to interpolate function as well as first-derivative values. Some functions
are not well approximated by polynomial expressions. Rational functions in these cases provide
bener approximations. [Note: A rational function is a ratio of two polynomials.]
Some of the common uses of interpolation are computer graphics and digital pictures.
5.1 Polynomial Interpolations

Given 11 + 1distinct points x 0 , x,. .. .• x. and their corresponding function values y0 • y,, ... , y,., there is
a unique polynomial P. (x) of degree II that interpolates to y1 at x, where i = 0, 1, . . . , n. This unique
<@>
<€,> Chapter 5: Interpolation
polynomial can be derived using techniques of Lagrange interpolating polynomial and Newton's
interpolating polynomial. Although tbesc two techniques develop the polynomial P. (x) differently,
tbey both result in tile same polynomial expression.
S.2 Lagrange Interpolating Polynomials

A function tbat passes through two distinct points, (x 0 , Yo) and ( x 1 , y 1 ) can be obtained tile same
way as approximating a function f with the help of a first-degree polynomial such that /(x0 )= y0
and /(x1 )= y1•
To accomplish the above, define tbe functions
and then define

P, (x) = L0 (x) Yo *Li (x)y, (5.1)
P, ( x) is an approximation of f.
X- X
1 X - Xo
P, (x) - - ~ yo +- - ~ y. (5.2)
x0 - x 1 x 1 - x0
X - X
Atx=x,, L,.0 (x,)= 1 1 Oand li.1(x1) x, - x., I
x0 - x 1 x, - xo
Therefore, P, (x,) = L,.o (x1 ) Yo+ Lu (x, )y, = 0. Yo + I. Y1 = Y,
So, P, ( x) is lhe unique linear function passing through (x0 , Yo) and (x,. y 1). Figure 5.1 shows the
actual function and its first-degree polynomial approximation.
It can be easily noticed from Figure 5.1 tl1at if P, (x) is an approximation oftbeactual function, J(x)
then we did not do a very good job of approximating. However, given only two data points this is the
best we could do. We need to utilize more data points for P, (x) to come closer to the actual function,
f {x). However, more data points do not always provide better approximations.
If we have three data points, (x0 , Yo), (x., y,) and (x2 , y2 ) . then the Lagrange interpolating polynomial
of degree two passing through tl1e three points can be expressed a~
Note that tllere are tbree tem1s on the right-hand side of (4.3) one for each data point, and the degree
of the polynomial is 2.
Chapter S: Interpolat ion ~
Yo =f(x,)
Xo X
Figure 5.1 First-degree polynomial approximation
The general fonn of the Lagrange i11terpolating polynomial passing through 11 +l data points (x0 , y0 ),
(.r1,y1), (x2 ,y2 ), •• • ,(x.,y.) has 11+1 terms on the right-hand side, one corresponding to each data
point. The degree of the polynomial is n.
(5.4)
All we need to do is to express L•.o (x). L,,., (x}, L,,.1 (x}, ... , L,,_,, (x) based on n+ I data points. We
need to construct L,,_t (x) in s uch a way that L,,., (x,) =0 when i 1' k and L,,_t ( x,) =I. This requires
that the numerator of L,,_, (x) contains !he tem1
(x- x0 ){r., - x){t-

l · ·:r2 )······{x - x.t- 1 )(x - x k+I )··-···(x - x)
r1 (5.5)
The tenn (5.5) vanishes at all data points except at x = xt. In order to satisfy L,,_, (x,) = I, the
de nominator of L. _1 ( x) should be equal to the tenn (4.5) when x = x t· Therefore,
(5.6)
The general fom1 of the Lagrange interpolating polynomial can be written as:
• •
P,, (x) =L,L..• (x)yt =L,L.., (x)f(x,} (5.7)
k=O
where
(5.8}
<$> Chapter 5: Interpolation
Let <l>(x) =(x- x0 )(x - x, )·· -(x - x.)
<l>'(x1 )=(xi - x0 )(xt - x,)• ··(xt - xt_, )(xt - xt••) ··•(x1 - x,,)
L (x)=I1 (x - x,) <l>(x)
•·• r-{J (xi - x,) (x - xt)'(xt)
I#
Therefore, the Lagrange interpolating polynomial can be written as

~ <l>(x)
~ (..:C _ Xk )<t>'( _.\'.t. )/(xi)
~(x)= k=O (5.9)
ExampleS.1
Find the Lagrange interpolating polynomial that approximates a function whose values at three data
points are given below.
(x0 ,y0 )=(2,0.7O71 I)
(x,, y,) = (2.5, 0.63246)

(x2 , y2 ) = (3, 0.57735)
Solutio11
,
P1 (x) =!, ½_t(x)J(x.)
= (2x 1 - llx +15)(0.70711)+(-4x 2 + 20x - 24)(0.63246)

+(2x 2 - 9x+ 10)(0.57735)
= 0.03908x 2 - 0.325 I 6x + 1.20 I 11

Chapter S: Interpolation ~
The three given points were taken from the function, f (x) = ~. The exact value of the function at
x=23is vX
I
f (2.3) = r,:;-,, = 0.65938
v2.3
Am approximation to f (2.3) can be obtained from the second degree Lagrange interpolating
polynomial as:
Pi (2.3) = 0.03908(2.3}2 - 0.32516(2.3) + 1.2011 1= 0.65998
with an absolute error of0.00059. •

5.2.1 Error bound of Lagrange interpolating polynomial
Lagrange intetpolating polynomial will be utilized extensively in this textbook. So, it is worthwhile
to find out U1e error every time we make when a function is approximated by a Lagrange interpolating
polynomial.
Theorem 5.1
Suppose .r0 , x,, ... , x. are distinct real numbers in [a., b]
and / a real-valued function such that
f E c••' [a, b]. Then for each x in [a, b] there exists a number~ (x) in (a, b) such that
f'"') (~(x))
f(x) - P.(x)=(x - x 0 )(x - x,)···(x - x.) (n+l)! (5.10)
where
P,, (x) is the Lagrange interpolating polynomial of degree 11. ~ (x) indicates U1at it depends on x.
Proof
From Equation (5. I0), f (x ) - P ( x) = 0 if x is one of the node points. If x is not a node point, say
x =t , then define
R(x)=J(x)- P.(x)andQ(x)=R(x) - <l>(x} R(t) forall.rin [a,b]

(t)
where
(x) = (x - x0 )(x - x,)•· ·(x - x,.)
The functions R(x), Q(x) and (x) arc 11 + I continuously differentiable in [a, b].
Q(x,) =R(x, )- (x,) R(t)=0 fori=0, l,· ··,11

<t>(t)
(r)
and Q(r)"" R(t)- -R(r)ee R(t) - R(t) ""O
(t)
<@> Chapter 5: Interpolation
Therefore, Q e c•• 1 [ a, b] and Q has n + 2 rustinct zeros in [a, b]. Using Generalized Reile's
Theorem. there exists a number ~ (x) in (a, b) for which Q(.+IJ ( ~ ( x)) = 0.
P,, (x) is a polynomial of degree at most 11. Therefore, (11 + I} th derivative of P,, (x) is zero.
P.t•'l (x) = 0 and, therefore, RC•••l (x) = t<•••l (x)- P,,t"•'l (x) = J<•••J (x).
<1><"•1>(x) = (n +I)!
<J>(n+I) (X)
Q (..+I) (x) = R(n+J) (x) <t>(t) R(t)
Q(n+')(~(x))=f•+'} (;(x)) (n+l)! R(t)=0

<l>(t)
Therefore,
ExampleS.2
In Example 5.1, f(x)=t/.Jx was approximated using the second degree Lagrange interpolating
polynomial. At x = 2.3, the Lagrange interpolating polynomial approximated f (2.3) with an absolute
error of 0.00059. Determine the maximum error using Equation (5. IO).
Solutio11
The error for the second degree Lagrange interpolating polynomial can be expressed as
. . J<2+1)(~(x)) . j<3l(~(x))
(,t - x0 )(x - .t,)(x - x2 ) - ~ ~ (x - 2)(x - 2.5)(.x - 3)- ~ ~
(2+1) 1 (3)!
Since j<'l (x) = - (l/2)x-312 , j<2>(x) =(3/4 )X-312 • f' 3>(x) =- (15/8)x-1l2,
the error is - (5/16)(~ (xr )(x - 2)(x - 2.5)(x - 3)
112
The maximum value of ~(xr112 in (2, 3) is 0.08839.

The maximum value of (x - 2)(x - 2.5)(x - 3) is 0.04832
The maximum error is
3
( x - x )(x - x )(x - x ) / l(~(x)) ~l(x - x )(x - x )(x - x )I fll(~(x)) =0.00133
0 I 2 ( 3 )! 0 I 2 (3)!
■
Chapter S: Inte rpolat ion ~
Problem Set 5.1

1. Use the second and the third degree Lagrange interpolating polynomials to approximate each
of the following:
a. /(1.4) if /(1.1)=0.82645, /{1.3)=0.59172, /(1.5)=0.44444 and /{I.7)=0.34602
b. f (0.26) if f (0. 1) =0.90909, f (0 2) =0.76923, / (0.3)= 0.66667 and f (0.7) =0.58824
c. /(2.5) if /(2.2)=1.48324, /(2.4)=1.54919, /(2.6)=1.61245 and /(2.8)=!.67332
2. Detennine the maximum error bound in each of the cases in I . if d1e data were obtained from
die following respective functions.
I 1
a. /(.r)= - , b. f(x)= - c. f(x)=Ji
x- l+x
3. Use the second and the third degree Lagrange interpolating polynomials to approximate each
of die following:
a. /(2.4) if /(2.1) =14.89123, /(2.3)=15.67893, /(2.6)=16.50467 and /(2.7)=

16.76854
b. f (- 0.25) if f (-0.6) =~.15632, f (-0.4)=-0.05719, f (~.3) = 0.26758 and f (-0. 1)=
0.856345
c. f (0.4) if /(0.2) =0.65743, f (0.3)=-0. 105342, f (0.5)=0.2 1654 and f (0.6)= 1.01745
S.3 Newton's Interpolating Polynomials

Newton's equation of a function dlat passes through two points (x0 , y 0 ) and (x,, y 1) is
P,(x)=a0 +a,(x - x0 ) (5.11)
We can evaluate the coefficients easily. Set x = x0 in (5.11).

0o = P, (xo) = Yo
Set x=x, in (5. 1 L).
P, (x,) = y 1 = a0 +,a, (x, - x0 )
Or a = y, - ao
' I
x, - xo
or, a, = Y, - Yo (5.12)
x, - xo
Now examine Newton's equation of a function that passes dlrough duee points ( x0 , Yo), (x,, y,) and
(x2,Y2)·
P2 (x) = <J0 + a, (x - x0 )+ ~ (x - x0 )(x- x,) (S. 13)
Note that Equation (5. 13) is constructed by adding a third term to the righ1-hand side of Equation
(5. 11 ). The third tenn in Equation (5.13) vanishes both at x0 and x,. Therefore. our first two coeffi.
cients a 0 and a, are still !he same as that found utilizing Equation (5. 1I) for the two-point case. All
we have to do is to find the third coefficient, a 2• In order to find "2, set x = x1 in Equation (5.13).
P1(x1) = y2 =a0 +a, (x 2 - x0)+a1(x1 - x0 )(x2 - x,)

or,
Yi = Yo + Y, - Jo (x1 - xo)+a1 (--½ - Xo}(x2 - x,)

X1 - Xo
After a few steps of manipulations we get,

Y1 - Y, y, - Yo
x 2 - x1 x1 - x0
a.., = (5.14)
- X - X
1 0
Newton's equation of a function that passes through four points can be written by adding a fourth
term to the right-hand side of Equation (5.13).
(5. 15)
The fourth term will vanish at all three previous points and, therefore, leaving all three previous
coefficients intact. All we have to do is to find the fourth coefficient a3• We set x = xJ in Equation
(5. 15) and do some algebraic manipulations. Actually, the coefficients can be written in a systematic
manner by using the divided differences of the function values.
5.3.1 Divided difference and the coefficients

The divided difference of a function, / with respect to x1 is denoted as f[x1 ] . It is called as zeroth
divided difference and is simply the value of the function, f at x,.
f[xi]=J(xi) (5.16)
The divided difference of f with respect to x1 and x1 +,, called as the first divided difference, is
de noted f [ xi' x1., ] and is defined as
(5. 17)
The divided difference off with respect to x 1, x1., and x1• 1 , called as the second divided difference ,
is denoted f [x1, x1.,, x1• 1 ] and is defined as
- f [ X,+1• X1+2 ] - f [x,, X1+1]

f [X1, X ;+t · .\'1+2 ] - ~--~~~--~ (5.18)
x,.~ - :x;
The divided difference off with respect to x 1, x,+i• x1+2 and x1+3, called as the third divided difference ,
is denoted f [x,, x,.,, x,.
1 , X 1+3 ] and js defined as
(5.19}
and so on.
From the definitions shown in (5.16) to (5.19} it is clear that the divided difference with respect
to two points is expressed in terms of divided differences with respect to one point, the divided
difference with respect to three points is expressed in tenns of divided differences with respect to
two points and the divided differerace with respect to four points is expressed in tenns of divided
differences with respect to three points and so on. The divided differences can be computed in an
inductive manner.
Omce the required divided differences are calculated, the coefficients of Newton's interpolating pol-
ynomial can be expressed in the fol lowing manner.
a0 = /[x0 ]
a,= f [x x, ]
0,
a2 = f [xo, x,, x2 ]
al = f [Xo, Xi' X1, X3)
and soon.
The prinlary advantage of Newton's interpolating polynomial is that more data points can be
incorporated to generate a higher-order polynomial wi thout repeating the calculations used for the
lower-order polynomial.
A table similar to Table 5. l can be constructed for the divided differences in a convenient manner.
Note that the table could still be extended by two columns on the right to include two fourth divided
differences and one fifth divided differences. The divided differences from the top diagonal become
the coefficients of the Newton's interpolation polynomial.
Using divided differences for the corresponding coefficients, Equation (5. 15} can be written as
~ (x) = f[x 0 ]+(x - x 0 )f[x0 ,x,]+(x - x0 )(x- x,) f [x0 , x,, x 2 ]
(5.20)
+ (x - x0 )(x - x, )(x - .~)J[x0 , x,, x 2 , xJ
Newton's interpolation polynomial of deg ree n can be written as
P,. (x) = P,,. , (x) +(x - x 0 )(x - x,) · ·· (x - x•. , )J[x 0 ,. x,. ·· ·, xJ (5.21)
where the coefficient of x• is

a. =J[~:0 ,x1,···,.x.J
Table 5.1
¢
Divided differences
X f (x) First divided differenc~ Second divided differences Third divided differences
n
:r
Xo f[xo ] ,,
Al
f [xo,X1J= ff Xi]- ![ x.,] ....i

V,
Xi -Ao
[. . ] f [xi, . 1:i ]- f (x0. x1]
...
:::,
(!)
-,
XI /[x1] j A0 , Ai, X2 - -0
X2 - Xo 0
...o·iii
f[xi, xi ] = f[x2 ]- f[x1] I [x . X ]-- f [xi' x 2.x3]-f [x0 , Xi, x2]
.t.,, :::,
0 , X1,
X1-X1 • 3 X3- Xo
/ [X ] - j [ X1, X3 ] - f [ Xi, X2]
X2 f(x2 J 11 X 1 ,X3 -
X3-X1
.f(x2,x ]= .f[x1J-.f(x2 ] fl ..\'.. , 1_- .r [x2,x3.x4]-.f[x1,x1 ,x1 ]

3 x - x 11 X 1, X 3, .X-1
x4-x1
3 2
/(x3] f [ _f [x.1, x. ]- f [xi, x3]

X3 Xz, X3, X4 -1 x,.-X 2
f [x , x, ] = f [xJ - f [ Xi] /[ X2, X3,• X4, X3 _ /[x,,x4 ,x5 ]- /[X;i,X3,x4 ]

3 X4 -x3 1- X5 - X2
X4 /[x4 ] f [X3, X,.11 .X5]-

-
J[x4, x5]- J[x3, x. ]
x, -x,
/[ X4,Xs ]-f
-
(x5]- /(x4]
X5 - X4
X5 f(x5]
Chapt er S: Interpolat ion ~
Comparing the coefficient of x• of Newton's interpolating polynomial with the corresponding

coefficient of the Lagrange interpolating polynomial in Equation (5. l 0) we get
" f(xt )
f[X o• X " · •• X ] - " ' -· ~ (5.22)
• " - £..'(
k=O Xt
)
ExampleS.3
Find Newton's interpolating polynomial to approximate a function whose 4 data points are given below.
X f (x}
2.0 0.85467
2.3 0.75682
2.6 0.43126
2.9 0.22364
3.2 0.08567
Table 5.2 Divided differences for the example problem
.i X1 / [xi] / [ x 1,., x1] f [x,_1, x,_., x1] f[x 1-a,•... x1] f [X1...11 ... ,x1]
0 2.0 0.85467
- 0.32617
] 2.3 0.75682 - 1.26505
- 1.08520 2.13363
2 2.6 0.43 126 0.65522 - 2.02642
- 0.69207 - 0.29808
3 2.9 022364 0.38695
- 0.45990
4 3.2 0.08567
Solutio11
We will prepare a table of divided differences similar to Table 5.1.
Tbe coefficients are
Go =0.85467
a, =-0.326 17
a1 ,:e - J.26505
a1 =2.13363 and
a, = - 2.02642
<0> Chapter 5: Interpolation
P.. (x)=a0 +a, (x - x0 )+a2 (x - x0 )(x - x,)+a3 (x - x0 )(x- x,)(x- xJ

+a, (x - x0 )(x- x, )(x - x2 )(x - x3 )
P, (x)= 0.85467 - 0.32617(x - 2.0) - 1.26505 (x - 2.0)(x- 2.3)
+ 2. 13363 (x - 2.0)(x - 2.3) (x - 2.6) (5.23)
- 2.02642(x- 2.0) (x - 2.3)(x - 2.6)(x - 2.9)
We can use P, ( x) 10 estimate the val uc of the function at x = 2.5.
P, (2.5) = 0.85467 - 0.326 I7 (2.5 - 2.0) - 1.26505(2.5 - 2.0)(2.5 - 2.3)

+ 2. 13363 (2.5 - 2.0) (2.5 - 2.3)(2.5 - 2.6)
- 2.02642(2.5- 2.0)(2.5- 2.3) (2.5 - 2.6) (2.5 - 2.9)
=0.53564
•
S.J.2 Error bound of Newton's interpolating polynomial
If we have n + I node points, x0 , x,, x2 , •.• , x,, then Newton's interpolating polynomial can be
ex.pressed as
P. (x) = f (x0 )+(x - x0 ) f [x0 , x,]+ (x - x0 ) (x - x,) f [x0 , · · · , x~]
+ •· • + (x - ·r0 } •·• (x - x1t-l ) f [x0 ··· xn ]
1 '
Suppose r is a real number distinct from x 0 , x,. x 2 , · • · , x., then

P,,., (x) = f (x0 )+(x - x0 ) f [x0 , x, ]+(x - x0 ) (x - x,) f [x0 , · · ·, x2 ]
+ ··· +(x - x0 ) ··· (x - x._,) f [x0 , · · · , x,.]+(x - x0 ) •·· (x - x,.) f [x0 , • · • • x., r]
P.+, (x)=P.(x)+(x - x0 ) •·· (x - x,.)J[x0 ,···,x., r]
In P.+i (x), r is a new node, and hence, P.+i (t)= /(1).
Therefore,
f(1)=P,.(r)+(t - x0 ) ··· (r - x,,)f [x0 ,---,x., 1]
f (1)- P,. (r)=(r - x0 ) ··· (r - x,.)J [x0 ,··•,x., r] (5.24)
Equation (5.24) is another fommla for error f (t) - P,. (1).

Comparing this error term with the error term of the Lagrange interpolating polynomial we get
Problem Set 5.2

1. Use Newton' s interpolating polynomials to approximate each of the fo llowing:
a. /(1.4) if /(1.1)=0.82645, /(1.3)=0.59172, f(I.5)=0.44444 and f(I.7)=0.34602
b. f (0.26) if /(0. 1) =0.90909, /(0.2) =0.76923, f (0.3)= 0.66667 and/ (0.7)=0.58824
c. f (2.5) if f (2.2) = 1.48324, f (2.4) = l.54919, f (2.6) = 1.61245 and f (2.8) = 1.67332
2. Use Newton's interpolating polynomials to approximate each of the following:
a. f (2.4) if/ (2. 1) = 14.89 E23, f (2.3) = 15.67893, f (2.6) = 16.50467 and f (2.7) = I 6.76854
b. f (-0.25) if /(-0.6) =-0.15632, /(-0.4)=-0.05719, f (-0.3) = 0.26758 and /(-0. 1)=
0.856345
c. /(0.4) if /(0.2) =0.65743, f (0.3)=-0. 105342, f (0.5)=0.2 1654 and f (0.6)= 1.01745
S.4 Hermite Interpolation

Hennite polynomial~ agree with/ at x,. x2 , x3 , .. . , x •. In addition, their first derivatives agree with
thefirstderivativcoffat x, ,x2'x3 , ... ,x•.
Hennite polynomial of degree at most 211 + 1 is written as:
ti ti
H1n+, (x)"' I,J(x1)H. 1 (x)+ L f'(x1 )H•.1 (x)

J=O J=O
where
H,,_1 (x) =[1 - 2(x - xJL;_1 (x,)]{_ L,._1 (x)J2

2
H,,_1 (x) = (x - x1){ L,,1 (x)}
L•.J (x) = jth coefficient of Lagrange polynomial of degree 11

Proof:
For a coefficient of Lagrange polynomial, it is either Oor I at the given data points. Therefore,
L (x I )--
0, if i *j
•J { I, if i = j
When i* j, H •.J (x1) =0 and H,, 1 (x1 )=0 when i = j,

2
H,,_1 (x1) =[1- 2 (.~1 - x1 } L;_1 (x,)]{ L•.J (x, )}
x1 )L;,1 (x1)1 L,, 1 (x1 )}

2
=[l - 2(x1 -
=[1 - 2·0 · L;., (x,)J 1=I

2 2
H,,.Ax,) =(x1 - x1 ) { L.1 (x, )}
=(x1 - x1 }1 {L•.1 (x1 )t =0

Therefore,
Now, we have to show that H~•• , agrees with f' at each node.
H•.1 (x) =[1 - 2(x - x1 ) L:,_1 (x1 )]{ L._1 (x)f

H:,_1 (x) =- 2L;_1 (.x1){ L•., (x)t +[1 - 2(x - x1 ) L;_1 (x1 )]2L:,J (x) L,,_1 (x)
:. H;._1 (x1)=0 when i * j
and H;_1 (x,) =- 2L;,_1 (x )+2L;J (x, ) = 0 when i= j
1
H._1 (x) = (x - x1 ){ L•./ x)f

H:,_1 (x) ={ L,,_1 (x)r +(x - x1 )· 2· L:,_1 (x) · L,,_1 (x)
A '
)r •
H:,_1 (x,)={ L. 1 (x, +(x, - x1 ) · 2 L:,_1 (x1) · L._1 (:(1 )

R:,_1 (x,) = 0 when i * j and
B;,_1 (x1)=1 wheni=j.
. "
:. H; ..,(x,)= L,/(x1 )H;_1 (x,)+ L,f'(A°;)H;,_1 (x,)
J=O J=O
•
H;••,(x,)= L,f(.x1 )·0+ f'(x 1 )· 1
J=O
H;••,(x1 ) = f'(x1 )
Example S.4
Given the following points and the respective first derivatives, determine the approximate value of y
at x = 2.25 using Hcm1ite interpolating polynomial.
X0 = 2.1 X 1 = 2.2 X2 = 2.3
Yo =0.7394 y, = 0.6849 _v2 =0.6359
y~=-0.574 y;=-0.516 y;=-0.4651
Solutio11
In this example, 11 = 2 and, therefore, the Heanite interpolating polynomial will have at most degree 5.
½.o(x) (x - x,)(x - x2 ) 50(x - 2.2)(x - 2.3)

(x0 - x,)(x0 - x2 )
½., (x ) -- ~
(x - x0 }(x - xJ
-~-- - IOO(x- 2.l)(x- 2.3)
(x, - x 0 )(x, - X 1 )
(x - x 0 )(x - x 1)
Lu(x) ~-~-~ S0(x - 2.l)(x - 2.2)
(-½ - x0 }(x2 - x,)
, ( )- (x - x ) (x - x,)
½_ox - - - ' - - =2 - - -+ - - ' - - = - - - lOOx- 225
(x0 - x1)(x0 - x2 ) (x0 - x 1)(x0 - x2 )
, (X ) = - ~
½.t (x -
- x~
2) (.l" - X0 )
-+-~-~ - 200x+440
(x1 - x 0 )(x, - xi) (x, - x 0 )(x, - x2 )
(x - x,) (x - x )
½~2 (x) 0
----'--=---+ ----'---'--'-- lO0x - 215
(x2 - x 0 )(x2 - x,) (x 2 - x 0 )(x2 - x,)
2
H2.0 (x) ={l - 2(x- x0 )½o (x0 )}{ ½.o (x)}
2
= {1 - 2(x - 2. l)(I00x2. l - 225)}{50 (x - 2.2)(x - 2.3)}
= 75000x 5 - 830000x• +3.67275x 106 x 3 - 8.12285xl0 6 x 2 +8.97897x - 3.968558xl06
H 2 J (x) = { l - 2(x- x1 )£.;.1 (xi)}{½., (x) }2

= {l - 2{x - 2.2)(- 200x 2.2+440)}{- IOO(x - 2.l)(x - 2.3)}2
"'I OO00x" - 88000x 3 +290200x 2 - 425040x + 233289
?
H2.2 (x)={l- 2(x - x2) ½.2 (x2)H L2.1 (x)r

= {1 - 2(x - 2.3)(100x 2.3 - 215)}{ 50(x - 2.1) (x - 2.2)}
4
= - 75000x5 +820000x - 3.58475x 106 x 3 + 7.83265x l06 x 2 - 8.55393x 106 x+3.73527 x 106
• 2 2
H 2 _0 (x) =(x - x0 ){ L 2_0 (x)} =(x- 2.l){S0(x - 2.2)(x- 2.3)}
=2500x5 - 27750x4 + 123175x3 - 273292.5x 2 + 303094x - l 34418.9
• 2 '
H 2 _1 (x) =(x - x,){ L 2_1 (x)} =(x- 2.2){- IO0(x - 2. l)(x - 2.3)r
= 10000x5 - ] 10000x4 +483800x 3 - I.06348x 106 x 1 + I.168377x 106 x - 513235.8
H2_2 (x) = (x - xi){ l-i 2 (x)} 2 = (x -
2.3){50(x - 2. l)(x - 2.2)}
2
= 2500x5 - 27250x4 + I 18775x 3 - 258777.5x 2+ 281820x - l 22730.3

2 2
H 5 (x)= L,J(x1 ) H2 _1 (x) + L f'(xi )H2 _1 (x)
J=O J=O
H 5 (x) = y0 H 2 _0 (x)+ y 1H 2 .1 (x)+ y2 H 2 _2 (x)+ y~H2_0 (x)+ y;H.,_, (x)+ y;H2.2 (x)
5 4 3 2
H 5 (x) =4.75x - 52.525x +232.1225x - 512. 1847x +563.465x - 245.9722
H 5 (2.25) = 0.6598 •
Problem Set 5.3

1. Use Hennite interpolation that agrees with the given data to approximate each of the
following:
a. /(1.4) if J(J.1)=0.82645, /'( l.l)=- 1.50263,
f (1.3) = 0.59172, /'(1.3) =~.91033, and
f (1.5) = 0.44444, J' (1.5) = ~.59259.
b. /(0.26) if /(0. 1)=0.90909. /'(0.1)=~.82645,
f (0.2)= 0.76923, f'(0.2) = 0.69444, and
/(0.3)=0.66667, f'(0.3)=~.59 172.
c. f (2.5) if f (2.2) = 1.48324, /'(2.2) = 0.33710,
f (2.4) = 1.54919, /' (2.4) = 0.32275, and
f (2.6) = l.61245, f' (2.6) = 0.31009.
S.S Cubic Spline Interpolation

A higher degree polynomial or the use of more data points do not necessarily improve the approx-
imation of arbitrary functions on closed intervals. Fluctuations over a small portion of an interval
I.hat may be approximated by a higher degree polynomial would create large fluctuations over the
remaining portion of the interval. To overcome this, we can divide the interval into a group of subin-
lel!Vals and apply different approximating polynomial on each subinterval This type of approxima-
tion is called piecewise-polynomial approximation.
A simple case where I.be approximating polynomial on each subinterval is a straight line is called
piecewise-linear interpolation. Although it is straightforward to do approximations by utilizing
piecewise-linear interpolations, the overall function is not coutinuous:ty differentiable and, therefore,
do not provide the smoothness in a geometric sense.
Cubic polynomials between successive pair of nodes are commonly used in piecewise-polynomial
approximations. A cubic polynomial has four constants and, therefore, provides for sufficient
flexibility to satisfy the conditions that the approximating function be continuously differentiable
and, in addition, has a continuous second derivative.
Let us assume that f (x) is defined on [a, b]. Given a set of nodes a= x0 < x, < x2 < · ·· < x,, = b,
detemline a cubic spline interpolating fw1ction S that satisfies the following conditions.
a. S1 (x) is a cubic polynomial on the subinterval [x1 ,x1 +,] for each j = 0, 1, 2, · ··, 11 - I
b. S,(x1 )=f(x1 ) and S1 (x1. ,) =f(xi.,) for each )=0, 1,2,···,IJ - l
c. S1. , (x1.,) =S1 (x1.,) for each j =0, L, 2, ···, n - 2
d. s;.,(xj+I) =s; (x1+1) for each j =0, l, 2, .. ·, I! - 2
e. s;., (XJ+I) =s;(x,.,) for each j =0, I, 2, · ··, 11 - 2 and
f. one of the following boundary conditions is satisfied:
I. S'(x0 ) = J'(x0 ) and S'(-r.) = J'(x.) (clamped boundary)
II. S'(x0 ) = S'(x.) =0 (natural boundary, also called free boundary)
The cubic polynomials and their first and second derivative can be expressed as
S1 (x) = a1 +b1 (x - x1 )+c1 (x - x 1)2 + d1 (x - x1 )3
s/ (x) =b1 +2c1 (x - x1 )+3dJ(x - x1}2 and
s/ (x)=2c1 +6d1 (x - xi)
wlnere a1' b1, c1 and d1 are the coefficients of the jth segment of the cubic spline.
A generalized set of linear equations can be developed based on the above conditions. To develop
the necessary set of equations, we aeed equations from conditions b, d, e and f.
Frnm condition b, we get the following 211 equations.
S, (x1 ) =l1 1 + h,b, +h,2c, + h,3d1 =y2
S2 (xi) =a2 = Y2
sl (x2) =<12 +ltib2 + h/c2 + h/d2 = Yl
Sn- 1 (x·,.-1)-- an- l -- yn- 1
Frnm condition d, we get the following 11 - l equations.
b0 +2h0c0 +3111,/ d'0 - b1 = 0
b1 +2h,c1 +3h,2d1 - b2 =0
b2 +2h2 c2 +3h_/d2 - b3 =0
From condition e, we get the following 11 - I equations.

c0 +3/i0 d 0 - c1 =0
c1 + 3'11d 1 - c2 =0
c1 +3h1d 1 - c3 =0
c,,_2 + 3h._2d._2 -c._,= 0

The remaining two equations come from Lhe boundary condition.
I ( I I I
For clamped boundary: S0 x 0 ) = .Yo and S,,_, (x.) = Y.
b0 = y0 ' and
,
b._, + 2h._,c__, + 3h._,-d,,_, = Y.'
For free boundary: s0• (x0 )=0 ands._,• (x,.)=0

Co =Oand
c._, + 3h0 _ 1d,._1 = 0
For free boundary. the last two rows of the coefficient matrix have to be modified. The resulting
coefficient matrix is:
a0 b0 Co do a, b, c, d, a, b, c, J, ... ... a.,._2 b... - 2 c.-1 d,._2 a"_' h,.-1 c._, d11_,
I 0 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0
h' h/ 0 0 0 0 0 0 0 0 ..• 0 0 0 0 0 0 0 0
1 "• • 0 ...
0 0 0 0 I 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 I h/ h/ 0 0 0 0 ... 0 0 0 0 0 0 0 0
0 0 0 0 0 "·0 0 0 0 0 0 .. . 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 h2 1,/ ,.,, ... 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 Ir._, h_,1 h,._, ]
0 I 21,. 31,_' 0 -1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
A= 0 0 0 0 0 1 2h, 3/~2 0 -I 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 I 2/,2 3/i,' 0 -] 0 0 0 0 0 0 0 0
... ... ...
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I 2/J,._? 31,,,_?? 0 -1 0 0
0 0 I 3/ro 0 0 -I 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 31,, 0 0 -1 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 I 3h, 0 0 -1 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I 31,11-? 0 0 -I 0
0 0 I 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3h._,
Chapter S: Interpolat ion
<0>
The last two rows of B matri x wiU be modified as weU.
Yo
Y,
Y,
Y1
Y2
B= )1,,_,
Y.
0
0
0
0
For clamped boundary, the coefficiellt matrix can be written as following.
llo bo Co do b;
a1 l"1 d, a, bi C1 d, ..• a,,,_1 bA- 1 CR- 1 d11-'! "•-I b.-1 C11-1 d,-1
0 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0
1 1,. h/ h.,/ 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0
0 0 0 0 I 0 0 0 0 0 0 0 .. . 0 0 0 0 0 0 0 0
0 0 0 0 I h, h/ 1,JJ 0 0 0 0 ... 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 ... 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 I h2 h,/ ,,., ... 0 0 0 0 0 0 0 0
0 0
0 0
0
0
0 0
0 0
0
0
0
0
0
0
0
0
0
0
0
0
0 0
0 0
0
0
0
0
0
0
0
0
0
0 J
,,__0, 0
h,H!
0
1,11- t
)
0 2\ 3i,/ 0 - J 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
A= 0 0 0 0 0 1 21,, 31,11 0 -I 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 I 2/i, 31,,' 0 -I 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 I 2lr.,._2 3/J•-2? 0 -1 0 0
0 0 I 3l1o 0 0 -1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 31,, 0 0 -1 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 I 31,, 0 0 -I 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3h,._2 0 0 -1 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2h._1 3/J,,_/
AG = B
where
ao Yo
ho Y,
Co Y,
do Y-2
a, Y2
b,
G= CI and B = Yn- 1
d, Y.
0
Qn-1
b._, 0
Cn- 1 Yo
d._, Y,, '
Example 5.5
Let us consider four data points, (I, 2), (2, 3), (3, 5) and ( 4, 6). Construct a natural cubic spline that
passes through these points.
Solution
We have four data points. Therefore, we will have 3 subintervals. We define three cubic polynomials,
one for each subinterval in the following manner.
For interval ( 1, 2] ,
2
S0 (x) =a0 +b0 (x - x0 )+c0 (x - x0 ) +d0 (x - x 0 f
3
i.e., S0 (x) = a0 +h0 (x -1)+c0 (x - 1/ +d0 (x -1)
For interval (2, 3),

2 J
S, (x)=a, +b1 (x - x,)+c, (x - x,) +d, (x - x,)
3
i.e., S1 (x) =a1 +b, (x - 2)+c1 (x- 2}2 +d, (x- 2)
For interval (3, 4 J.

S2 (x) = ~ + b2 (x - x 1 )+c2 (x - x2 )1 + d1 (x - x 2 )
3
2
i.e., S2 (x)=a1 +b2 (x- 3)+~(x - 3) +d1 (x - 3)'
We will get 6 equations from the fact that the splines S0 (x), S, (x) and S2 (x) must agree with the
nodes at the data points.
Chapter S: Inte rpolation ~
Therefore,
S0 (1)=f(l)i.e.,o0 =2
S0 (2)=a0 +b0 +c0 +d0 = f (2) i.e., a0 +b0 +c0 +d0 =3
S, (2) = f (2) i.e., a,= 3

S, (3) =l11 +b, +c, +d, = /(3) i.e., a, +b, +c, +d, = 5
The first derivative of the cubics are:
s; (x) =b + 2c (x - 1)+3d (x - 1}2

0 0 0
S,' (x)=b, +2c 1 (x - 2)+3d1 (x - 2)

2
2
S2' (x) = b2 + 2c2 ( x - 3)+ 3d1 (x - 3)
We get 2 equations from the first derivatives in the following manner.
so' (2) = s,' (2) i.e., /Jo+ 2co + 3do = bl
s,' (3) =s; (3) i.e., b + 2c, + 3d, =b

1 2
The second derivative of the cubics are:
S.," (x)=2c0 +6d0 (x - 1)
s,'' (x)=2c, +6d1 (x - 2)

s;' (x)=2c1 +6d2 (x - 3)
We get 2 equations from the second derivatives in the following manner.
S.," (2) = S," (2) i.e., 2c0 +6d0 = 2c1

s," (3) = s/' (3) i.e., 2c, + 6d, = 2Ca
Frnm the natural (free) boundary condition, we get 2 equations in the following manner.
s.," (I)= s;' (4) = 0, therefore,

2c0 =0, i.e., c0 =0 and
2c2 +6d2 =0
Reviewing the equations, we find the values for d1e following coefficients.
a0 =2
a, =3
We still have to determine the values for 8 coefficients utilizing the following equations.
b0 +d0 =l
b, +c, +d, =2
b2+C2+d2=l
b0 +3d0 - b1 =0
b, +2c, +3d, - b2 =0
6d0 - 2c1 =0
2c, +6d, - 2c2 =0
2~+6d2 =0
Solving the system of linear equations, we get

2 l 5 2
b0 = , d0 = , b, = , c, =l, d, = -
3 3 3 3
5 I
b, =- . c, =-1, d, =-
- 3 - - 3
The spline is
6.0
5.5
5.0
4.5
S(.t ) 4.0
3.6
3.0
2.5
2.0
1.0 1.3 1.6 1.9 2.2 2.5 2.8 3.1 3.4 3.7 4.0
.t
Figure 5.2 Natural cubic spline

Chapter S: Inte rpolation ~
ExampleS.6
We now consttuct a spline with claimped boundary condition that agrees with the nodes at the same
four data points, (I, 2), (2, 3), (3, 5) and ( 4, 6). The clamped boundary condition is s0' {I)=/ (1) = 2
ands/ (4)= / (4)=3.
For this example, we will be able to use all equations from the previous example up to the point of
the boundary conditions. Due to the clamped boundary condition, our last two equations will be
different from the previous example.
b0 =2
b1 + 2c2 + 3d2 = 3
Reviewing the equations we get the following coefficients right away.
a 0 =2
a, =3
a~=5
b0 =2
For the remaining 8 coefficients we have to solve the following system of 8 linear equations.
a0 +b0 +c0 +d0 = 3; i.e., c0 +d0 = - I
a 1 +b, +c1 +d 1 =5; i.e.. b, +c1 +d, =2
a 2 +b2 +c2 +d2 = 6; i.e., b2 + c2 +d1 = I
b0 +2c0 +3d0 =b,; i.e., 2c0 +3d0 - b1 = - 2
b, + 2c1 +3d, =b2 ; i.e., b, +2c1 + 3d1 - b2 =0
2c0 +6d0 = 2c,; i.e., c0 + 3d0 - c1 = 0
2c, +6d, = 2c2 ; i.e., c, + 3d1 - c2 = 0
b2 +2c2 +3d2 =3
Solving the system of linear equations, we determine the remaining coefficients.
37 22:
Co=- -, do= -
15 I5
22 29 7
b, = - , c1 = - .d, = --
15 15 5
17 34 32
b., = - , c., =- -., d..., = -
- 15 - 15 - 15
The clamped spline is
,.o
5.5
5.0
,.5
S(.Y) , .o
3.5
3.0
2.6
1.3 u u u 2.5 2.8 3.1 3;1 3.7 ,.o

.,
Figure 5.3 Clamped cubic spline
ExampleS.7
Let us consider five data points (1, 3), (2, 4), (3, 6), ( 4, 6.5) and (5, 7). Construct a cubic spline with
free boundary that passes through these points.
We have five data points. Therefore, we wiJJ have 4 subintervals. With more segments, it is better to
utilize the generalized approach for developing the set of linear equations.
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 ho h/ h/ 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 ] It, 11I 2 h,3 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 I h2 h,/ 1,/ 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 I 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 I h, h) 1 h3'
A=
0 I 2h0 3h/ 0 -) 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 I 21,, 3h12 0 - 1 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 I21,2 3/i/ 0 - I 0 0
0 0 I 3'10 0 0 - I 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 J 3h, 0 0 -I 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 1 3f½ 0 0 -1 0
0 0 I 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 3"3
/r0 =h =h =/,3 =l. Therefore,
1 2
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 I 1 I 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 I 1 1 l 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 1 I 1 1 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 1 I I l
A= ., 0 - I 0 0 0 0 0 0 0 0 0 0
~
0 l 2
0 0 0 0 0 1 2 3 0 - I 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 I 2 .," 0 - ] 0 0
0 0 1 3 0 0 -1 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 3 0 0 -1 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 1 3 0 0 -I 0
0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 I 3
Go
bo
Co
do
a,
b,
G= c, and
d,
03
b;
C;
d,
3
Yo 4
Y, 4
Y, 6
Y2 6
6.5
Yi
6.5
7
B= Y.-, = 0
Y. 0
0 0
0
0 0
0 0
0 0
0
G=A-'B
ao 3
ho 0 .625
Co 0
do 0.375
al 4
b, J.75
CI 1.125
d, -0.875
G= =
a2 6
h2 1.37S
C2 - 1.5
d2 0 .625
a3 6.5
b3 0.25
C3 0.375
d3 -0.125
The cubic spline is:
x0 ~x ~ x,
3
2.375 + 0.625x+ 0.375{x -1)
0.5 +l.75x+ 1.125(x - 2)2 - 0.875{x - 2)3
S(x)=
1.875+ !.375x- l.5(x - 3)2 +0.625(x - 3)3
5.5+0.25x+0.375{x - 4) 2 - 0.!25(x - 4 )3
7.4~ - - - - - - - - - - - - - - - - -~
6.7
6.0
S(,Y)
2.7
2.0+
1.0
-1.4- -1.8
- -2.2
- ~2.6
- ~3.0- ~
3.4
- -3.8- -4.2
- -4.6
- --<5.0
Figure 5.4 Cubic spline with free boundary
Example 5.8
Let us consider the five data points (1, 3), (2, 4), (3, 6), (4, 6.5) andl (5, 7) utilized in the previous
example. Construct a cubic spline with clamped boundary that passes through these points and
y0' = 3 and y;= 3.
We have five data points. Therefore, we will have 4 subintervals. Let us utilize the generalized
approach for developing the set of Linear equatioas.
I 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Iho h/ h0 3 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 1 h, h2
I
hI l 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 I 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 1 h1 h/ h/ 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 I 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 I It, ,, 2
3
hl ,
A= 0 I 2110 3ft0 2 0 -] 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 I 2'11 31,12 0 -I 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 I 2/1-,3'1/ 0 - I 0 0
0 0 I 3h0 0 0 -l 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 3ft, 0 0 - I 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 I 311-, 0 0 -] 0
0 I 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 I 2~ 3/J/
'10 =It, =h ='11 =1. Therefore,

2
I 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 1 l I 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 I 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 I I I I 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 I 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 1 1 I ] 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 I 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 l J I
A=
0 1 2 3 0 - I 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 I 2 3 0 -l 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 I 2 3 0 -1 0 0
0 0 l 3 0 0 -] 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 I 3 0 0 -1 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 I 3 0 0 -] 0
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 I 2 3
Chapter S: Interpolat ion
<€P
Yo 3
Y, 4
Y, 4
Y2 6,
Yi 6,
)'3 6.5
)'3 6.5
Y, 1
B= =
0 0
0 0
0
0
0
0
0
0 0
0
Yo'
3
y. ' 3
G = A-'B
Oo 3
ho 3
l'0 4.071
do 2.071
a, 4
b, 1.071
c, 2.143
d, - 1.214
G= =
a2 6
h2 1.7 14
C2 - 1.5
d2 ,0286
03 6.5
b3 ---0.429
C3 ---0.643
d3 1.571
The cubic spline is:
3x- 4.07l{x - l)2 + 2.07I{x- J)3 x0 ~ x ~ x,

I.857+1.07 1.r +2.L43(x - 2)2 - I.214(x - 2)3 x, ~x~ x2
S(x)=
0.857+l.7 14x- l.5(x - 3)2 +0.286{x - 3)3 x2 ~x~x3
8.214 - 0.429x - 0.643(x- 4)2 +1.57I{x - 4) 3 x3 ~ x ~x,
7.4~ - - - - - - - - - - - - - - - -~
6.7
6.0
5.4
S(.Y)
2.7
2.0+ - - - ~ - ~ - ~ - - ~ - ~ - ~ - --
1.0 1.4 1.8 2.2 2.6 3.0 3.4 3.8 4.2 4.6 5.0
X
Figure 5.5 Cubic spline with clamped boundary
Problem Set 5.4

1. Consider four data points; (l, 3), (2, 4), (3, 7) and ( 4, 8). Construct a natural cubic spline that
passes through these points.
2. Construct a spline with clamped boundary condition that agrees with the nodes at the four data
points; (1, 3), (2, 4), (3, 7) and (4, 8). The clamped bow1dary condition is s0' (1) =/(I)= 2
and S2' (4)"' / (4)aa3.
3. Consider five data points; (2. 3), (3, 5), ( 4 , 7). (5, 8) and (6, 9). Construct a cubic spline with
free boundary that passes Lb.rough these points.
4. Consider five data points; (2, 3), (3, 5), ( 4 , 7), (5, 8) and (6, 9). Construct a cubic spline with
clamped boundary that passes through these points and y;
= 2 and = 3. y;
S.S.1 Constructing cubic splines from the second derivative
In this method we will express the second derivative of the cubic spline at the given nodes in tenns
of wtknowns M0 , M,, M 2 , .. . , M •. TI1e unknowns then can be detennined from a set of linear equa-
tions developed from the continuity of the first and second derivatives at the given nodes and the
boundary conditions.
Let the second derivatives of the ;piece-wise cubic spline at the given nodes be expressed with
variables M0 , M,, M2 , • •• , M,. such that
s"(x1 )=M1 i=0,1,2, ... , 11
s(x) is defined as:
s0 (x) on (x0 ,x1 )
s, (x) on (x,, ~]
s2 (x) on [x1 , x3 ]
s1 ( x) on ( x,. .x,•.]
S
0
(x) = a0 +b0 x + c0 x 2 +d0 x 3
s, (x) =a,+ b,x +c,x 2 +d,x3
The first aud the second derivative of the ith segment of the cubic spline can be expressed as
s;(x)= b1 +2c1x+3d1x 2
s;(x)=2c1 +6d1x
The second derivative of the cubic spline at the given nodes iu tenns of lhe unknowns M0 , M 1 ,
M2 , ~-- , M" are:
s;(x0 )=M0
s;(x1 )=M,
s;(x,)=M,
s~(x2 )=M2
s;(x2 )=M2
s;(xJ=M 3
s;(x3)=M3
<€,> Chapter 5: Interpolation
· n- 1 )=M n- 1
s•,i- l (r
Since sr(x) is a linear function, M 1 .and M ,., can be expressed as

M 1 =mx1 +c and
From the above, we can write

m M1- M,., and
Xi - Xi+I
Therefore.
M 1 - MM x + ~~~~~
'( ) -- ~-~~
S; X
M ,.,x, - M,x,.,
X1 - xl+I X; - xf+l
X;+1 - X 1
(5.25)
Integrate Equation (5.25).
(5.26)
where C is a constant of integration.

Integrate Equation (5.26).
where D is a constant of integration.

h/ M,
s, ( x1) =- -+ Cx1 +D= y1 (5.27)
6
h,2M,., C
s, (X1+1 ) 6 . + x,.,+ D = Y1+1 (5.28)
Subtract Equation (5.27) from Equation (5.28).

1,2M 1,2M
y - y
/+I I
= "I
6 /+I l
6 · /+I - ..i)
l+C(T
I
Y,., - Y; -h, (M 1., - M 1 ) +C

", 6
C= y,.,-
II,
Y, + h, (M - M )
6 I /+I
Multiply Equation (5.27) by x1H
-11/M, C _
- xM + x1x 1._, + Dx,., - y 1x;., (5.29)
6
Multiply Equation (5.28) by x1
'
1,,-M;., C D
x I + x1x,., + x I = y1.,xI (5.30)
6
Subtract Equation (5.30) from Equation (5.29)
D = yI x1., - yI• I x I + h1M1.,x I hI M I x1.,

h1 6 6
(x,., - ..1:)3 M1 +(x - x1/ M,.,
s1 (x)
+ .Y1x,., - Y1.,x, + h,M ,.,x1 h1M1x,.,

h, 6 6
(x,., - x}3 M 1 +(x - x1 }3 M,.,

S( X) = --'---'-''---'---'---'---'-'----'cc.
6h1
(x,., - x)y1 +(x- x1)y,.,
+~-~-~-~--
It,
1
- " {(x1., - x)M1 +(x - x,) M,.,}
6
x 1 ~x~x,.,
0~i~11 - I
Om [x,.x,., ]
- (x, - x/ M 1_1 +(x - x )2 M,

1_1 y - y
_ (M 1 - M 1_ 1)h,_1
s'(X) 1 1 1
---'---'-------'----'--'----'----'-'-'------'- + -"-'------"-'---'-
2h1_1 hi-I 6
Therefore, the first derivative at x1 determined from both expressions: should be equal to each other.
- (xt.1 - x,)2 M1+ Y1+1- y, (M,., - M1 )h1

2hl h, 6
(x, - x,_1/ M , Y, - Yi- , (M, - M,_1)h,_1

-'--'--'--'-'----'- + "-'-----"-'---'-
Using 2h,_1 '11_ 1 6
''1-1= x,-x,_,
h1 =x1, 1 - x 1
- h1M 1 + y,.. - y, h1M I, I + hI M 1 = h1_I M 1 + yI - y1_ I h,_,M I + h,_,M,_,

2 hI 6 6 2 h1_I 6 6
h,_, M + II,_, +h1 M + h: M _ Y1+1 - Y1 Y, - Y1-1
6 1-1 J / 6 l+l - /,/ 1,1-I
From the continuity of the first derivative at nodes i = I, 2, . .. , 11 - I, we get 11 - I equations. We have
11+1 unknowns M0 , M 1, M 2 , ••• , M•.
Case offree boundary

In a case of free boundary, the second derivative at the two end points is set to zero.
M 0 =M.=0
A free boundary case implies thats (x) is linear on (~. x 0 ] and [x,, . oo) .
The equations for i = 1, 2, •· ·, n -1 can be arranged in the following manner.
Y2 - Y1
h,
)'3 - y2
h,
h.-2 M + h,,_, + h,,_2 M = Y,, - Y.-1 Y,,-1- Y,,-2

6 n- 2 J n- 1 / 1__,
h,,_2
The linear equations above can be arranged as AM = B.
ho+ h, !!L 0 0
3 6
h, h, +h1 h1
0
6 3 6
A= h1 k, +h,, Ii,,
0
6 3 6
h,._2 hn- 2 + h,._,

0 0 0
6 3
Y2 - Y, Y, - Yo
h, ho
)'3 - Y2 Y2 - Y,
h2 h,
B= Y, - yJ J3 - J2
Ii,, h-i
Y11 - Yn-l Yn - f - Yn- 2

11__,
h.-2
ExampleS.9
Consider 7 data points, (l, 2), (2, 2.6), (3, 5), (4, 6), (5, 5), (6, 8) and (7, 8.2). Construct a natural
ct1bic spline with free boundary thal passes through these points.
There will be 6 segments, and, therefore, will have 6 piece-wise cubic polynomials. We can write
the data as:
Yo =2, y, =2.6, y2 =5, y3 =6, y, =5, y5 =8, y6 =8.2
Frnm the free boundary condition we know that M 0 =M6 =0.

From the data we calculate the following.
i=O, 1, .. . ,5
2 1
0- 0 0
3 6
l 2 l
- - - 0 0 1.8
6 3 6
- l.4
1 2 1
A= 0 - - 0 B= -2
6 3 6
4
l 2 l
0 0 - - - 2.8
6 3 6
1 2
0 0 0 - -
6 3
We calculate M, , M 2 , • · · , M 5 from the following.
M,
M1
Ml = A- 1B
M•
Ms
M, 3.122
M2 - 1.686
M3 = -4.n1
M. 8.794
Ms -6.398
Frnm the free boundary condition we already have M 0 = M 6 =0. Therefore,
Mo 0
M, 3.122
M2 - 1.686
Ml = -4.777
M. 8.794
Ms -6.398
M6 0
(x,+ 1- x)3 M,+(x - x,)3 M 1+1

S( X) = -'-'-'-'-----'---'---'--'-'-----'-'cc.
6h,
+ (x1+1 - x)y, +(x- x1)Y1+1
h,
- h, {(x1+1 - x)M1 +(x - x 1 )M1+1 }

6
x, SxSx,+,
0SiS11 - I
0.52x3 - l.56x1 + 1.64 Ix+ I .4

-0.80lx 3 + 6.368x 2 - 14.2 18.r+ 11.972 x, Sx Sx2
3 2
-0.5 1Sx + 3.793x - 6.492x + 4.246 x 2 Sx$x3
s(x)=
2262x3 - 29.53x 2 + 126.801x- 173.477 x 3 Sx Sx,
- 2.532x 3 +42.378.r2 - 232.738x+425.754 x, SxSx5
l .066x 3 - 22.39Sx 2 + 155.896.r - 35 1.514 x5 S x S x6
9.0
8.0
7.0
6.0
S(.Y) 5.0
4.0
3.0
2.0
1.0
1.0
Figure 5.6 Cubic spline from the second derivatives (free

boundary)
Case of damped boundary

In a clamped boundary case, conditions are imposed on tbe first derivative at the endpoints.
s'(x0 )= y/ and s'(x.)= Y,.'

<$> Chapter 5: Interpolation
The first derivative on [x0 , x,] is
'=- ho M _ ho M + Y, - Yo
Yo 3 o 6 , ho
h,, M + ho M = Y, - Yo ,
3 o 6 , ho .Yo
The first derivative on [x,,_, , x.]
s' (x) - (xn - •r)? M n-1 +(~. - xn- 1 )2 M n + Yn _ Y,J- 1 (M,. - M,,_,)lr,,_,
2h._, ''•-I 6
2
s'(x,,) (X
n
- x ') M "+ -Vn - yn-1
tJ - .
( M,. - M•-') h,,_,
2h._, "•-I 6
2
s'(x,i)= /r,1- I M V -
"+"''' \I
, n- 1
(M,. - M._,)h,,_,
2h,._, h,,_, 6
( M. - M ,,_, )Ir,,_,
6
h._, M + h,~, M = \' ' Y,. - .\',,_,

,1-1 3 ,, . ,, I 1,_,
6
The set of linear equations can be arranged as
ho M + ho M _ Y, - Yo ,
3 o 6 , - ho Yo
y,.__, - Yn- 2
"•-2
h h , Y,. - y,J-1
...!ci. M +....!!=LM = y
6 ,1-I 3 " ." h,,_1
The linear equations above can be arranged as AM= B.
ho Ito
0 0 0
3 6
ho l1o +h,
6 3 "·6 0 0
0 !i h, +/t2 ~ 0
A= 6 3 6
0 0
h.-1 It,,_, + h.-1 h._,
0
6 3 6
0 0 0
h
~
h._,
6 3
Y, - Yo
Yo
ho
Y2 - Y1 Y, - Yo
h, ho
y3 - Y2 Y2 - Y,
~ h,
B= Y. - J3 J3 - Y2
h3 lti
_.V,. - .,Vn - 1 Y1t- l - Yn- 2
"·-• h.-1
Y,. - Y,1- I
.)',,' -
h_,
Example 5.10
Consider die previous example. However, instead of considering free boundary condition, we will
now consider clamped boundary condition.
Consider y0' = I and y6 ' = J
1 I
- 0 0 0 0 0
3 6
l 2
-6 3
-61 0 0 0 0
--0.4 Mo - 3.183
I 2 1 1.8 M1 3.966
0 - - - 0 0 0
6 3 6 -1.4 M2 - 1.882
l 2 I
A= 0 0 - - 0 0 B = - 2.0 M=A-'B = Ml = -4.840
6 3 6
4.0 M. 9.242
l 2 I
0 0 0 - - 0 - 2.8 Ms - 8.126
6 3 6
I 2 I 0.8 M6 6.463
0 0 0 0 - - -
6 3 6
I l
0 0 0 0 0 - -
6 3
3 3
s(x)=(x,+1 - x) M 1 +(x- x1 ) M1+1
6h,
+ (x,+, - x)y1 +(x- x1)Yi+,
h,
- : {(x1+1 - x)M1 +(x - x,)M1+1 }

x, $x$x,.,
0:;;i:;;11- l
l.192x 3 - 5.166x 2 +7.758x - J.783 x0 $x$x1

--0.975x 3 + 7.83 lx 2 - 18.236x+ 15.546 x, $ x $x2
3 1
--0.493x + 3.497 x - 5.235x + 2.545 X2 $ X $ X3
s(x)=
2.347x 3 - 30.583x2 + 13 l.08Sx -179.215 x 3 $x $x,
- 2.895x 3 +48.04x 2 - 262.03x+475.977 X4 $x $x5
2.432x 3 - 47.83 1x 2 + 313. 19Sx - 674.472 x 5 $x $x6
Chapter S: Inte rpolat ion ~
9.0
8.0
7.0
6.0
S(.Y} 5.0
4.0
3.0
2,0
1.0
1.0
Figure 5.7 Cubic spline from the second derivatives

(clamped boundary)
Problem Set 5.5

L Considetr 7 data points; (2, 3), (3, 3.6), (4, 6), (5, 7), (6, 6), (7, 9) and (8, 9.5). Utilizing the
second derivatives, construct a natural cubic spline with free boUJ1dary that passes through
these points.
2. Consider Problem 1. However, instead of free boundary condition, consider clamped boundary
condition. Consider y0' = 2 and y; = 2
_J
ER6
DI H+-a.RENTIA I
-D tftl AT ON
6.1 Numerical Differentiation

In many signal processing, filtering as well as control systems algorithms. differentiation is required
to accomplish Lhe intended objectives. In most cases, the signals are discretized making differen-
tiation by analytical techniques useless. Hence the use of numerical differentiation that employs
techniques to achieve differentiation utilizing discrete points.
Let us first introduce the formula for the first derivative of.f(x) at x0•
(6.1)
In Equation (6.1), it is stipulated that the formula works when h approaches to zero. However, an
approximation to Equation (6. 1) is sought in the following manner where his considered to be small.
(6.2)
This approximation, known as forward difference formula, works if It is small. A backward dif-
ference formula can also be used to approximate the first derivative of a function in the following
manner.
(6.3)
Let us utilize the forward difference formula as shown in Equation (6.2) to approximate the first
derivative of .f(x) = e-• at x = 1.5. Table 6. 1 shows the approximate values of the first derivative for
three values of h.
<$> Chapter 6 : Numerical Differentiation and Integration
Table 6.1 Effect of step size on cLifferentiation
h / (1.5) f (l.5+h) / (1.5 + h) - / (1.5) Absolute error

h
0.1 0.223 1302 0.2018965 --0.2123364 0.0107937
0.01 0.2231302 0.2209100 --0.2220182 0.OOJJ 119
0.001 0.2231302 0.222907 1 --0.2230186 0.0001] 15
The exact value of the first derivative off (x) = e-, at x = l.5 is - 0.2231302. ll can be noticed that
the approximation becomes closer to the exact value as h becomes smaller.
We can also derive other approximate formulas for the first derivative utilizing Lagrange interpo-
lating polynomial. Assume that a function goes through three points, (x0 .f (x0 )), (x, .f (x,)) and
(x~.J{x2 )). The function can be approximated in the following manner.
f(x)"" P(x)
P(x) = Lo (x) f (x0 )+ L, (x)f (x,) + ~ (x)f (x2 )
P(x) (x - x,)(x - x 2 ) f(xo) + (x - x0 )(x - x 2 ) .f(x,)

(x0 - x 1)(x0 - x 2 ) (x1 - x-0)(x 1 - x 2 )
+ (x - x0 )(x - x 1 ) f(x,)
(x1 - x0 )(x 2 - x,) -
In order to find the first derivative of j (x), we take the first derivative of its approximate version, P(x).
J'(x)"" P'(x)
p '( X ) -- 2x~
-- - x,-- ~
x2 - f( X 0 )
(x0 - x 1 )(x0 - x 2 )
+ 2X - Xo - X2 f(x1)
(x, - x 0 )(x1 - x 2 )
+ 2x - x0 - x, J(x )
2
(x1 - x 0 )(x2 - x,)
If the three points are equally spaced, i.e., x 1 =x 0 + Ii and x 2 =x 0 + 2h then

Chapter 6: Numerical Differentiation and Integration ~
Therefore. the approximation to the first derivative off (x) at x0 can be written as
(6.4)
Equation (6.4) is known as the Three-point formula for the first derivative.
If the three points are again equally spaced but this time with x 0 in the middle so that x, = x0 - hand
x 2 =x0 +h then
Therefore, the approximation to the first derivative can be written as
(6.5)
Equation (6.5) is also known as a Three-point formula.
Example6.1
Find approximate values of f'(J.5) for f (x) = xe·• using both Three-point formulas with h = 0. L
Solutio11
In order to solve this problem we will need values of J (x) = xe··• at several points. Let us prepare a
table showing the values of the function at x = 1.4, 1.5, 1.6 and 1.7. In practical cases, the values of
the function in the fom1 of data will be available, not the function itself.
<8> Chapter 6: Numerical Differentiation and Integration
Table 6.2 Values of/ (x) = xe-"
X f(x)
1.4 0.3452357
1.5 0.3346952
1.6 0.3230344
1.7 0.3105620
Using the first Three-point fommla
/'(1.5)"' l [- 3/(1.5)+4/(1.6) - /(1.7)]

2x0. l
l
= - [- 3 x 0.3346952 + 4 x 0.3230344 - 0..3105620]
0.2
=-0.J 125500
Using the second Three-point formula

1
f'(x0 ),,. - {J(x0 +Ii) - f(x0 - h)}
2h
/'(1.5)"' l [ /(1.6) - /(1.4)]

2x0.l
1
= - (0.3230344 - 0.3452357]
0.2
= -0.1110066
The exact value of f'(L5) is: - 0.1115651 •

Exercise:
Fillld the approximate value of /'(0.9) where f(x)=sinx. Use '1=0.001 and both Three-point for-
mulas. Repeat the problem for Ii= 0.002.
6.2 Alternate Approach and Error Estimates

We have noticed that all values of the first derivatives resulting from the forward (and backward)
difference and two Three-point formulas are approximate values. The question is: bow far away are
these approximations from their lrue values? To appreciate the level of error that we introduce each
time we use one of these formula~. we will utilize Taylor's series expansion and derive the differen-
tiation fom1ulas. Take Taylor's series expansion off (x + h) about x.
3
,
li2 n21 h nll h • 1•1
f(x+h) = J(x)+ hf (x)+- 1 (x)+ - 1 (x)+ - J (x)+··· (6.6)
2 3! 4!
Equation (6.6) can be rewritten as

h2 13 1,•
f (x+h) - f (x) = lif'(x)+ - f' 2>(x)+ -
2 3!
1
f 3l (x)+-4!· Jl'l (x)+·· · (6.7)
Divide both sides of Equation (6. 7) by It
3
f(x +h)- f(x) J'( X ) + -1z (2J( X ) + -/,2 /(ll( X ) + -/1 Jl"l( X ) + ·· · (6.8)
h 2
1 3! 4!
f(x +h)- J(x) f'(x) +O (li)

h
, f(x +h)- J (x) O(I)

f ( X) = ~--'---'--'--'- I
h
f'(x)"' J(x +h)- f(x)

h
Therefore. each time we use the forward difference formula we introduce an error of O (h ). Since
the step size h is small, we can say that the error of the forward difference approximation of the first
de,rivative is proportional to the step size h.
Let us derive another approximation formula for the first derivative by tal<ing Taylor's series expan-
sion of f(x+2h) about x.
4
f(x+2h)=J(x) +2hf'(x)+ I? Jt21 (x)+ g1,J J(l► (x)+ 16"' f'l(x)+ ···
2 3! 4!
4/,2 2 8h3 31 l6h4

f(x+2h) - f (x)=21if'(x)+ - f >(x)+ - / (x)+- /'>(x)+· ··
2 3! 4!
I (x+ 2h)- I (x) f'( X ) + -2h 1111 ( X ) + -4/i2 Jn3>( X ) + -8/i3 Jc•> (X ) +··· (6.9)
2h 2 3! 4!
SL1btract Equation (6.9) from two times of Equation (6.8).

<@> Chapter 6: Numerical Differentiation and Integration
3
- f(x +2h)+4f(.x+h) - 3f(x) f(x) - 21i2 Jf3l(x)- 611 j<•l (x) -· ··
21! 3! 4!
- f (x +2h)+4f (.x+h)- 3/ (x) f (x)+0(ii2)

2/r
0(h2)=- 21/ f l>(x)- 61/ /•l(x) - ···

3! 4!
J' (x).,. _-f_ (~x_+ _2 h~)_+ 4_/~(_x +_l~r) _-_

3f_(~x )
(6. IO)
2/r
Equation (6. IO) is lhe first Three-point formula and the error introduced by this approximation of the
first derivative is 0(112 ) which is proportional to the square of the step size h.
lo order to derive the second Three-point formula, take Taylor's series expansion off (x - h)
about x.
, 3
, J,- (2) /r (3)
f(x - h)=f(x) - 1,f (x)+ - f (x) - - f (x)+ ··· (6.11)
2 3!
Subtract Equation (6.11) from Equation (6.6).
2h 3 < 21,6
f (x+h) - f(x - '1) =2hf'(x)+- f 31(x)+ - J<•i (x) + ··· (6.12)
3! 6!
Divide Equation (6. 12) by 2h.
(6.13)
2 5
f'(x)= f (x+h) - J(x - h) h / l l (x) - h / 6l (x) - ...
2/, 3! 6!
f(x+h) - J(x - h) + 0(ll 2 )

f ' (x) = -'--'---'---'---'----'- (6.14)
2/r
2 5
where0 ( h 2) = -h-j •(ll (x) -h-f !6} (x) - ···
31 6!
f'(x)"' .f(x+h) - f(x - h) (6. 15)

2h
Equation (6. 15) is the second Three-point formula and the error introduced by this approximation of
the first derivative is also 0(li2) which is proportional to the square of the step size h. This time the
dominant error term is, however, half of that for the first Three-point formula.
6.3 Second-order Derivatives

We can find an approximation of the second-order derivative by utilizing Taylor series expansion of
f (x +h) about x in the following manner.
h1 J,3
>(x) - -f 31 (x)+ ·· ·
1
f (x - h)= f (x) - hf'(x)+- f
2 3!
Add these two expressions.
2 4
f(x +h)+ f(x - h)=2J(x)+-2h J <'-> (x)+ -2h J <•l (x)+···
2 4!
2/i2 <'> 21, • <•)
f( x+h) - 2/(x)+ f(x - h)=- f - (x) +- f (x)+···
2 4!
f(x+h) - 2/(x)+ J(x - h) ,11 2h 2 !(') \
h2 1 (x)+4! (xi' +···
J(x +h) - 2f (x) + J(x - h) 21,2 f'l (x) +· ··

!'21 (x)
h1 4!
f(x+h) - 2/(x)+ f(x - h )
!'21 (x) li2 f"l (~) (6. 16)
1i2 12
for~, where x 0 - h <~ <x0 +h.
(6. 17)
Example6.2
Find an approximate value of /<2\ 1.5) for f (x) = x[' with h =0.1.
Solutio11
In order to solve this problem we will utilize Table 6.2 prepared for Example 6.1 .
Using the formula g iven by Equation (6.17)
/ 21 (x)"' f (x +h) - 2~~x)+ J(x - h)
Jl21 (I. )"'

5
I (1.6) - 2/ (i.5) + J (1.4)
(0. 1)2
1'2) (l.5),. 0.3230344- 2(0.3346952) + 0.3452357

(0.1}2
2
/ >(1.5) == - 0.1120302
The exact value of f 2

l (1.5) is: - 0.11 I5651.
•
Problem Set 6.1
1. Find the approximate values of f'(l. 2) and J'2>(1.3) given that
f (I. I)= 8.525013
f (1.2) = 10.52318
/(1.3) = 12.96374
/(1.4)= 15.94465
2
2. Find the approximate values of /'(2.1) and / >( 2.2) given that
f (2.0) = 4.6897983
f (2.1) = 4.6915701
f (2.2) = 4.6698 I92
f (2.3) = 4.6255909
3. Find the approximate values of f'(-2.6) and J'11(- 2.6) given that
/(- 3.0) =7.367979

f (- 2.8) = 6.233341
/(- 2.6)=5. 180450
/(- 2.4) =4.209429
/(- 2.2)=3.320405
6.4 Numerical Integration

We know that the integration of a fu.nction/(x) over some interval [a, b) is the area under the curve
.f(x) between x = a to x =b. However, in many cases a mathematical expression forf(x) is unknown
and in some cases even if f(x) is known its complex form makes it difficult to perform the integra-
tion. In those cases we resort to nwnerical integration.
The general form of 11umerical integration off (x) over [a, b] is a weighted sum of the function
values at a finite number of sample points, known as quadrature, andl is:
b N
J f(x)dx =I, w.f(x,) (6. 18)
" k=-0
where a=x0 <x1<x2··· <xN = b.

The sample points are equally spaced for the midpoint rule, the trapezoidal rule and Simpson' s rule.
6.4.1 Trapezoidal rule

In Trapezoidal rule, the curve J(x) is approximated with a straight l.ine segment connecting points
/(a) and f(b). Therefore, finding the area J: f(x)dx becomes the problem of finding the area of the
trapezoid as shown in Figure 6.1.
y
/(b)
/(a) ------------
a b X
Figure 6. 1 Trapezoidal Rule
We would, therefore, calculate the area of the trapezoid whose venices are (a, 0), (b, 0), ( b, f(b) )
and ( a, /(a)). The length of the two parallel sides are /(a) and J(b). The height (perpendicular
distance between the two parallel sides) trapezoid is (b - a).
b b- a
J" f(x)dx., - 2
[f(a)+ / (b)]
b h
J" f(x)dx., - [f(a)+ f(b)]
2
(6.19)
<$> Chapter 6: Numerical Differentiation and Integration
whcrch=b - a
It js obvious from Figure 6.1 that if f ( x) is linear then die Trapezoidal rule would produce no error.
However, for f ( x) second degree or higher there wi U be error produced by the Trapezoidal rule.
Example6.3
Find the approximate value of r x ~dx using Trapezoidal rule and calculate the absolute error.
Sallltio11
Using Equation (6.18)
b h
J• f(x)dx .. -2 [f(a)+ /(b)]
, 2- 1
J- f(x) "' - 2 [f(I)+ f(2)]
I
,-x• "' _·2 -- 1[1+16)=8.5

J 2
I
The exact value of die integral is r x• dx = 6.2

The absolute error is 2.3. •
6.4.2 Simpson's rule
In deriving Simpson's rule we integrate the Lagrange interpolating polynomial of degree two that
approximates the function. Given the interval [a, b] we i11troduce a midpoint x 1 such that x 0 = a,
x, =a+h and x 2 = b where h =(b- a) 12.
The Lagrange interpolating polynomial of degree two is
P (x) = (x - x,)(x - x1) f(xo) + (x - xo)(x - x2) f(x,)

(x0 - x 1)(x0 - x 2 ) (x, - .:c0 )(x, - x2 )
+ (x - x 0 )(x - x 1) f(x.,)
(.:C2 - Xo)(X~ - .:c,)
f•• f(x)dx"' f'' P (.:c)dx

.\o .l'G
(6.20)
Example6.4
Find the approximate value of J,2 x 4 dx using Simpson's rule and calculate the absolute error.
Solutio11
Using Equation (6.20)
r: f(x)<ll'. "'~[f (xo) + 4/(x,) + f(x2)]
h = (b - a) 12 =(2 - 1)/2 =l/2
x 0 =a= 1, x, =a+h = 1+ lf2= 1.5 and x 1 = b= 2
• l
J- f(x) dx "' -6 [/(1)+4/(l.5)+ /(2)]
I
J-f(x) dx "' -6 [l + 4(5.0625)+ 16)]= 6208333

• l
I
The exact value of the integral is J x • dx = 6.2

I
2
The absolute error is 0.008333. •

6.5 Error Analysis
We noticed from Examples 6.3 and 6.4 that the absolute error of Trapezoidal rule is quite significant
and on the other hand the absolute error of Simpson's rule in comparison is very smaU. Can we find
an analytical basis of the difference in errors between these two rules? The answer is yes, by utiliz-
ing error in Lagrange interpolating polynomial.
6.5.1 Error of the Trapezoidal rule

A function f (x) can be expressed as a combination of the linear Lagrange interpolating polynomial
and its corresponding error term as
<@> Chapter 6: Numerical Differentiat ion and Integration
where
and~ is an unknown quantity between the minimum and the maximum of x0 and x, .
The linear Lagrange interpolating polynomial can be used to derive the Trapezoidal rule.
The truncation error is
J: f(x)dx"' J: P, (x)dx
b
J• f(x)dx,,, _h
J 1•{- (x - b)f(a)+(x - a)f(b)}dx
•
f• f(x)dx"' -hl[
b (x - b)2 f(a)+ -'-----'-
-'------'-
2
2
(x - a) f(b)
2
I
, . f(x)fil' "' -h {/(a)+ J(b )}. the formula for Trapezoidal rule with the truncation error
f• 2
Using the fact that (x - x0 )(x - x,) is always positive in [a, b], we can use the Weighted Mean Value
Theorem of integration.
~(x) indicates that it depends on ..rand by applying the Weighted Mean Value Theorem it can be
replaced by 17 s uch that 17 is in (a, b).
Using integration by paits, r{ (x - Xo)(x - x,)} (il' = - 1~

J,3
The error term of the Trapezoidal rule, therefore, is - -/"(T/). The error term indicates that the
12
Trapezoidal rule provides accurate results for linear functions.
6.S.2 Error of Simpson'srule

To derive Simpson's rule, we use Lagrange interpolating polynomial of degree two.
where
s
and (x) is an unknown quantity between the minimum and the maximum of x 0 • x, and x 2 that
depends on x.
The truncation error is (x - xo)(x - x,)(x - x 2 ) /(3) ( ; (x)).
3!
Therefore, the error of Simpson's rule is
We cannot apply the Weighted Mean Value Theorem to the error term directly since
(x - x0 )(x - x 1)(x - xi) changes sign in (x0 ,x2 ).
Expand / about x 1 using Taylor's polynomial of third degree.
2
(x - r) (x - x) 3
J(x)=f(xi)+ (x - x,)/'l(x,)+ ~ , /(2l(x,)+ 6, j<J>(x,)
+ (x - x,)' J'·1(s(x))
24
f~..,, J(x)dx= f'' [f(x,)+(x- x,)/ 11 (x,)+ (x - x,}1 f 21 (x,)+ (x - x1,}3 / 3>(x,)]dx
,. 2 6
+f''[(x- x,)" f•> (s (x)) ]dx

,. 24
Jx.''J( ·)d
X X -
1
(x - x,}2 jA•l(,}
- [ X /( X } +-'----'-'-
2
(x - x,}3 JA1l( X 1 }+-'----'-'-
· ,l 1 +-'----'-'-
6
{x- x,}4 JAll( X 1
24
)I'
'•
+ f'[(x - x,)' /•>(s(x))]d.x
'• 24
J,.,,f (x)dt = (x x f (x, }+ {(x -2x, )

2
-
0
) 2
2
(xo - ..-,)2
2
}f•l (x,)
+[(x2 ~x,t (xo ~x, t]jl2l(x,}+[ (x2;;•r (xo;;,r ]f3l(x,}
+J.," [(x-x,)' f"'(s(x))]dx

'• 24
Using Equation (6.16) to replace f 2>(x,)
_ 2 /( ) /,3 [f(x,+h) - 2f(x,)+f(x, - !t)

f"f(·)d
x.
X X- Ji x, +
3
2
h
+r•[(x- x,)' f•>(s(x))] dx

.. 24
Since (x - x,)' in the last term is always positive. we can apply the Weighted Mean Value Theorem
of Integral to the last tem1.
Since we are looking for the absolute maximum error, we choose a common number between .;1 and
<;2 that will give the absolute maximum of / 41 (x) in (x0 ,x1 ).
The error term inclicales that Simpson's rule provides accurate results for functions of third degree
or lower.
Problem Set 6.2

1. Approximate the following integrals using Trapezoidal Rule. Determine the error in each
case.
a. f x dx
o.s
3
b. J'.s x
1
2 En(x)dx
C.
J'.s - ?2x
I
-
[ - 3
d d. t e 2 ' s.in{3x )dx
2. Repeat (! ) using SLmpson's Rule. Deterrn.ine the error in each case.

3. Given
/( I.8)=3.12014
f (2.0) = 4.42569
f (2.2) = 6.04241
/(2.4) =8.03014
f (2.6) = I0.46675
find the approx.irnate values off 22 f (x)dx and

1..8
J J(x)dx.
2
•
2.0
6.6 Composite Numerical Integration

In order to reduce the error, we can divide the .interval into multiple subintervals and apply the
Trapezoidal Rule or Simpson's Rule on these subintervals.
6.<6.1 Composite Trapezoidal rule

Divide the interval into n subintervals and apply Lhe Trapezoidal Ruic in each subinterval Composite
Trapezoidal Rule for II subintervals can be wrinen as:
b- a
wlaereh= - - andx,=a +klt fork=0, I, 2, ... ,11
n
Example6.S
Find J:sin(x)dx by dividing the interval into 20 subintervals.

= 20
ll
b - a 1t
h= - - =-
n 20
Im
x,=a+kh= , k=0, I, 2, ....., 20
20
J.• sin(x)<fr=- -2h [ f(a)+2 L, f(x,)+ f(b)

0
k=I
-1 ]
= .2:..[sin(
40
1
0)+ 2I,sin(/m) + sin(n)]
t=I 20 •
= J.995886
6.i6.2 Composite Simpson's rule

Divide the interval into II subintervals and apply Simpson's Rule on each consecutive pair of subin-
tervals. Note that n must be even.
b- a
whereh= - - andx,=a +kh fork=0, I, 2, ... ,n
n
Example6.6
Find Jo" sin (r)dx by dividing the interval into 20 subintervals.
b- a 1t
n.=20 h= - - = -
11 20
Jo" sin(x)dx-,. _2:_[sin(O)+ 2I

60 k=l
sin(
2
20
'm)
+4fk=I
2
sin(C k - l)1t)+ sin(n)]
20
= 2.000006 •
Problem Set 6.3
1. Use Composite Trapezoidal Rule to approximate the following integrals.
2 3
a.
f. -x-, -+4 dx,
0
n. = 6
r2 2x
c. Joe cos(3x )dx, 11 =&
2. Repeat (I) using Composite Simpson's Rule.
6.7 Gaussian Integration

In Gaussian integration, we are loolking for a way to find the integration of polynomials of as large
a degree as possible diat would be exact. The exercise becomes finding lhe weights and nodes that
would make the result of d1e numerical integration exact. We consider the following integration.
f J(x) dx
I "
J f(x)dt = L, w, f(x,)
-I l=I
where
w1 is d1e ith weight and x1 is the ith node.
Consider the case of 11 = I.
With 11 = l, we have two wilcnowns, 1111 and x 1 that can be determined by restricting the integration to
be exact for two functions,
f(.r)=l and f(x)=x.
For f (x) = I, The exact integration is 2 which forces the weight iv, to be 2. For f (x) = x the exact
integration is
J, xdx=O
-I
This implies that
w,x,=0 (6.21)
To satisfy Equation (6.2 1), x, should be 0, because 1111 is already determined to be 2. Hence for
f(x)= l wehave
f ,xdx = w, /(0)= 2/(0) (6.22)
For J(x)= x 2 , however, we need 2 nodes and corresponding two weights.

Consider the case of 11 = 2.
(6.23)
There are four unknowns x,, x 2 , w, and 1112 in Equation (6.23). To determine the unknowns, we
require Equation (6.23) to be exact for the following four monomials,
Therefore,
w, +w2 =2
W 1X 1 + W 2X 2 = 0
(6.24)
The set of Equations (6.24) is nonlinear and its solution is:

Therefore, the following integration fommla
will give exact values for all polynomials of degree equal or less tham 3.
Now, consider the case of n = 3.
(6.25)
There are six unknowns x 1• x 2 , x 3 , w" w 2 and w 3 in Equation (6.25). To determine the unknowns, we
require Equation (6.25) to be exact for the following six monomials,
2 3 45
/)(.x =J ~x,x , .t , x ,x.
Therefore,
+ W~ + IV3 = 2
1111
IV1X 1 + w 2 x 2 + w3 x 3 =0
W1Xf =t
+ W1Xi +w~x:
(6.26)
w x;+ w x;+ w xJ= 0
1 2 3
-I 4 4 _ 2
wlxl + IV1 X2 + W y-X3 -,
IVIXI
5
+ W 2X 15 + W 3 X35 = 0
The set of Equations (6.26) is nonlinear and its solution is:
IV1 = 0.5555555556, W1 = 0.5555555556, W 3 = 0.8888888889,
x1 = 0.7745966692, x 2 =--0.7745966692 and x3 = 0
The unknowns are already available for most commonly used value of n. Table 6.3 shows d1e nodes
and weights for 11 = 2, 3, · · ·, 8.
The nodes can also be determined using Legendre polynomials as they are the zeroes of the Legendre
polynomials on tbe interval [- 1, I] . Legendre polynomials are defined as:
P0 (x)=l
P (x)= -I · d"-{( x-• - 1)"} , n=l , 2, ·· ·

" 11!2" dxn
<€,> Chapter 6: Numerical Differentiation and Integration
Table 6.3 Nodes and weights of Gaussian integration formulas
II x, w,
2 ± 0.577350269 1896257 1.0
3 ± 0.77459666924 14834 0.5555555555555556
0.0 0.8888888888888888
4 ± 0.86113631 I 5940526 0.3478548451374538
± 0.3399810435848563 0.6521451548625461
5 ± 0. 9061798459386640 0.236926885056189]
± 0.5384693101056831 0.4786286704993665
0.0 0.5688888888888889
6 ± 0.9324695142031521 0.1713244923791704
± 0.6612093864662645 0.3607615730481386
± 0.2386191860831969 0.4679139345726910
7 ± 0.9491079123427585 0.129484966 I 688697
± 0.7415311855993945 0.2797053914892766
± 0.405845 1513773972 0.38) 830050505 ll 89
0.0 0.4 l.79591816734694
8 ± 0.9602898564975363 0. 1012285362903763
± 0.7966664774136267 0.22238 10344533745
± 0.5255324099 163290 0.3137066458778873
± 0.1834346424956498 0.362683783'3783620
For example,
P, (x)=x
Pi (x)=½{3x 2 - 1)
~(x)=½{5x 3 - 3_t)
P. (x)=½(35x• - 30x 2 +3)
P; (x) = t(63x 5 - 70x 3 + 15x)
In most cases integrations would be sought over [a, b) instead of [- 1. 1). In such cases, we can covert
the integration by substituting a new variable and then apply the formula for Gaussian integration.
Consider the following integration.
I= f f(x)dx
Replace x with a new variable, r.
-(b-a) (b+a)
X- - - I+ - -
2 2
I= (-b-a)f'
- _,f(r)dr
2
Example6.7
Determine the following integral using Gaussian integration formula.
Solutio11
The function we are integrating is
f(x)=l+2x - 1.7x2
Replace x with -(b-a)

2
- + (b+a)
- -
1
2
wherc a=2 and b=4
f (1)= 7+ 21 -1.7 (1+3)1
I= f,J(t}d1
I= w,f (1,) + w 2 f (1 2)
Since /(1) is a quadratic polynomial, we can use n =2 for exact resllllt.
I= 1(-l} /( ~ )=- 17.7333333333
which is also the exact value of /.

•
The function used in Example 6.7 is a straightforward one that could be integrated using the stand-
ard integration rule instead of Gaussian integration rule. It would be worthwhile to examine the use
of Gaussian integration rule to integrate more complex functions. We consider the following four
integrals.
11 =
f2vxrI dx, f ,- = J2 -1+x
1 1
·I - dx,1 = i2
3 -I- dxand f ,= J2e-' dx
1+x
1 1
1
Io all four cases, the variable x has to be replaced in order to apply Gaussian integration rule. Also,
none of the four functions are polynomial, therefore, we expect some error. In all four cases, a= I
and b=2.
First we use 11 = 2
J'
I,= -2I _, f(t)dt
J'i
where/(1)= ,.--:--;;
vt + 3
The exact value of / 1 is 0.828427 1247 producing a relative absolute error of 0.03%.
? I
- J
I,= - - dx
' I +x
r+3
x= - -
2
where/(t)=--2_
t+5
The exact value of 12 is 0.405465108 ! producing a relative absolute error of0.015%.
r+3
x= -
2
4
13 = -If 1 f(t) dt where/ (1)
2 -1 4+(r+3)1
The exact value of/3 is 0.3217505544 producing a relative absolute error of 0.01 I%.
r+3
x= -
2
,.3
14 =.!.f
2
I
-I
f(r)dt whcrc/(r)=e 2
The exact value of/4 is 0.2325441579 producing a relative absolute error of 0.022%.
Let us now use 11 = 3.
11 = -I
2
f I
f(t)dr
✓2
where/(1)= r:-;-;,
- 1 ,JI +3
W1 =0.5555555556, 1112 =0.5555555556, IV3 =0.8888888889.
x 1 =0.7745966692, x 2 =-0.7745966692 and x 3 =O

/ 1
j
0.5555555556/ (0.7745966692)
=½ +0.5555555556! (--0.7745966692)
l
+0.8888888889 f (0)
/
1
=0.8284202635
The exact value of / 1 is 0.828427 1247 producing a relative absolute error of 0.00083%.
2 l
l, = -
- ' l+x f
dx
1+3
x= -
2
/ 2 = -If'_,f(t)dr where/(1)= - 2
2 1+5
l
l
11 = {wJ (x, )+ w 2.f (x1 )+ w 3/ (x,)}
2
0.555555SS56f (0.7745966692)
11
j
=½+0.5555555556/(--0.7745966692)
+0.8888888889J (0)
/2 = 0.4054644809
The exact value of / 1 is 0.405465108 ! producing a relative absolute error of0.00015%.
2 I
13 = 1~l+x- x
l
1+3
~t= -
2
4
1. = ..!..J ' f( r)dr where/(1)
, 2 _, 4+(t+3}2
l
I
/3 =
2{wJ(x,)+ w1 f (x2 )+ w3J(.,J}
0.5555555556! (0.7745966692)
l3 = ½+0.5555555556/ (--0.7745966692)
! +0.8888888889 f (0)
13 = 0.3217559457
The exact value of / 3 is 0.3217505544 producing a relative absolute error of0.00167%.
r+3
x= -
2
,.3
1. =..!.f' f(r)dr where f(r)=e-1
2 -I
l
I
I. =
2{wJ (x, )+ 1V2f (Xi) + w3 f (x3 )}
!
0.5555555556/ (0.7745966692)
1. =½ +0.5555555556/ (-0.7745966692)
+0.8888888889/ (0)
1. = 0.2325440464
The exact value of/4 is 0.2325441579 producing a relative absolute error of 0.00005%.
Problem Set 6.4

1. Evaluate the following imegrals using Gaussian integration rule with 11 = 2.
l.5
b.
f
1
x 2 Ln(x)dx
c. f
I
15
_l:!_dx
X2 - 3
2. Repeal I. with n=3.

_J
ER7
INITil1 L-VAt R
-- ---- D- ~LJlY DIH AL
- . - - - - - - + - - - - - + ~ - - - . - - - - - - - + - - - I - - -~ ~ rfB
Variables in many engineering problems are related in the fom1 of differential equations. Properties
of materials, dynamic behavior of devices and current and fluid flow can be described by differential
equations. In many applications, general solutions of differential equations are sought to understand
lhe overall characteristics of a device or a process. But in a vast majority of cases number specific
solutions are desired for design and stability purposes. In this chapter, we will introduce several
nwnerical techniques to solve first-order differential equations. We will specifically consider the
equations whose initial values are given and as such they are also called initial value problems.
Most differential equations applicable to engineering problems have unique solutions. We can start
with a generalized solution for such a problem and force that 10 satisfy the given initial condition.
However, it is still worthwhile to address the basics related to the solution of differential equations.
7.1 Some Basics of Differential Equations

Consider the following differential equation.
y'(x)=r(x)
Botil the left hand and the right hand side are explici t functions of the independent variable, x.
Therefore, we can integrate both sides with the following result.
y(x) = f r(x)dx+c
wlhere c is an arbitrary constant of integration and f r(x) dx is any fixed antiderivative of r.
A particular solution for the above differential equation can be obtained by detemiining c for a
specific value of y(x) at some given point, y(x0 ) = y0 •
<0> Chapter 7: Initial-Value Problems for Ordinary Differential Equations
However, if we have the following rufferential equation

y'(x)=r(x,y)
where the right hand side is a function of both the independent and dependent variables, we just can-
not integrate both sides to obtain a solution. In Lb.is case, we have to solve the differential equation
using numerical techniques.
Regardless of the right hand side, it is sometimes more efficient to solve a differential equation with
an initial value using numerical techniques.
Before we proceed, we would like to address what makes an ordinary differcmial equation solvable
and is the solution s table? An ordinary differential equation is well-posed if it has a unique solution
and small changes to the equation only imply small changes to the solution.
Definition 7.1
A function/ (x,y) is said to satisfy a Lipschitz condition in the variable yon a set D c R 2 if a con-
stant K >0 exists with If (x, y, )- f (x, y )I~K IY,- y
2 2
1 whenever (x, y,) and (x, y2 ) are in D . The
constant K is called a Lipschitz constant for/. •
The relationship II (x , y
1) - f ( x, y 2 )I~KIy, - _v Ican be viewed as.
2
IJ(x, Y, )- .f (x, y2 )I< K

IY, - Y2I
which implies that if/ (x,y) is a Lipschitz function then aJ(x, y) etists and bounded on D.
ay
Theorem 7.2
Let / (x,y) is continuous on D c IR2 where D = {(x, y)la:,; x $; b and - oo < y <=}.If J satisfies a
Lipschitz condition on D in the variable y, then the initial value problem
y'(x) = f(x,y), a~x$;b, y(a) =r
bas a unique solution y(x) for a:,; x:,; b.
Theorem 7.3
Let D = { (x, y)la:,; x:,; b and - 00 < y < 00}. If/ is continuous and satisfies a Lipschitz condition on
D in the variable y, then the initial value problem
y'(x) = f(x,y), a~x$;b, y(a)=r
is well posed.
Chapter 7: Initial-Value Problems for Ordinary Differential Equations ~
7.2 Forward Euler'sMethod

In Forward Euler's method, the slope at a current point is used to calculate the next point along the
direction of an independent variable. Al each stage, the independent variable is increased by a dis-
crete step. The process continues in a successive manner until the independent variable reaches a
specified point. Fundamentall y, Euler's method is very straightforward to implement. However, like
many other numerical techniques, its accuracy depends on the s tep size of the independent variable.
Figure 7.1 illustrates the Forward Euler's method. T he slope at the initial point, (x0 • y(x0 )) is given.
=
The slope line when extended to x x, intersects with the vertical line, y = x, at a point (x,, y, ). y, is,
therefore. an approximated value of the function y at x = x,.
y
y(xo)
y(xo+h)
Yi
X
Figure 7. 1 Forward Euler's method
The process is repeated by estimating the slope at x = x, and extending the slope line to x = Xr The
slope line will intersect with the vertical line, y = x 2 at ( x 2 , y2 ). y2 is, tiberefore, an approximate value
of the function y at x = x r
Slope at (x0 , Yo) is y'(x0 ). The slope can be written as
y'(xo)"' y(x,) - y(x0 )

x, - xo
'( ) y(x1 ) - y(x0 )

or, y x0 ""
h
where h = x , - x 0, is called the step size. Perfom1 cross multiplication and rearrange the terms.
(7.1)
The slope y' is usually expressed as an implicit function of x and y i.e., y' = f(x,y). Therefore,
Equation (7.1) can be written as
(7.2)
Considering Y,, is an approximation toy( x.). Equation (7.2) can be written in the form of a sequence
equation in the following manner.
Y.+1= Y,. +lif(x,., Y.) (7.3)
wlnereh=x••,- x.
The sequence shown in Equation (7.3) is the implementation of Forward Euler's method. We will
ex.amine the implementation of Backward Euler's method later.
Example 7.1
Consider the initial value problem y' =2+ y - 2x, 0~x~I . y(0)=l. Solve for y using Forward
Euler's method. Use a step size of 0.1.
Solution
y' =f(x, y) =2+ y - 2x and h=0.1
x 0 =0, Yo = land therefore, f (x0 , Yo) = 2+ y0 - 2x0 =3.000000
Y, = Yo +lif (x0 , y0 ) = I +0. 1(3.00000) = 1.300000
x, =x0 +h = 0+0.1 =0.l f (x,, Y,) = 2+ y1 - 2x, = 3..JOO0OO
Y2 = Y, +hf (x,, y,) = 1.300000+0. ! (3.100000) = l.6l0000
x2 =x0 + 2h=0+ 2x0. l = 0.2 f (x2 , yJ =2+ y2 - 2.ti =3.210000

Y3 = Yi +!if (x 2 , Y2) = l.6 10000+0.1(3.210000) = 1.931000
x_,= x0 + 3/t = 0+ Jx 0.J = 0.3 f (x3, y3 ) = 2+ )'3 - 2x3 =3.331000
Y4 = y3 +hf(x,, y3 ) =1.93!000+0.1(3.331000) =2.264I00
Continue until you determine y 10 •

The exact sol ution to the initial value problem is
y(x) =2x+e'. The results with h = 0.1 and corresponding errors are s hown in Tables 7.1 and 7.2.
The IVP is also solved with h = 0.05. Table 7.2 shows the results and the absolute errors.
Compare the absolute error columns in Tables 7.l and 7.2. It can be noticed that the en·ors are
reduced when the step s ize is reduced to half. However, we needed twice as many steps as we needed
when the step size was 0. I. ■
Table 7.1 The results of the example IVP with /1 = 0.1
11 x. Y. Exact Solution Absolute E rror

I 0.1 1.300000 1.305171 0.005171
2 0.2 1.610000 1.621403 0.011403
3 0.3 1.931000 1.949859 0.018859
4 0.4 2.264100 2291825 0.027725
5 0.5 2.610510 2.648721 0.038211
6 0.6 2.971561 3.022119 0.050558
7 0.7 3.348717 3.413753 0.065036
8 0.8 3.743589 3.825541 0.081952
9 0.9 4. 157948 4.259603 0 ..101655
10 1.0 4.593742 4.718282 0.124539
Table 7 .2 The results of the example IVP with h = 0.05
n XH Y. Exact Solution Absolute Error

2 0.1 1.302500 1.305171 0.002671
4 0.2 1.615506 1.621403 0.005897
6 0.3 1.940096 1.949859 0.009763
8 0.4 2.277455 2.291825 0.014369
IO 0.5 2.628895 2.648721 0.019827
12 0.6 2.995856 3.022119 0.026262
14 0.7 3.379932 3.413753 0.033821
16 0.8 3.782875 3.825541 0.042666
18 0.9 4.206619 4.259603 0.052984
20 1.0 4.653298 4.718282 0.064984
7.2.1 Error and convergence of forward Euler's method

It can he noticed from Figure 7.1 that at every step of the Forward Euler's method. the value of the
function is approximated with an error. To appreciate this let us expand Yin the neighborhood of
x =x. using Tay!or's series.
2
Y (x. +Ii)= Y.+1 = Y (x,.)+ liY'(x,.)+ h Y' (s.) (7.4)
2
where x. ~t ~x. +h
Y,,..1 is approximated using Forward Euler's method as y,,..1 "'Y(x,.)+hY'(x,,). This is a truncated
version of the Taylor's series shawm in Equation (7.4). Therefore, at every step of Euler's method, a
local truncation error is introduced as
A local truncation error is defined as the error in the presem step if tbere were no error in the previ-
ous step.
In Forward Euler's method we compute y,,..1 as
Yn+I = Y,. +lif(x.,y.)
where J(x.,y.) is the slope at (x,., Y. ).

The error of Forward Euler's method, therefore, can be expressed as
If Y (x) represents the true solution of the differential equation theu.
Y'(x.) = f (x., Y ( x. ))
Y (x. + h) - y_. 1 = Y (x. )- Y. + hf (x., Y (x.)) - l1f(x., Y. )+ /~ Y'(;.)
The propagated error is
Y(x.) - Y. +hf (x.,Y (x,,)) - hf (x., Y.)
Considering f (x.,/3) as a function of f3 we can apply the Mean Value theorem as
, J(x•. Y(x,.)) - f(x.,y.)

f(x.,17)= Y(x) -
" Y,J
where 77 is a point bct:\veen Y ( x,.) and Y,..

The propagated error tem1 can be written as
where£. =Y(.-1',,) - .Y.-

£,. is the global truncation error at step 11.
The combined error which is the global truncation error at step n + l lbecomes
As the process moves to the next step, the previous error is added to the current error. For Forward
Euler's method. the local truncation error is O ( h2 ). The method is referred as a first order technique.
A method with a truncation order of O ( ht+t } is called kth order. In general, for the same step size, a
method with higher order local truncation error provides lower local tnmcation error.
A numerical method for solving initial value problem is stable if a small perturbation in lhe initial
vaJue results in a small change in the solution.
A numerical method is convergent if the nwnerica!Jy computed solution approaches to the exact
so lution as the step size approaches to zero. A model problem shown. in (7.5) is often utilized to test
the convergence of a numerical method.
Y' =- aY, x> O, a>O Y(O)=l (7.5)
The exact solution of (7.5) is Y = exp (- ax) that satisfies the condition Y (0) = l and decays smoothly
to zero at x =oo. If the numerical method is stable then its solution should satisfy y(x.) ➔ 0 as
x. ➔ oo for any choice of the step size h.
The solution of the model proble m utilizing Forward Euler's method is
Y,,., =Y. +lt(- ay"), 11<::0, Yo =l
The discrete sequence equation of the solution can be arranged as
By induction
Y.., =(1 - ah/ Y•. ,
Y•• , =(1 - ahf Y•. 2
•+I
y_.1 =1
( - ah ) Yo (7.6)
Equation (7.6) implies that for the stability of Forward Euler's method,
I1- ahl <L

The stability condition can be further simplified as
ah(ah - 2}<0
i.e.,a h<2
The stability condition, therefore, implies that the step s.ize should be smaller than 2/a.
The Forward Euler's method is an explicit method since we compute y,,.1 based on die known com-
ponents y,, and f(x,,,y,,). Explicit methods are easy to implemem, they. however, have the limita-
tions on step size to maintain numerical stability. Euler's mediod converges widi order l. In general,
a numerical mediod is convergent with order p if it satisfies the following.
where k is a constant.
From die results shown in Tables 7. I and 7 .2 a general trend can be observed that in a first order
technique when the step size is halved the error is hal ved.
7.3 Backward Euler'sMethod

The slope in the form of backward difference equation can be written as
(7.7)
Equation (7.7) can be written in the form of a sequence as
(7.8)
Equation (7.8) is the required sequence expression for Backward Euler's method. Equation (7.8) dic-
tates that at each step of Backward Euler's method, a nonlinear algebraic equation has to be solved.
Let us apply the Backward Euler's sequence to the model problem shown in (7 .5).
Y• = (l )',,_,
+ah)'
n>0
- '
" =I
JO
By induction
Yn- l
Y. = l
(1 +a I,)
y = Yo (7.9)
" (l +ah)"
Equation (7.9) satisfies that y(x,,) ~ 0 as x. ➔ oo for any choice of the step size h. Hence, Bacl..-ward
Euler's method is convergent and stable for au h > 0.
Like Forward Euler's method, Backward Euler's method also has first order convergence property
making both methods unsuitable for many applications.
Example 7.2
Consider the initial value problem y' = 2+ y - 2x , 0 5 x 51, y (O) = 1. Solve for y using Bacl-.-ward
Euler's method with a step size of 0.J.
Solutio11
Y.+i = y. +h(2 + Y.+1 - 2x.+1 )
v. + 2h - 2h(x,, +h) y - 21, x

=· " • +2h
Yn+ I 1- !J 1- h
Y1 Yo - 21txo +2h J- 2x0.lx0+2x0. l =I.311111

J- 1, 1- 0.l
y, = y1 - 2hx1 + 2 h= 1.31111 l - 2x0. l x0.l + 2 xO.l=l.634568

· 1-/i 1- 0.J
Y = y1 - 2hx2 + 2 h = l.634568 - 2x0.Ix0.2 + 2 xO.l = 1.971742

3
1-h 1- 0.1
Continue until you determine y10•

The exact solution to the initial value problem is y(x) =2x+e' . The results with I, =0.1 and corre-
sponding errors are shown in Table 7 .3.
Tabl e 7 .3 The results of the IVP using Backward Euler's method with h = 0.1
11 x. Y. Exact Solution Absolute Error

1 0.1 1.311111 1.305 I71 0.005940
2 0.2 l.634568 1.621403 0.013165
3 0.3 1.971742 1.949859 0.021883
4 0.4 2.324158 2.291825 0.032333
5 0.5 2.693509 2.648721 0.044788
6 0.6 3.081676 3.022119 0.059558
7 0.7 3.490752 3.413753 0.076999
8 0.8 3.923057 3.825541 0.097516
9 0.9 4.381175 4.259603 0.121572
JO 1.0 4.867972 4.718282 0.149690
7.4 Trapezoidal and Modified Euler'sMethod (Heun'sMethod)

Consider the differential equation
Y'(x)=f(x,Y(x)) (7.10)
In order to solve Equation (7. 10), we can integrate both sides of the equation from x. to x..,.,.
Y(x..,)- Y(x,.)= r•··•
"•
f(x, Y(x))dx (7. ll )
Utilizing Trapezoidal rule to approximate the integration, we can evaluate the right hand side as
The solution of Equation (7.10) becomes
Y (x•• ,) "'Y (x.) +~{/(x., Y (x. )}+ f (x.,1 , Y (x.,, ))}

2
where h=x,1+ 1 - x,,.
The discrete solution sequence using Trapezoidal method becomes
Yn+t = Y,, +%{/ (x., Y,. )+ f (x,..,, y,,.,)} (7.12)
where 11 ~ 0 and Yo =Y (x0 )

Trapezoidal method has second-order accuracy and is absolutely stable. Equation (7 .12) is an implicit
equation, therefore, at each step, algebraic equation bas to be solved to determine Y.+i·
A simplification without degrading the error characteristics can be made by evaluating y., 1 in the
right hand side of Equation (7. 12) using Forward Euler's method as the predictor. The resulting
sequence equation is
h
Y.., = Y,, +- {f (x., Y.) + f(x.,,, Y. +hf(x,,, Y.))} (7. 13)
2
and is known as Modified Euler's method, also referred to as Heun 's method in many texbooks.
Modified Euler's method, however, is no longer absolutely stable.
Example 7.3
Consider the initial value problem y' = 2+ y - 2x, 0 ::;x::; I, y(0) = l. Determine y using modified
Euler's method with a step size of 0.1. Compare the results with exact solution y(x) = 2x +e".
Solutio11
y' ;;; J( x, y) aa 2+ y - 2x and It ;;; 0. 1
x0 = 0, Yo = I and therefore, / ( x0 , Yo} = 2 + Yo - 2x0 = 3.000000

Yi = Yo + hf (x0 , Yo ) = I+ 0. I (3.00000) = I .300000

f(x,,y,)=2 + y, - 2x, =3.100000
Y, =Yo+ ~{/ (x0 • Yo)+f (x,, Yi)}= I +.Q:.!.(3.000000+ 3.100000) = 1.305000

2 2
x, =x0 +h=0+0. l= 0. I
f (xi, Y,) "' 2 + Y1 - 2x1 "' 3.105000

h = Yi +lif(x,, Yi) = l.305000+0. 1(3.105000) = 1.615500
f(x2 ,y2 )=2+Yi - 2x2 =3.215500
y1 = y, +~{f (x" y,)+ f (x2 , yJ} = 1.305000+ O~I (3.l05000+3.215500) = 1.621025
f (xi, Yi) =2 + Yi - 2x2 =3.221025
YJ = y2 +hf(x 2 , y2 ) = l.62 1025+0.1(3.22L025)= J.943127
f (x3, J3) =2 + J3 - 2X3 =3.343128
Y3 = Y2 + ~{/ (x2 , Y2 ) + f (x3 , y3 )} = 1.621025 + O~l (3.221025+3.343 l28)= 1.949233

Table 7.4 shows the results obtained using modified Euler's method with h = 0. l.
Table 7 .4 The results of the example IVP using modified Euler's method with h = 0.1
n X Y. Exact Solution Absolute Error

•
l 0.1 1.305000 1.305171 0.000171
2 0.2 1.621025 1.621403 0.000378
3 0.3 1.949233 1.949859 0.000626
4 0.4 2.290902 2.291825 0.000923
5 0.5 2.647447 2.648721 0.001275
6 0.6 3.020429 3.022119 0.001690
7 0.7 3.4 11574 3.413753 0.002179
8 0.8 3.822789 3.825541 0.002752
9 0.9 4.256182 4.259603 0.003421
JO 1.0 4.71408 1 4.718282 0.00420[
•
<@> Chapter 7: Initial-Value Problems for Ordinary Differential Equations
7.5 Higher-Order Taylor'sMethods

If y(x), y'(x), y'(x) etc. arc continuous then we can write the following Taylor series.
y(x+ h)= y(x)+ R 0
y(x+h)= y(x)+hy'(x)+ R,
y(x+ h)= y(x)+hy'(x)+ :: y'(x)+ Ri
1 1
y(x+h)= y(x)+ hy'(x)+ ~ y'(x)+ ~; y• (x)+ R3
Ro, R1, Rz and R3 are called Taylor's remainders. If It is small then

y(x+ h)"' y(x)
y(x+ h)"' y(x)+ hy'(x)

1,2
y(x+ h)"' y(x)+ hy'(x)+- y'(x)
2!
112
y(x+ h)"' y(x)+ hv'(x)+ y' (x)+ ~ y-(x)
. 2! 3!
Example 7.4
Consider the initial value problem y' =2+ y - 2x, 0 ~ x ~ l, y(O) =l. Determine y using Taylor's
method of ordertwo wi th a step siz,e of 0.1. Compare the results with exact solution y(.x) = 2x+e' .
Solutio11
y'=2+ y - 2x
, t,2 I
)'!t+I = y,, + hY,, + - .v"

2
ti2
y,.., = y,, +h(2 + y,, - 2x.)+ (y,, - 2x,,)
2
y,,., =(I +h + I~ )Y. -2(h + I; )x,, + 2h
y,.. 1 = I. I0Sy. - 0.2Ix,, +0.2 (7.14)
The sequence (7. 14) can be used to estimate y in a successive manner. The results with /, = 0.1 are
shown in Table 7.5.
Table 7 ..5 Results of Taylor's meuhod of order two with h =0.1
n x. Y. Exact Solution Absolute Error

I 0.1 1305000 1.305171 0.000171
2 0.2 1.621025 1.621403 0.000378
3 0.3 1.949233 1.949859 0.000626
4 0.4 2.290902 2.291825 0.000923
5 0.5 2.647447 2.648721 0.001275
6 0.6 3.020429 3.02211 9 0.001690
7 0.7 3.411574 3.413753 0.002179
8 0.8 3.822789 3.825541 0.002752
9 0.9 4.256182 4.259603 0.003421
10 1.0 4.71408 I 4.718282 0.004201
•
7.6 Runge-Kutta Method of Order Four
Instead of calculating higher order derivatives. Runge-Kuna method uses a sampling of slopes
through an interval and takes a weighted average slope to calculate the solution with the same degree
of accuracy as can be obtained from techniques using higher order derivatives. By using fundamen-
tal theorem as shown in Figure 7.2 we can write:
dy=y'(x)dx (7.15)
Integrating both sides of Equation (7. 15) we get
Jdy = Jy'(x) dx (7.16)
I
I
I
I
I
I
I
I
I
I
__________ ..,--=:;;,,_ __, _.
dv_________
I
~
•
y (x. ) I
dx I
I
I
I
I
I
I
I
x,,
Figure 7 .2 y as a function of x
<€,> Chapter 7: Initial-Value Problems for Ordinary Differential Equations
Applying appropriate limilS to Equation (7 .16) we get
(7.17)
or, alternatively
f,.
.t,,+li
y(x. +h)- y(x.) -= y'(x)dx (7.18)
where x n+I = x • + h and /, is d1e step size.

Let us now concentrate on the right -hand side of Equation (7 . 18).
f x,+h
y'(x)dx""
f x,+h
P(x)dx (7.19)
x. ..r..
where P(x) is an approximation to the actual function, y' (x). This approximation is necessary
because y' (x) is a fw1ction of x and y and is unknown until we could calculate the value of y. As
mentioned before, in an IVP, y'(x)js usuaJJy given in the form of y'(x) = f (x,y).
P(.x) can be generated by utiJizing Lagrange Ime1polaring Poly110111ia/. Assume that the onJy infor-
mation we have about a function, g(x) is dlat it goes through three· points: (x0 ,g(x0 )),(x,,g(x,))
and ( x2 , g (x2 ) ). The mathematical expression for g (.x) is unknown to us. However, an approximation
to g ( x) can be generated as:
P(x) ""g(x)
P(x) = Lo (x) g(x,o)+ Li (x) g (x,)+ L1 (x)g(x2 )
P( X ) -- -(x-- ~
x,)(x
-~ - x ~g
1) ( . )
,t 0
(Xo - x, )(Xo - X2)
(x - x 0 )(x - x,) ( )
+ - g x,
(x, - x0 )(.x, - ,\'..i)
(X - ,t )(.K - .-\'
0 1 ) ( )
+---"---'-- g x,
(X2 - Xo}(.~2 - x,) -
If we need to differentiate or integrate the function, g (x) which is unknown. we can _perform differ-
entiation or integration on P(x) instead.
Using Simpson's integration rule,
fxo,, g(x)dx ""fxo,, P(x)dx
where x, = x0 + /{ and x2 = x0 + 2/r.'.

(7.20)
We can apply Simpson's Integration to Equation (7.19) with the following substitutions:
x0 =x.; x 2 =x. +hand therefore, h' =I,/ 2 and the midpoint, x 1 =x,. +% and g(x,.) = y'(x,.).
(7.21)
Compared to Euler's Fonuula, an average of six slopes is used in Equation (7.21) instead of just
one slope. Actually, the slope at the midpoint has a weight of 4. The slope at the midpoint can be
estimated in two ways.
y(x. +I,)- y(x,.) "'~[y'(x. )+ 2y'(x. +1) +2y'( x. + ~)+ y'(x,. + h)]
+
y(x. h) "'y(x. }+~[ y'(x. )+ 2y'(x. +%) +2y'( x. + %
)+y'(x. +h)]
+ +i)+2y'( +i)+ y'(x.+,)]
y( x.+,)"' y(x,, )+ i[y'(x.) 2y'( x. .t ,,
(7 .22)
The slopes can be estimated in the following manner:
'( h)
·2 - "2"2
( h h)
y X +- ;k_,; f X + - ,)' +- k1
<$> Chapter 7: Initial-Value Problems for Ordinary Differential Equations
l(x,, +&)=k1 =t ( x. +%,Y. +&k2)

.v'(x. + h)= k4 = J(x,, + h,y. +hk3 )
Substituting the slopes into Equation (7.22) gives us the formula for Runge-Kutta Method of Fourth
Order:
(7.23)
Example 7.S
Consider the initial value problem y' = 2 + y- 2x, 0 $ x $ I, y(O) = l. Solve for y using Runge-Kutta
method of order four with a step size of 0.1. Compare the results with exact solution y(x) = 2x+e'.
Solution
.v'= f(x,y) =2+ y - 2x and h=0. l
Xo =0, Yo = I
k, = /(x0 ,y0 ) =2+ Yo - 2x0 = 3.00000000
k3 = f(x 0 +%,Yo +%k1 )= 2 +(Yo +~k2 ) - 2( .r0 +~)=3.05250000
k, = J(x0 +h,y0 + hk3 ) = 2+(y0 + hkJ - 2(x0 + h )= 3. !0525000
)' 1 = Yo +~[k, + 2k1 + 2k3 + k,] = 1.30517083
k3 = J(x, +%,y, +~k2 )= 2+(y, +~k 2 ) - 2(x, +~)= 3.16319230
k, = f (x, +h,y, +hk3 ) =2 +(y, +hk 3 )- 2(x, +h) = 3.22149006
It
Y, = Y, + (k, + 2k2 + 2k3 + k4 ] = 1.62140257
6
and soon.
Table 7.6 shows the results obtained using Rw1ge-Kutta method of order four with h = 0.1.
Table 7.6 Results of Runge-Kuna method of order four with h =0.l
n x. Y. Exact Solution Absolute Error

0 0.0 l.00000-000 1.00000000
I 0.1 1.30517083 1.30517092 0.00000009
2 0.2 1.62140257 l.62140276 0.00000019
3 0.3 J.94985850 1.94985881 0.0000003 1
4 0.4 2.29182424 2.29182470 0.00000046
5 0.5 2.64872()64 2.64872127 0.00000063
6 0.6 3.02211796 3.02211880 0.00000084
7 0.7 3.41375163 3.41375271 0.00000108
8 0.8 3.82553956 3.82554093 0.00000137
9 0.9 4.2596014] 4.2596031 l 0.00000170
JO 1.0 4.71827974 4.71828183 0.00000208
•
Example 7.6
For die initial value problem y' = - l OOy +5exp(- 1001), 0 ~ 1 ~ 0.05, y (O) =-4 the actual solution is
y = St exp(- l001) - 4cxp(- 1001). Utilizing die Forward Euler' s, Modified Euler's, Taylor's Mediod
of order two and Runge-Kuna method of order four for solving initial value problems calculate die
absolute errors of each technique. Use a step size of 0.0025.
Solution
Ta ble 7.7 shows the absolute error of Euler's, Modified Euler's, Taylor's method of order two and
Range-Kutta method of fourth order.
Table 7 .7 Absolute errors
Absolute E rror
Modified Taylor's Runge-Kutta
11 t Euler's Euler's Order two Fourth order
0 0.0000 -4.0 -4.0 -4.0 -4.0
l 0.0025 0.11796812 0.00997687 0.01015688 0.00003196
2 0.0050 0.18006938 0.01554058 0.0158214 0.00004970
3 0.0075 0.20616661 0.018 15522 0.01848379 0.00005797
4 0.0100 0.20983898 0.01885312 0.01919484 0.0000601 1
(Co111inued)
Table 7.7 Absolute errors (Co111i1111ed)
Absolute Error
Modified Taylor's Runge-Kutta
n I Euler's Euler's Order two Fourth order
5 0.0125 0.20024752 0.0.1835424 0.01868743 0.00005842
6 0.0150 0.18346835 0.01715381 0.01746569 0.00005452
7 0.0175 0.16344154 0.01558657 0.01587039 0.00004946
8 0.0200 0.14264302 0.01387349 0.0141265 0.00004395
9 0.0225 0.12255773 0.01215574 0.01237777 0.00003845
10 0.0250 0.10401055 0.01051918 0.01071162 0.00003322
11 0.0275 0.08739581 0.00901193 0.00917705 0.00002842
l2 0.0300 0.07283515 0.00765684 0.00779735 0.00002410
13 0.0325 0.06028456 0.00646034 0.00657907 0.00002031
14 0.0350 0.04960607 0.00541855 0.00551829 0.00001700
15 0.0375 0.04061468 0.00452158 0.00460493 0.00001417
16 0.0400 0.03310836 0.00375632 0.00382567 0.00001175
J7 0.0425 0.02688643 0.00310838 0.00316586 0.00000971
18 0.0450 0.02176025 0.00256331 0.00261079 0.00000799
19 0.0475 0.0175587 0.0021073 0.00214639 0.00000656
20 0.0500 0.01413047 0.00172761 0.00175971 0.00000537
It can be noticed from Table 7.6 that the errors produced by the Euler's melhod are the highest, errors
by the Modified Euler's method and the Taylor's method of order two are comparable and the errors
produced by the Runge-Kutta method of order four are the lowest. It is also interesting to note that
the Runge-Kutta method achieves this degree of high accuracy without having to compute higher
order derivatives of y . Runge-Kutta method of order four is also known as classic Runge-Kutta
method. There are also several variations of Runge-Kutta method :available in literature, such as
Larson·s fifth-order Runge-Kuna method, Butcher's sixth-order Runge-Kutta method and Runge-
Kutta-Fehlbcrg method.
Problem Set 7.1

1. Use Forward Euler's method to solve the following initial-value problems.
a. y'=xe2x - 2 y, 0:=,x:=,l, y(0)=0, fr=0.l
2
b. y'=1+(x - y) • l$ x $ 2, y(l)=I, h=0.l
c. y , = 1+-X , 15x52, y(1)=2, h=O.l

y
d. y' =sin(2x)+cos(3x) . 05.xSl, y(0)=l, h=0.l

2
e. y'=I - (1'.) +1, JSxS2, y( l)= - 1. h=O.l

X X
f. / =l+{x+I)(y+l), 05x52, y(0) =2, h=0.1

g. y'aa l+ y+e', 05x5 1, y(0)aa 0.5, haa0.1
2. Repeat Problem I with Backward Euler's method.
3. Repeat Problem l with Modified Euler's method.
4. Repeat Problem I with Second-order Taylor method.
5. Repeat Problem l with Classic Rungc-Kutta method.
6. Solve the following initial-value problem using Forward Euler's method, Backward Euler's
method, Modified Euler's method, Second-order Taylor method and Runge-Kutta method of
order four.
y' = y - x+l+lOexp(- x), 05xS2, y(0) = - 5, h=0.l
Compute the absolute errors in each case if tbc exact solutiorn is y = x - Sexp(- x).
7.7 Runge-Kutta-Fehlberg Method

One way to improve the accuracy of the solution of an initial value problem is to solve the problem
twice, first with step size h and then with step size h/2 and then compare the results. If the absolute
difference of tbe results is acceptable then tbe step size h does not have to be reduced. On the other
hand if Lhe absolute difference is not acceptable then the step size bas to be further halved and the
computation has to be repeated until we get good agreement in the results.
Runge-Kutta-Fehlberg method resolves the accuracy issue by determining an appropriate step size.
Two different approximations using the same step size h are made and the numbers compared. If the
two numbers are in good agreement, then the approximation is accepted. If the numbers do not agree
within a specified accuracy, then the step size is reduced. If the numbers agree to more significant
digits than required. the step size is increased.
The determination of a reduction or an inflation factor of the step size starts by selecting two approx-
imation techniques such that one bas a higher order local truncation error than the other one.
Suppose we have the following initial value prob lem.
y'=f(x,y), a5x5b, y(a) = y0
The first approxin1ation technique can be derived from an 11th order Taylor method. The correspond-
ing initial condition and the difference equation can be written as
illo = Yo
The second approximation technique can be derived from an (n + l) order Taylor method. The cor-
responding initial condition and the difference equation can be wrinen as
illo = Yo
We can compute an approximate value of the local truncation error as
Since the local truncation error T,., (h) is O (Ii"), a constant C independent of the step size h exists
such that T,., (h) "'Ch".
The local truncation error produced by the same 11th order method. with a step size of sh can be
approximated as
T,., (sh)"'C(sh)" =s"(Ch")=s"T,+1 (h)
Select s such that the local truncation error is bounded by the given tolerance, ,:;.
(7.24)
Runge-Kutta-Fehlberg method estimates the local truncation error ofRunge-Kutta method of order
four by utilizing Runge-Kuna method of order five.
First. we find an approximation to the solution using Runge-Kutta method of order four in the fol-
lowing manner.
(7.25)
Then we seek a better approximation to the solution using Runge-Kutta method of order five in the
following manner.
We assume that ro1 ""w1 when a fixed step size h is used. Therefore,
ill =W +~k + 6656 k + 28561 k4 _J...k. +!:....k6 (7.26)

M I J35 I ) 2825 l 56430 50 ' 55
At each step, both Equations (7 .25) and (7 .26) require the evaluatioru of foUowing coefficients.
k, =hf(x1 , (J)1 )
k2 =l!f(x1 +{1t,CtJ, +¼k,)
Note that both Runge-Kutta methods as utilized in Runge-Kutta-Fehlberg method use the same
six coefficients. This amounts to a computational savings since a stand-alone Runge-Kutta method
of order four would require four coefficients and a stand-alone Runge-Kutta method of order five
would require six coefficients to be computed.
If (l/h)l@ - w,.,I
,+1 >t:. then reject the injtial choice of step size hat the ith step. Calculate a new
step sjze sh and repeat the step. If (1/h)l<O,+, - w,+,l~t:. then accept the computed value at the ilb
step with step size I, and calculate a new step size sh to be used io the next step. In most of the
applications of Runge-Kutta-Fehlberg method, Runge-Kutta method of order four is utilized to
compute the solution giving us 11 =4 to determine the reduction or inflation factor of the step size
in Equation (7 .24).
Since we are mainly interested in controlling the error which calls for reductions in step size,
a min imum limit on the step size could be placed to avoid computations in the zones where the
error is acceptable. Moreover, the step size inflation part could be completely omitted. There
are, however, more than one ways to implement the error control and step size changes using
Runge-Kutta-Fehlberg method. A conservative version of the step size reduc tion (inflation) fac-
tor as shown in Equation (7 .27) is utilized in many implementations of Runge-Kutta-Fehlberg
method.
(7.27)
Example 7.7
Consider the initial value problem y' = 2+ y - 2x, 0:,; x::;; I, y(O) = I. Use Runge-Kuna-FeWberg
method to approximate the solution with a tolerance e = !0-6 and a maximum step size of 0.2.
Compare the results with the exact solution y (x) = 2x + e'.
Solutio11
First we use the maximum step size of 0.2 to calculate 1he six coefficients.
k, =lif(x1 ,mi) =0.6
k = /if( x, + ¾h, m, +¾k,) = 0.61

1
k 1 = lif(x1 + ~h.m, +2-k1 +.2.k, )= 0.6155625

· 8 32 32 -
12 1932 7200 7296 )

k, =!if( x1 +- h.m, +- -k, - - -k, +- -k 3 =0.64070496
13 . 2197 2197 - 2197
439 3680 845 )

ks= /if ( x, +h,m, +- k, - 8k, +- _- k 1 - - - k, = 0.64465 128
216 - ) 13 · 4104
I 8 3544 I 859 11 )
k6 =!if( x, + - h,w, - -k, +2k., - - -k3 +- -k, - -k5 =0.62093136
2 27 - 2565 4104 40 ·
25 1408 2197 I
m1+ 1 =m1 +- k, + - -k3 +- -k4 --k5 = 1.62 140308
216 2565 4101 5
• 16 6656 28561 9 2
w,+1 =m1 +- k1 + - -k3 +- - k, - -ks +- k6 =l.6214027
135 12825 56430 50 55
The local error of this step is
and the corresponding reduction factor of the step size
The local error is greater than the tolerance, therefore, the step size is reduced by a factorof0.7157 !07
and the computalion is repeated. The results are shown in Table 7 .8.
Table 7.8 Results of Runge-Kutta-FehJberg method of order four
Absolute
11 X Step size Y. Exact Solution Error
•
0 0.000000 1.00000000 1.00000000
I 0.168750 0.168750 1.52132430 1.52132415 0.00000014
2 0.328931 0.160181 2.04734315 2.04734286 0.00000030
3 0.480977 0.152046 2.57960913 2.57960866 0.00000047
4 0.627272 0. 146294 3.12703859 3.12703793 0.00000066
5 0.768032 0.140760 3.69158332 3.69158245 0.00000087
6 0.905314 0.137283 4.28333890 4.28333779 0.00000111
7 1.000000 0.094686 4.71828307 4.71828183 0.00000124
•
Problem Set 7.2
1. Use Runge-Kutta-Fehlberg method of order four 10 solve the following initial-value
problems. Use a tolerance of 10-6 aad a maximum s1ep size of 0.2 aad a miaimum s1ep
size of 0.0 I .
a. y' =xe 2' - 2y, 0::;x::; 1, y(0)=0
2
b. y'=l+(x - y) , 1:,;x::;2, _v(l)=l
c. y' =I+~. 1:,;x:;;2, y (l}=2

y
d. y' = sin(2x)+ cos(3x) , 0:,; x S 1, y (O) = I

2
e. y =l - (l) + l , l~ x ~2. y(l)= - 1
X X
f. y' =l+(x +l}(y +I}, o,::;x::;2, y (0)=2
g. y'=l+ y +e' . 0:;;x:;;1 , y(0)=0.5
2. Solve the followiag iaitial-value problem using Runge-Kutta-FehJberg method of order four.
Use a toleraace of 10-6 and a maximum step size of0.2 and a rninimun1 step size of0.01.
y' = y - x +l+!Oexp(- x), 0:;;x:;;2, y (0)=- 5, '1=0.1
Compute the absolute errors if the exact solution is y = .r - 5exp(- x).

7.8 Multistep Methods

Multistep methods utilize results obtained in the multiple mesh points to approximate the results
at a current mesh point. The order of a multi.step method is determined by the number of s uch
mesh points utilized in calculating the approximate value at the current mesh point. A multistep
method is explicit when only the previous mesh points are utilized in the approximation. A mul-
tistep method is implicit when the previous as well as the current mesh point is imbedded in the
approximation.
7.8.1 4th order Adams•Bashforth technique

4"' order Adams-Basbfortb techniqllle is an explicit one that utilizes a.p proximations from four mesh
points. Suppose</>'(x) can be expressed as a polynomial, P3 (x) of degree three. Let the polynomial
pass through four points; ( x., f. ), (-rn-P f._1) , (x•. 2 , /,,. 2 ) and (x•. 3 • f. _3 )·
Utilizing Lagrange interpolating polynomial, we can express ~ (x) as
The approximation at the current mesh point is
Define step size as

h=xn - x n- 1 =xn- D - x11- 2 =xn- 2 - x n- 3
,.(x 1
)=<f>(x )+ f.
3
(551i•)_f._,(591,•)+l- (37/z•) _f._(91i•)
2 3
3
'i' "' " 611 4 21,3 12 21,3 12 6h 4
</>(x..,) = </>(x. )+ ;
4
{5Sf. - 59f.,. + 37/..
1 2 - 9/,,. 3 ) (7.28)
Equation (7.27) is known as 4'h order Adams-Basbfortb fonnula. We need f.,, f.,_,, J•.1 and f._3 to
calculate the approximate value al x.+i· At the beginning tbey are calculated using Runge-Kutta
method of order four.
Example 7.8
Consider tbe initial value problem y' =2+ y - 2x, 0 ~x ~ I, y(0) =I.Use 4m order Adams-Bashfortb
method to approximate the solution with a step size of 0.1. Compare the results witb the exact solu-
tion y(x)=2x+e·'.
Sollltio11
Let f(x.y)=2+ y - 2x
The first four approximations can !be obtained from Rungc-Kutta method of order four shown in
Example 7.5.
Yo =I, y, =l.30517083, y2 =l.62140257, y3 =l.94985850
The approximation for the fifth mesh point is
The results are shown in Table 7.9.
Table 7.9 Results of 41h Order Adams-Bashfortb method with h =0.1
II X Y, Exact Solution Absolute E rror

•
0 0.0 J.00000000
I 0.1 l.30517092
2 0.2 1.62140276
3 0.3 l.94985881
4 0.4 2.291820Ll 2.29182470 0.00000459
5 0.5 2.64871099 2.64872127 0.00001028
6 0.6 3.02210208 3.02211880 0.00001672
7 0.7 3.41372826 3.41375271 0.00002445
8 0.8 3.82550725 3.82554093 0.00003368
9 0.9 4.25955853 4.25960311 0.00004458
10 l.O 4.71822444 4.7 1828183 0.00005738
•
<$> Chapter 7: Initial-Value Problems for Ordinary Differential Equations
7.8.2 4th order Adams-Moulton technique

A slight modification to the 4., order Adams -Bashforth technique can be made if the polynomial
~ (x) to approximately represent ¢'(x) is considered to pass through (x••,,J..,), (x,., /,, ). (x,..,. /,..,)
and (x•. 2 ,f.,_2 ) . The previous integration results shown in 7.8.1 can be utilized if we substitute
(x...,, f.+,) for (x,,_3 , f,,_3 ).
(7.29)
Equation (7.29) is known as the 4"" order Adams-Moulton formula. We need J..,. f., J,.., and f.,_2 to
calculate the approximate value at x,,..1• Like the 4., order Adams-Ba.;;hforth technique, at the begin-
ning they are also calculated using Runge-Kuna method of order fou.r. Equation (7.29) is, however,
an implicit expression. Some algebraic manipulation is, therefore, necessary at each step.
Example 7.9
Consider the initial value problem / = 2 + y - 2x, 0:,; x:,; I, y(0) = L Use 4., order Adan1s-Moulton
method to approximate the sol ution with a step size of 0.1. Compare the results with the exact
solution y(x) = 2x+ e'.
Solutio11
Let/ (x, y) =2+ y - 2x
The first four approximations can be obtained from Rw1ge-Kuna method of order four as shown in
Ex.ample 7.5.
Yo = 1, y, = 1.305 17083, y 2 = 1.62140257. y3 =1.94985850
(7.30)
Since Equation (7 .30) is an implicit expression, we need some algebraic manipulations.
l+1=2 + ytt+I - 2x,...,
h (9(2+ Yn+I -
y,.., = Y,. + 24 2x,.. , )+ I 9 /,, - sf•. ,+ 1.,-2)
Y. +~(l& -18x••, + 19.f., - 5/,.. 1 + J~.i)
y,.., = (1- ~~)

24 { Ji }
Y•• , = -23.I Y. + -24(18 - 18r
. ,..., +19 f. - 5.f.., + J,..2 )
The approximation for the fourth mes h point is
The results are shown in Table 7.10.
Table 7 .10 Results of 4w Order Adams-Moulton method with h = 0.1
II X Y. Exact Solution Absolute Error

•
0 0.0 1.00000000
l O.l 1.305 17092
2 0.2 1.62140276
3 0.3 1.94985892 1.94985881 0.00000012
4 0.4 2.29182519 2.29182470 0.00000049
5 0.5 2.64872222 2.64872127 0.00000095
6 0.6 3.022 12029 3.02211880 0.00000149
7 0.7 3.41375485 3.41375271 0.000002 14
8 0.8 3.82554384 3.82554093 0.00000291
9 0.9 4.25960693 4.259603 11 0.00000382
10 1.0 4.7 1828671 4.71828183 0.00000488
•
Example7.10
Show the absolute errors of Runge-Kutta 4w order. 41h order Adams-Bashforth and 41h order Adams-
Moulton techniques for the initial value problem in Example 7.5.
Solution
The absolute errors of Runge-Kutta 4w order, 4th order Adams-Bashforth and 4 th order Adams-
Moulton techniques are shown in Table 7. l l .
Table 7 . 11 Absolute errors
Absolute Error
Runge-Kutta Fourth Order Fourth Order
n X Fourth O rder Adams-B ashfortb Adams-MouJtoo
•
0 0.0
1 0.1 0.00000009
2 0.2 0.00000019
3 0.3 0.00000031 0.00000012
4 0.4 0.00000046 0.00000459 0.00000049
5 0.5 0.00000063 0.00001028 0.00000095
6 0.6 0.00000084 0.00001672 0.00000149
7 0.7 0.00000108 0.00002445 0.00000214
8 0.8 0.00000137 0.00003368 0.00000291
9 0.9 0.00000170 0.00004458 0.00000382
10 1.0 0.00000208 0.00005738 0.00000488
■
Problem Set 7.3

1. Use 4m Order Adams-Bashforth metbod to solve the followin_g initial-value problems.
a. y'=xe 2' - 2y, 0::;x::; 1, y(0)=0, h=0.l
, ,
b. y =l+(x - yr, 1:,; x::;2, y(I}=l, '1=0.l
c. y' =I+~. I ::;x $2, y( I} =2, h =0.1

y
d. y'=sin(2x) + cos(3x), o::;x:,;1, y(0)= l, h=0.l
e. y' = r
~- ( ~ + I, I :,; x:,; 2, y(I} = - l, h = 0.1
f. y'=l+(x+l){y+l) , O,:,;x::;2, y(0)=2, h=0.l

g. y'=l+y+e' , 0::;x::; 1, y(0)=0.5, h=0.l
2. Solve the foUowing initial-value problem using 4th Order Adams-Bashforth method.
y' = y - x+l+lOexp(- x), 0SxS2, y(0)=- 5, h=O.I
Compute d1e absolute errors if the exact solution is y = x - Sexp(- x).

3. Use 4m Order Adams-Moulton method to solve the foUowing initial-value problems.
a. y'=xe 2' - 2y, OSxS l , y(O)=O, h=O.I
b. l =l+(x - y)2, l$ x~2. y(l)=l, h=0.1
c. y' =l+::, IS xS2, y(1)=2, h=O.I

y
d. y' =sin(2x)+cos(3x) , OSxSl, y(0)=l, h=0.l
e. / =; - (; J + I, I S x S2, y( l)=- l, '1=0.l
f. y' =l+(x+l)(y +I) , 0SxS2, y(0)=2, h=0.l
g. y'=l+ y+ex, OSxSI , y(0)=0.S, h=O.l
4. Solve the following initial-value problem using 4th Order Adams-Moulton method.
y' = y - x+l+lOexp(- x), 0Sx$2, y(0)=- 5, '1=0.1
Compute the absolute errors if ilie exact solution is y = x - Sexp(- x).

_J
R 0D10 . IONS
---+-+-~----+------,----;F- RIES
Fourier series analysis is used in solving boundary-value problems, harmonic analysis of vibrations
and electrical current in transmission line and the analysis of input and output current and voltage of
power electronic devices, signal processing, filter design to name a few.
At the heart of Fourier series analysis is the expansion of periodic functions as sum of trigonometric
functions associated with individuaJ frequency components. From the expanded trigonometric sum
each frequency component can be picked up for further investigations.
8.1 Periodic Functions

A function f (8) is said to be periodic if it is defined for all real 8 and if there is some positive num-
ber, T such that f (8+ T)= f (0). The nmnbcr Tis d1en called a period off (0). A group of periodic
functions are shown below.
/(0)
(a)
Chapter 8: Periodic Functions and Fourier Series
/(8)
0
8
~T~
(b)
/(8)
--- T---•I
(c)
Figure 8.1 A g.roup of periodic functions
In engineering applications, 0 is the angle in radians related to angular velocity as

0=(.i)I
where ro is the angular velocity in radians per second and r is time in second.
The angular velocity is related to the frequency as ro= 27Cf where f is the frequency in cycles per
second. The frequency f is d1e inverse of time period, T. In angular domain, T = 21r.
A periodic function can be interpreted in angular as well as in ti me domain depending on the
mathematical convenience and need.
8.2 Fourier'sTheorem
A periodic function that satisfies certain conditions (Dirichlet's coruditions) can be expressed as a
sum of a number of sine functions of different amplitudes, phases and periods.
If f (0) is a periodic function with period T ilien
f (0) =Ao+ A, sin(0 +1/>1 )+ A2 sin (20 + 1/12 )+ ···+A,, sin (110+1/>.)+ ·· · (8.I)
where As are amplitudes, ¢s are phase angles.

The term A1 sin (0 + ¢ 1 ) is called ilie first harmonic or fundamental component. A fundamental com-
ponent has the san1e period of its parent function, f (0). A. sin (110+ ¢.) is the 11th harmonic that has
an angular frequency of nm, 11 limes the frequency of the first harmonic. A,, is lbe amplitude, 1/J. is
the phase angle of the nth harmonic and nm=n(2n/).
For such representation of /(0) as depicted in Equation (8. 1), lbe number of terms should approach
infinity. However, in most practical applications, the amplitude gets smaller as n gets higher and,
therefore. only a finite number of terms can be utilized without losin_g the desired accuracy.
Using trigonometric identity, the nth harmonic in Equation (8.1) can be split into two terms, a sine
and a cosine, in the following manner.
A,, sin(n0+</J.) = A. cos(n0)sin(</J.) + A,, sin (n0)cos ( </J.,)
=A.sin (<P. )cos(n0)+ A. cos( <P.)sin (110)
=a. cos(n0)+b,, sin(n0)
wlnere
a. =A.sin(</>.,) and b., =A.cos(</>.,)
Equation (8. 1) can be rewritten as
f {0) = Ao +a, cos(0) +b1 sin (0) +a2 cos(20)+b2 sin (20) + ··· +a., cos(110)+b., sin (110) + · ··
= Ao +a, cos(B)+a.i cos(20) + ··· +a. cos(nB)+···
+b1 sin(B) +b2 sin(28) + ·· · +b. sin(nB)+ ·· ·
f(B) = 0 0 + f, a., cos(nB)+ f b.sin(nB) (8.2)

2 11=1 n=I
where a 0 = 2A,i.
There is no mathematical reason for replacing Ao other than using a general format in finding as. The
amplitude and the phase of the 111h harmonic can be expressed as
A. =✓(a.)2 +(b.) 2
and 1/J,, =tan- •(::}
The expression (8.2) is called the Fourier series expansion of f(B). a,, and b,, are Fourier coefficients
of the 11th hannonic.
8.2.1 Dirichlet's conditions

At d1e time when Fourier proposed bis theory of series expansion while solving heat-flow problems, it
wa~ dismissed by many mathematicians of his ti.me due to a lack of mathematical rigour. Later, Dirichlet
and Reimann provided the mad1ematical rigour to establi~h Fourier's theorem under solid footings.
In order to satisfy the condition that a Fourier series expansion of J (t) converges to/ (1), Dirichlet
proposed the following conditions.
If J (t) is a bounded periodic function that in any period has (1) a finite number of isolated maxima
and minima, and (2) a finite number of points of finite discontinuity. then the Fourier series expan-
sion off (t) converges to /(1) at all points where f(t) is continuous and to the average of the right-
and left-hand limits of J (I) at points where J (t) is discontinuous.
8.3 Fourier Coefficients

For a periodic function we would like to determine the coefficients, a,. and b. shown in
Equation (8.2). In determining the coefficients we will encounter some integrations. Let us first
list those integration results in a general form.
J-,:•cos118cosm&d8= -2lJ"-• cos(11+m)8d8+ -2lJ"_,, cos(11 - m)8d8

;r
J- 1'
cos118cosm&d8=0 if n *"'
J_,,• cos118cosm&d8=1t ifn=m
J• cos11.8d8 = 0 for all n
- ,:
J• si11118cosm8d8=]_ J• si11(11+111)8d8+.!. J" sin(11 - m)8d8

- 2 -• 2 -•
J"si11118cos1118d8 = 0 for all values of 11
- J<
J• sin 118d8 = 0 for all 11

- ,:
First, determine Go· To do this, we integrate both sides of (8.2) from - 7C to 7C. Alternately, you will
get the same result if you integrate expression (8.2) from O to 2n. The selection of the limits of
inetegration is, however, a matter of convenience. The only requirement to be fulfilled is that you
integrate expression (8.2) over one period. This applies to a. and b. as well
J_:J(8)d8 = L:[~ t,a. + cos118 + t.b•

sin118 ]d8
J_:f(8)d8 = L: 0
; d8+ L:(t,a. cos118 )d8+ J:.( t,b. sinn8 )d8
f "f(8)d8=1ta +0+0
- ,:
0
Therefore, a0 = _!_ J• f (8)d8

7f -•
a0 /2 is the average value of the function /(8).
In order to determine a1, a1 •.• , multiply both sides of (8.2) by cos 1118, where III is any positive integer,
and then integrate from - 7r to 1t.
J_:f (8)cosm 8d6 =L:[ ~ + t,a. cos118+ t,b. sin118 ]cos1118d8
Let us do the integration on the right-hand-side one term at a time.

First term:
•a 0 a J" 0
J-• -2 cosm8d8= -2 -• cosm8d8=0
Second term:
f •I a. cosn8cosm8d8 =a,.1C when

- Jf
11 equals m and is zero elsewhere
,1=J
Third tenn:
f-• I,b.sinn8co.sm8d8=0 for all 11.

8
n=I
Therefore,
f •J(8)cosm8d6=a,,.1C
-n
a.,= -1 f H
J(8)cosm8d8 m=l,2, ···
JC - •
In order to determine b1, b2' ... multiply both sides of (8.2) by sinm8, where m is any posi tive
integer, and then integrate (8.2) from - 11: to n.
L :f(8)sinm8d8 = L:[i + t a• cosn8+ t b,, sin118}inm8d8
Let us again do the integration on the right-hand-side one tem1 at a time.

First term:
a f"sinm8d8 =O
f_,,•a~sinm8d8
2
=-.J!.
2 -•
Second term:
f-••I a. cosn8sinm8d8 = 0 for all 11
Third term:
=•
J., I,b. sinn8 sil!lm8d8 = b,. n when II equals III and is zero elsewhere
-•
=•
Therefore,
f ".f(8)sin1118d6= b,.1e
- J<
b,. = -l
1t
f" J(8)si.nm8d8
- lC
m=l,2,···
The Fourier coefficients, therefore, are:
a0 = -I
77:
f "f( 8)d8
_,,
a.,= - If" J(8)cosm8d8

JC -•
111= 1,2,···
b., = -
I f,, f(8)sinm8d8 Ill= 1, 2, ···
1C - ,"t
We can write n in place of III without losing the generality of how to detemune the Fourier coefficients,
a0 = -If •f (0)<10
1f -•
a.= -I f •f(0)cos110d0 11=1, 2, . . .

7f -•
b. = -If" J(0)sinn0d0 II= J, 2, ...

7f - •
Example8.1
Find the Fourier series of the periodic function /(0) shown in Figure 8.2.
/(8)
0
8
-A
1t 21t 31t 41t 51t 61t
Figure 8.2 A square wave
Solution
The periodic function shown in Figure 8.2 can be expressed as
/(0) =A when O<0<n
= - A when n<0<27r
and due to its periodicity, f (0+ 27r) = f (0).
a0 = -
21C
}fO
l• f(0)d0= -j
1C
[ J•f(0)d0 + f l• f(0)d0 ] = Hf •AdB+ f 24 - AdB] =0
O " 1C O "
a= - - f(B)cosnBdB= Aff' Acos118d8+ J'•

If '" - (- A)cos118d8]
"no no .I(
a.= _!_[A sinnBI +..!.[-Asin118] = O

2
"
1C II 7r 11,
b. = -I f f(0)s.inn8d0=-I [f "
2,,
Asin118d8+ J (- A)sin118d8 2, ]
1C O 7r O •
2
cosne
b. = -I [ - A- -]" + -l [ A- 110] " = - A [- cos,m+cosO+cos2mr - cos111r)
cos -
1C 11 o 7f II • fl7f
Chapter 8 : Periodic Functions and Fourier Series
A A 4A
b. = - [- cosnn +cosO+ cos2111r - cosnn ]= - [I+ I+ 1 + I]= - when II is odd
ll7r 117r 117t
A A
b. = - [- cos111r +cosO+cos2111r - cosnn]= - [- 1+I+ I- I]= 0 when 11 is even
ll7r 117r
Therefore. the corresponding Fourier series is
4
n
A (sin0 +.!sin 30+.!.sin50+.!sin 70+·· ·)
3 5 7 •
8.4 Convergence of Fourier Series and Gibb's Phenomenon
Although Dirichlet's conditions provide the foundation for the convergence of Fourier series expan-
sion, the rate of convergence and the manner in which it approaches to f (0) at a discontinuity is of
practical interest in many applications.
Let us consider the Example 8.1. One important question is: how many harmonic terms should be
added to recreate f (0) with a reasonable accuracy? If we consider the first four harmonic components,
we get the following resulting wave shown in Figure 8.3. For graphing purpose, consider A= 5.
,~
..,
J.3
1.,
0
. 1.6
.J.J
••••
·•-" G Ll L5 u !
Figure 8.3 Re-creation of a square wave with tb.e first four

harmonic components
Now, consider the first eight harmonic components. Figure 8.4 shows the resulting wave.
When we consider the first 20 harmonic components tl1e resulting wave looks like Figure 8.5.
The rate at which a Fourier series expansion converges to tl1e function f (0) dictates how many
harmonic components we should consider for practical applications. Also, it can be noticed from
Figures 8.3-8.5 that at a discontinuity there are overshoots and undershoots. With more harmonic
components added these overshoots and undershoots get sharper but they never disappear, an
important observation known as Gibb's phenomenon. Hence, for practical applications, some kind
of filtering may be utilized to smooth out these over- and undershoots.
'-"
4J
J.J
u
0
, 1.6
.J.3
~-•
.,.s
• u u u 5 ,.J 7.5 I.I u u
Figure 8.4 Re-creation of a square wave with the first eight
harmonic components
6.5
1.6
0+-- - - --+-- - - -- - - - - --1-- - - -~
-6.$+ - - - - ~ - - - ~ - ~ - - ~ - - - - - ~ - ~
Q l.3 2..5 3.S 5 '--3 7~ 8~8 10 it ll
Figure 8.5 Re-creation of a square wave with th.e first

twenty harmonic components
8.5 Even and Odd Functions

Some computational work in detennining Fourier coefficients of a function can be avoided if the
function is odd or even.
A fw1ction f (8) is said to be even if f (- 8) = f (8). The graph of such a function is symmetric with
respect to the y-axis as shown in Figure 8.6.
A function f (8) is said to be odd if f( - 8}= - /(8). The graph of s uch a function is symmetric with
respect to the origin as shown in Figure 8. 7.
Chapter 8 : Periodic Functions and Fourier Series
/(0)
Figure 8.6 Even function
/(0)
e
Figure 8.7 Odd function
The Fourier series of an even function f (0) is expressed in terms of a cosine series.
i.e., f (0) = i + I,a. cos110
11= 1
The Fourier series of an odd f~nction / (0) is expressed in terms of a sine series.
i.e., f (0) = L,b. sinn0
Example8.2
Find the Fourier series of the periodic function f(x) in Figure 8.8.
/{x)
- lt O lt 3 lt Slt 7lt 9lt
Figure 8.8 A parabolic wave

The analytic representation of the function shown in Figure 8.8 is

J(x) =x 2 when - ,r-5,xs',n
and f(x +2n)= J(x)
Solution
l• l• , l..r3 •=• 2 0
a0 = - f J(x)dx= - f x- dx= - - =- n -
7C - • 7C - • 7C [ 3 ] 3
.x=-ll
a.= -I
J't
J•f(x)cosnxdx=-l [J
- II '/C - R
n x 2 cos11xdx ]
Using integration by paits,
I= fx 2
cosnxdx=x
2
f cosnxd.x - J{;(x )J cos,udx }dx 2
I = -x2 Slfl/lX
. 2
--
JXSUIIU'
. d·x
II II
2
x • 2x 2 .
I= - SID/IX+ COS/IX - J SJn nx
2
n n n
a• = -
7t
l. [ x- .
- '
11
Slll /IX
2x
JI
2 .
+ 2 COS /IX - - 3 S10 /U'.
JI
]"- Jr
4
= - 2 COS Jl7C
fl
4
a,.= - 1 when II is odd
n
4
a =- 0 when 11 is even
" II-
.1.e., an= (- 1)" -4,

IZ-
J (x) is an even function, therefore, b. = 0

The corresponding Fourier series for f (x) is
n2
-3 - 4 cosx
( cos2x cos3x cos4x
22 + 32 42
+···) •
Example8.3
Find the Fourier series of the perioilic function f(x) in Figure 8.9.
ftx)
0 lt llt 3Jt 4Jt
Figure 8.9 A full-wave rectified sine
The analytic representation of the function shown in Figure 8.6 is

f (x) = lsinxl
Solutio11
a 0 = -I f 2"
f(x)dx= -2 f "sinxdx = -4
TC o TC o TC
Forn=I, a,= ![f:sinxcosxdx]= ~[f :sin2xdx]=o

f (x) is an even function, therefore, bn = 0
The corresponding Fourier series for f ( x) is
2 4 (cos2x cos4x cos6x )

--- - -+- -+- -+···
TC TC 3 15 35 •
8.6 Functions Having Arbitrary Period
As mentioned before, a function f (t) can be expressed in time that has period, T. We already know
how to relate angle (0) with time (1).
Using the angle, time relationship we can change the limits of integration in the following manner.
2TC T
When 0 = - TC, - TC = - t and therefore, 1 = - - .
T 2
2TC T
When 0 = TC, Tr = - t and therefore, 1 = - .
T 2
2
Also, since 0 = 2nft we have d0 = TC dt
T
Using the angle. time relationships shown above, the Fourier coefficients of the periodic function
.f (t) of period T arc given by
T
a0 = -
2 ?
T _!.
r-
f(t)dt
2
n= I, 2, ...
b"=T-2 J1/() . (211:11 )d

1sm - -1 1 II= 1, 2, ...
_!. T
2
Fourier series off (t) then becomes
~
f(t)=a 0 + ..i::.,a,,cos (- ~ . (211:11
211:11- / ) + ..i::.,b,,sm - - - t)
n=I T r,:J T
Example8.4
Find the Fourier series of the periodic function /(t) in Figure 8.10.
fl()
Figure 8.10 A triangular wave
f(!) =I when
T
- - $/$ -
T
4 4
=- t+ -
T when -T $/$ -3T
2 4 4
and J(t+T)=J(t)
Solution.
This is an odd function. Therefore, a,. = 0.
T
2 2 2
b,. =!fr/(1)sin ( f(t)sin( 1tn 1)d1
11
T O
1t
T
1)d1= ~J
T o· T
b,, = -4 J isin ( - f
2,rn
-, ) dr+ -4 - 1+ - sin ( -
21r11 ff(
-, ) dt T)
To T T!. 2 T
4
Let us carry out the integrations one tenn at a time.
f . ( 2nn
/Slll - -1 ) dt
T
=f JS1ll. (2nn
- -1 ) dt +-T- Jcos( -
T
21r11
-/ ) dt
2m1 T
1
=---rcos
T - -t ) +( -T-) sm
(2n:n . (-
2nn
-t )
2n T ~11 T
Therefore,
ff Ism. (2m,
- -1) dt= [ - -T-/cos (2nn
T
- -1) + ( -T-)
2nn T 2nn
2
• (-
sm 2nn
T
-t )]f
0 0
2n) + (2nn
2 2
11
T cos (
= - 81r11 T ) • ("1t)
sm 2
Jf . (2ml )]f
2
)
- f Slll - -t dt = [ T - - f COS - - / (21m ) ( T - - -) •(21t11
Slll - -f
!. T 2nn T 21tn T r
• 4
1
T -cos (nn)
1, 1 1. (ll7r)
= -T -cos(111r) - ( -T-) sm(111r) - - - + ( -T-) Sill -
4nn 2nn 81r11 2 2nn 2
- -r )]f
ff -T sin. (2m1
- -r )dr = [- -
T -cos (2m1
1
! 2 T 41t11 T T
4 4
= --
4nn
2
T -cos(111r}+ -T -cos -
4nn 2
1
("n)
b _ 4[2( T )
"
- -
T
- -
· 2nn
2
I • (11,r)]
Slfl -
2
-- -
2T .
-SID
11
2
n2
("n)
-
2
h. = 0, when 11 is even. Therefore, the Fourier series is
•
8.7 Complex Form of Fourier Series

The Fourier series expansions presented so far is based on the basis functions, sin 9, sin 29, sin 39, ...
and cos 9, cos 29, cos39, .... An alternate presentation ofFourier series is used extensively in many
fields tha1 is based on the basis fllllctions, e16 , e 216 , e316 , • • • and e- 19 , e-216 , e-316 , ••. • The conversion
of the basis functions can be achieved very conveniently by utilizing llhe following Euler's fornmlae.
e;o +e-Jo . el8 - e- Jo
cos9= - - - and sm9= - - -
2 2i
The nth harmonic component of (8.2) can be expressed as:
=a,. - -
2- -
Denoting
a (a" -2;·b" ) and c_,, = (a" +;·b
c0 = i 2' c• = 2 '
) 1
I.he Fourier series expansion off (9') can be expressed as:

-
J(9)=c0 + I, (c.,e1" 6 +c_.e-1" 6 )=
- c.eJn6
I,
The coefficients c. for n > 0 can be evaluated in the following manner.
c,. = ( a• - jb• ) = - J
2 2ir -•
J" j
f(9)cosn9d9 - -
2ir -•
J"
f(9)sinn9d9
=-I J"
2ir _,,
f(9)(cosn9 - jsinn9)d9
=-
1
2ir -•
J"
f(B)e- 1" 6 dB
Similarly,
Note that c_. is the complex conjugate of c,..

For 11=0,
Co= - l
2n-
J•f(0)d0
- ;r
Hence we may write that c. = - I J."f (0)e-fn d0 for n = 0, ±I,± 2, ...

8
2n- -•
and the complex form of the Fourier series expansion off (0) with period 21t is:
If the period of the function f(0) is T then

- flrrnt
f( t ) = I, c.e_r_
T
1 !
and c,. =TL!. f (t)e _ j2nJrl
r dt for 11 =0, ± 1, ± 2, ...
?
Example8.S
Find the complex Fourier series of the periodic function f (0) shown in Figure 8.2.
Solutio11
The periodic function shown in Figure 8.2 is a square wave that can lbe expressed as
f(8) =A when 0<8<n-
=- A when 1t<0<2n-
and due to its periodicity, f (8+ 2n) = f (8).
c =-
"
-J
1
2;,r
2
0
" f (0) e-1" 8 d0
c = -2;,r1 Juf(8)d8= -21t1 1•f(8)d8+ -2n-1 J•

0
O O
2
' f(8)d8
= -1 J" Ad0+ - I J - Ad8 2,

2,r O 2n- "
C0 =0
c. = __:!:___
2n-
J 2
0
• - f.Ae-
f (8)e-;,, 8 dB = - 1
2,r O
1" 8 d8+ - -·
1
2n-
Ii" ..- Ae-;nB dB
c,. = -¢-(1- 2e_1,,,. + e- Jn~)

2
2;111r
Therefore, the complex Fourier series expansion off (8) is
n=-
2A [ e - SJB e -JJe e - JB elB eJJs esJB ]

/(0)= -
jlt
···+- - +-
-5 -3
+-
-l
+-
I
+-
3
+-
5
+ ··· •
8.8 Functions Defined Over a Finite Interval
In some applications, functions are not periodic, they are defined only over a finite i11terval.
Mathematically, artificial repetitions that satisfy Di rich.let's conditions can be incorporated to extend
these functions so that they behave as periodic waves and then we can apply Fourier series expan•
sions to determine the underlying harmonic components of the original functions.
Suppose f (r) is defined over the finite interval O < r < 1/f. We can find a Fourier series representation
of f (1) if we define ¢(r) as I.he periodic extension off (1) in the following manner.
¢(1)=/(1) 0<£<1/f
ifr(t+lfl)=l/>(t)
If lhe representation contains both sine and cosine terms, then it is called a full-range series. However,
we may incorporate the repetitions in such a way that the representation can be made up of either
sine terms only or cosine tenns only. In this case, it is called a half-range series.
Let us examine Figure 8.11 that contains f (1) and its extension ¢(1). cf>(r) is neither odd nor even
and, therefore, its Fourier series expansion will contain both sine and cosine 1em1s.
ft.r ) /(1)
0
(a) (b)
Figure 8.11 (a) A finite interval function and its (b) fulj.
range periodic extension
The Fourier series expansion of the full-range extension ¢(r) is
<p(I)= -A - ~
£_,- A SID
. (2nn
- ()
2 •=•111C lfl
However, /(1) is only valid in 0<1 < lfl
/(1)=A - I , ~sin(2111r1), 0<1<1{1

2 •=• nn: l{I
The periodic extension off(r) can be done in such a way that the extension ¢(r) becomes an even
function as shown in Figure 8. I2. In this case the Fourier series expansion will contain only cosine
tenns.
_/{I) f(t )
A
0 'II
(a} (b}
Figure 8.12 (a) A finite interval function and its (b) even
half-range periodic extension
The Fourier series expansion of lhe even half-range extension ¢(r) is
ip(t)= -A - ~
£.., -4A- ,cos (nTC
- l)
2 ,,.-(111r) lfl
However, f (I) is on!y valid in O< I < If/
~ -( 4A
A - -'-'
f(t) = - - 2 cos(nTC )
- I . 0<1<1/f
2 ,r-odJ n 1C ) If/
The periodic extension of /(1) can be done in such a way that the extension ¢(1) becomes an odd
function as shown in Figure 8. 13. In this case the Fourier series expansion wilJ contain only sine
terms.
The Fourier series expansion of the odd half-range extension 1/)(r) is
¢(1)= I,(- 1),..., 2A sin(1111: 1)

n=I 1/7[ lfl
However,/(r) is only valid in 0<1 <'If.
( )'"' -2A srn

f (r) = ~£..,,-1 . ( -/ITC r,
) O<t<lfl
,r-1 II TC lfl
,/(I) /(1)
A A
0 ljl -ljl 'l' 2111

(a)
-A V
___ (b)
FigureS.13 (a)A finite interval function and its (b) odd

half-range periodic extension
It should be noted that in all three cases the Fourier series expansion, converges to f (t) only within
the interval O< 1 < 1/f.
Problem Set 8.1

1. Find the Fouries series of the function J (x) shown below.
f(x) =Alxl - n:SxSn and f(x +2n )=J(x)
j[_x)
- it 0 3n 51t X
2. Find the Fourier series of the function /(1) shown below. Show the first 12 non-zero har-
monic terms.
}[_I)
3. Find the Fouries series of the function f (x) shown below.

f(x) =Ax - n$x$n and f(x+2n)=f(x)
4. Find the Fourier series oflhe function f(t). Show the first 12 non-zero harmonic terms.
f(t)=500At _ T s1sT , f(1+T)=f(1) , T=2ms

2 2
./U)
5. Find Lhe Fouries series of the function f (x) shown below.

J(x) =x 0SxS2n and f(x+2n)=J(x)
f{x)
0 2:rt 41t 61t X
6. Find the Fourier series of tl1e function f (t) shown below. Show the first 12 non-zero har-
monic terms.
f(t) =500At O St ST, f(t+T)=J(t), T=2 ms
./(/)
0 T 2T 3T f
7. Find the Fourier series of the periodic function f (x) shown below. Calculate the coefficients
of the first 12 non-zero ham1onic components.
fl.x)
X
Sn
8. Find the Fourier series of the periodic function/ (x) shown. below. Determine the first 12
non-zero tem1s. Considering the amplitude of the fundamental component as 100, express
the other hannonic components as a percentage of the fundamental.
./{x)
9. Find the Fourier series of the periodic function f (t) of period T.

J(1)=0 - 2srso, J(1)=S 0s1s2. T=4
10. Find the Fourier series of the periodic function f (1) of period T.
J(1)=r - 2s1s2. T=4
11. Find the Fourier series of the periodic function f (1) of period T.
/(1)= 111 - 2 St:S 2, T=4

9.1 Eigenvalues and Eigenvectors
Ei,genvalues and the corresponding eigenvectors are widely used i.11 many areas of mathematics
and engineering, specially in the analyses of dynamic problems. Eigenvalues are the scalars for
which non-trivial solutions to the matrix equation AX= AX exist. This often is tenned as eigenvalue
problem.
The set of equations represented by the matrix equation AX= ,lX wlhere A is an 11 x II matrix and X
is an II dimensional column vector can be written as
(9.1)
where I is the identity matrix.

I I I I
Equation (9. I) has non-trivial solutions if ,U - A = 0 . A - A.I is called the characteristic
I I
polynomial of A and the equation ,U - A = 0 is called the characteristic equation of A. The roots
of the characteristic equation are called the eigenvalues of A . For each eigenvalue .A.1 , the non-trivial
solution X, is called a eigenvector of A corresponding to the eigenvalue A.1• It can be noticed from
Equation (9. 1) that any scalar multiple of X, i.s also a solution to Equation (9. 1), and, therefore, the
eigenvectors are not unique. ln many applications we utilize the normalized eigenvectors where the
magnitude is always unity.
The matrix equation AX= lX suggests that eigenvectors of A are those special vectors whose
magnitude when multiplied by A may shrink or get enlarged but whose di.rection will either remain
the same or get reversed.
Chapter 9: Eigenvalues and Eigenvectors and their Applications
Example9.1
Determine the eigenvalues and eigenvectors of matrix A shown below.
Solution
First, set up the characteristic equation.
,l -1 2 -4
1i 1- A I= -I ,l - 2 - J =0
0 - 1 ,l+l
i.e., A.l - 2A 2 +I= 0 (9.2)
Solving (9.2), we get Ai= 1.618, ~ = 1 and,¾ =-0.618

The eigenvectors for each corresponding eigenvalues are detemlincd by solving the appropriate
homogeneous equations in the following manner.
For ).1 = 1.618
0.618 2
[
-l
0
-0.382
- I
-]
-4
2.618
l[ l
x,,
X11
Xu
=0
0.6 I 8x11 + 2x12 - 4xu = 0

- X11 - 0.382X12 - X13 =0
x,2 +2.6 l8x 13 =0
The normalized eigenvector corresponding to the eigenvalue A, = 1.618 is
[
x ,,
X12
X 13
l l
=
[ -0.5809
0.7604
0.2904
For~ =I
The normalized eigenvector corresponding to the eigenvalue ~ = I is
[
X1 ,
X22
X23
l[ l
=
0.8018
--0.5345
--0.2673
l[ l
The normalized eigenvector corresponding to the eigenvalue ~ =--0 .6 18 is
X3, 0.8817
X 32 = --0. 1684
[
X 33 --0.4408
Now, we will discuss another method of finding eigenvectors. For this method, we will rewrite
Equation (9. 1) for each eigenvector.
{,\1-A)m' =0 (9.3)
where m1 is the eigenvector for the eigenvalue i!.1•
3
We know Lhat KK-' = I and also K- 1 = ~~
_, adjK
Therefore, KK = K 7ir =I
KadjK = IKII
( i!.11- A) adj( il,I - A) = Ii!.11- All
I
However, i!.11- Al is equal to zero since it, is an eigenvalue.
Therefore, we can write
(9.4)
A comparison of Equations (9.3) and (9.4) sbow that the eigenvector m1 is proportional IO any
nonzero column of adj (il;I - A). ■
Example9.2
Find the eigenvalues and the eigenvectors for the system with a system matrix A.
A=[ - -~
13
~I ~
0
l
Chapter 9: Eigenvalues a nd Eigenvectors and their Applications
Solution
The characteristic equation is
l+5 - 1 0
Il l -Al = 0 A+4 - I =0
13 -1 A
(9.3)
Solving Equation (9.3), we get l, =-0.5607, Ai = - 2.3389 and .A..:i =-6.1004
- 2.9376 -0.5625
[ 10031
adj(A,1- A)= - 13.0409 - 2.4969 4.4533
-44.8518 - 8.5876 15.3162
l
-4.6634 - 2.2327 09546
[
adj(A..il- A) = -12.4099 - 5.9415 2.5403
l
l
- 20.6143 - 9.8696 4.2198
[ 117133 -6.0487 0.9915

adj(~ I- A)= - 12.8897 6.6562 - 1.0911
27.074 - 13.9808 2.2918
All columns of the adjoints are non-zero ones. Therefore, we may pick any column as an eigenvec-
tor. Let us pick the third column from each adjoint.
l[ l
The eigenvector for A, = -0.5607 is
11
xX = 1.003 l
4.4533
12
[ X13 15J](i2
The eigenvector for
[
X21
X22
X23
l[ l
Ai = - 2.3389 is
=
0.9546
2.5403
4.2198
The eigenvector for~ =-6.1004 is
[
X31 ] -
X32
X3J
-
l[ 0.9915
- 1.0911
2.2918
In the following sections, the application of eigenvalues and eigenvectors will be discussed. ■
9.2 State-Space Models

Dynamic behaviour of systems can be modeled by differential equations. In general, nonlinear
differential equations are required to model actual dynamic systems. In many cases, however, linear
approximations can be obtained to describe lbe dynamic behaviour of such systems in the vicinity
of some nominal operating point. The malbematical models of some simple linear dynamic systems
are described below.
9.2.1 Aseries RLC electrical circuit

Figure 9.1 shows a series circuit connecting a resistance R, an inductance Land a capacitance C. A
voltage v1 (t) is applied to the circuit which results a loop current i(r). The dynamic response of this
circuit, i (t ) can be described by a differential equation using Kircl:ithoff's voltage law. According
to Kirchhoff's law, voltage applied in a closed loop at any instant must be equal to lbe sum of all
vol tage drops around that loop.
R L
Figure 9.1 A series RLC circuit
Therefore,
(9.5)
Where
v11 (r) = voltage drop in the resistance, Rat time I
vL (t) = voltage drop in the inductance, Lattimer aud
Ve (1) = voltage drop in lbe capacitance, Cat time/.
dt C
f
v1 (r)= Ri(r)+ L di(r) +..!._ i(t)d1 (9.6)
Chapter 9: Eigenvalues a nd Eigenvectors and their Applications
Differentiate Equation (9.6) with respect to time. The mathematical model of the system is:
2
dv, (I)= Rdi(t) + L d i(r) + i(t)
(9.7)
dt dt dc 1 C
9.2.2 An automobile suspension system

The spring-mass-damper system shown in Figure 9.2 can be used to represent a simplified suspen-
sion system for each wheel of an automobile.
Figure 9 .2 An automobile suspension system
A force applied to the mass, M in the direction of x wiU be balanced by three other forces in the
opposite direction of x. The balancing forces are the gravitational force on the mass, the damping
force due to the shock absorber and the spring force.
The damping force is: F8 (t) = B dx (t) (9.8)

dt
The spring force is: FK (t) = Kx (t) (9.9)
The gravitational force is: Mg

Applying Newton' s Second Law we can write
- Mg - f~ (r) - F8 (t) = M d;:it)
d 2 x(r) dx(r)
or, M ? +B- -+Kx(r)=- Mg (9.IO)
dr dr
9.2.3 Atwo-tank liquid level system

Consider the two-tank liquid level system shown in Figure 9.3.
Figure 9.3 A two-tank liquid level system
Assume linearized How rates. Therefore, the How rates can be considered proportional to the
corresponding liquid heights. If R, and R2 are constant resistance factors of the control valves C,
and C2 then
q, (t) "· (t) ~ "2 (t) (9. U)

I
(9.12)
If V, (t) and V2 (t) are volumes of li,quid in tanks l and 2 respectively and q, (i) is the inflow in the
first tank then
d\'i (t) d { A,I,, (1)}

q, (t) - q, (t) (9.13)
dr dt
(9.14)
where A1 and A2 are respective surface areas of tank 1 and 2.

Substituting Equation (9.11) in Equation (9.13) we get
A, dh,(t) =q,(t)- {"• (t)- /12(1)} (9.15)

dt R,
Substituting Equations (9.1 I) and (9.12) in Equation (9.14) we get
A, dh2 (t) {", (t) - h2 (t)}- l1i (t) (9.16)

- dr R, R2
Equations (9. 14) and (9.15) mathematically model the dynamic bebaviour of this two-tank liquid
level system.
9.2.4 State-space representations

The dynamic behaviour of many physical systems like the ones discussed in Sections 9.2.1 - 9.2.3
can be modeled with the help of a set of n first-order differential equations involving n state variables.
These II state variables completely define the internal behaviour of the system. The state variables
are usually denoted as x, (1) , x2 (1), ... x. (1). The state model of a system is not unique, but depends
on the choice of a set of state variables.
The general state-space model of a linear time-invariant system can be expressed as:
x, (1) A., A12 A... x, (1) 811 Blm u1(1)

i2 (1) ~I An A1n X2(t) Bi, B1,,. u2(1)
= + (9.17)
x. (1) A.1 Ani A.. x. (t) B., B,i,,. u,. (1)
Equation (9 .17) can be wrinen in a concise form as

x(1) = Ax(1)+ Bu(r) (9. 18)
where A is the nxn state matrix and B is the 11x111 input matrix.
It is important to note at this point that although the entire 11-dimensionaJ state vector x (1) characterizes
the complete internal behaviour of a state-space system, it is usually impossible to directly measure
or observe all n components of x (t). The external output vector can be expressed as
Y, (1) c,, cl• x, (1)

Yi (1) C21 Ci. X 2 (1)
= (9. I9)
Y,,(1) c,,, c,,. x. (1)
Equation (9. 19) written in a concise fom1 as

y(1)=Cx(1) (9.20)
where C is the p x II output matrix.

Equations (9. 18) and (9.20) are the state-space representation of a linear, time-invariant system.
9.2.S Transfer function mat11ices and stability

Transfer function is utilized to determine the stability of a dynamic system. For an 11-dimensional
dynamic system the transfer function matrix can be found by appl ying Laplace transform to Equations
(9. 18) and (9.20). The Laplace transfom1 of a vector is the vector o f the Laplace 1ransfom1 of its
elements. TI1e Laplace transform of x (I) and x (1) are as follows:
L{x, (1)} X, (s)
(X(s)]= L[x(t)] = L{ x2(1)} =

x 1 (s)
L{x. (1)} X., (s)
L{i,(t)}
L [:i: (t)] = L{i2 (1)}
L {.i:,. (t)}
where
L { x1 (t)} = sX, (s) - x 1 (0)

Therefore, the Laplace transform of
x (1) =Ax(1)+ B0(1) is
sX (s) -x0 = AX (s)+ BU (s)
or, (s1-A) X(s) = BU(s)+ x0 (9.21)
where I is the unit matrix.

The Laplace transform ofy(t)= Cx (1) is Y(s)= CX(s)
1
Pr,e multiplying both sides of Equation (9.21) by (sl- Af we get
X (s)=(sl- Af' BU (s) +(sl-Af' x0
The output in Laplace domain is
Y(s)= C (sI- Ar' BU (s)+C (sl-Af ' x0
Y (s)= G (s) U(s) +C (sl-Af' x0
where G (s) = C (sI-Ar' B is called the transfer function matrix.

A transfer function matrix relates the transforms of the input and output vectors for zero initial
conilitions.
The inverse of matrix (sl - A) can be expressed as
-1 adj(s.I- A)
(sI -A ) = ~~~~ (9.22)
isl-Al
Using Equation (9.22), the transfer function matrix can be written as,
G Cadj{sI-A) B
(9.23)
(s) Isl-A l
The dynamic system modeled by Equations (9.18) and (9.20) is stable if all the roots of the
characteristic equation isl- A I= 0 lie on the left half of the s-plane. The roots of the characteristic
equation, isl- A I= 0 are called the eige11val11es of the system coefficient matrix (system matrix) A.
That means the eigenvalues of a stable system are all negative.
Example9.3
Consider a 2-dimensional system with the following system and input matrices.
Determine if the system would be s table.
Solution
(s1- A) = [ s -3 ]
2 s +5
(sl-Af' = adj(sl- A) l [ s+ 5 - 2 ]
2
isI-AI s +5s +6 3 s
The transfer function of the system is
G (s) =C (sI- Af' B
G (s) = [, l O ][ s+ S 3 ][ I ] s+ 5
s- + 5s+6 - 2 s O - (s + 2)(s+3)
The denominator of the transfer function is
isl-Al = s2 +5s+ 6 =(s+ 2)(s + 3).

(s + 2)(s+3)=0=0
The roots of the characteristic equation, i.e., the eigenvalues are - 2 and - 3. Both eigenvalues are
negative, therefore, the system is stable. ■
9.2.6 Coordinate transformation and eigenvalues

It has been mentioned earlier that the state model of a dynamic system is not unique. The model
depends on the choice of a set of state variables. It is often beneficial lo define a new state vector z
with the help of a coordinate transformation x = Tz.
The state model of the system described by x= Ax+ Bu and y = Cx will be transformed by this
coordinate traosfonnation in the foUowing manner.
T:i = ATz+ Bu (9.24)
y= CTz (9.25)
PremuJtiply Equation (9.24) by T- 1•
T -'T :i = T -'ATz + T-'Bu
The transformed system with new state vector z is
z = Az +B u with the output y = Cz
where A = T-'AT and C = CT

The use oftbe new state vector (internal state variables) should not aftect the stability (i.e., eigenval-
ues) and the input-output relationship of the system. However, this can be verified by evaluating the
c haracteristic equation and the tran.~fer function matrix of the transformed system in the following
manner.
Characteristic equation:
Isl- A l = lsI -T-'AT I= lsT -'TI- T-'AT I
Because, [ I] is the unit matrix we can write
Utilizing the properties of detennin.ants,
Isl-Al=1T-1{sl - A)TI=1r -1lsI-AIITI.

Isl- AI=IT-'ll 'FllsI-AI=Isl- Al
Therefore, the characteristic polynomial and , hence the characteristic equation remains unchanged.
Transfer function matrix:
The transfer function matrix of the transformed system is
G(s)= CT {T-1(sl-A)T r' T-1B
G(s)=CTT-1 (sl-Af' TT-1B

• -I
G(s)= C (s1-A) B =G(s)
The transfer function remains unchanged by the transformation as well.
The question is, what kind of transformation would be helpful?The particular transformation that we
are interested is the one for which Awould be a diagonal matrix. Why we want a coefficient matrix
I.hat is diagonal? To answer this question, let us have anolher look at the system s tate equations,
particularly Eqnation (9. 17).
x. (1) Au A,a A1,, x, (1) B11 B 1_,,, u 1 (1)

x2(t) A2, An A2-11 X 2 (1) B2, B2., u2(1)
= +
.r.(1) A,., .. . A.. x. (1) s•• 8nm 11,.(1)

A"1
In order to solve for each state, we have to integrate each equation. Integrating the second term on
the right-hand side of the equation should be done with little difficulty. However, the integration
of the first term on the right-hand side cannot be done that easily. Because, we.will have to wai t for
Lhe determination of other state variables. Now consider a coefficien I matrix, A with only diagonal
elements, all other elements being zero.
x1 (I) Au 0 0 x 1 (1) B11 B,m u1 (I)

x2 (t) 0 An 0 X 2 (1) B2, B2"' 112(1)
= +
.r.(1) 0 0 A.. x. (1) B., Blf"' 11., (1)
Each individual equation as shown in the matrix form can be expressed in tenns of its self-terms
onl y. Therefore, each equation can be integrated with little difficulty.
.x1 (1) = A11 x 1 (1) + 8 11111 (1)+ B12ui(1) + · ··+ B,,.11., (1)
x2 (1) = Anx2 (1) + B2I u1 (1) + B12112 (1) + ·· ·+ 8 1mu., (1)
x3 (1) = A33 x 1 (1)+ B31u1 (1) + B3211.i (1) +···+ B1.,.u,. (1) (9.26)
The set of Equations (9.26) represents a system model that contains only the self-terms. All mutual
tenns are zero. This type of system is often called a decoup led system. It is relatively very straight-
forward to carry out mathematical analysis of a decoupled system.
We are, therefore, interested in a particular type of coordinate transformation that will transfonn our
coupled system into a decoupled systeDL We have already discussed that the transfer function matrix
and the characteristic equation wou11d remain unchanged even after this transformation. Assume that
th.is transformation can be achieved with the belp of a transformation matrix M .
Therefore, x = Mz and A=M -'AM =A=diag{A, .~.··· ,..t,.}

The characteristic equation of the system is
isl- A I= isl- AI= dct{diag[(s- A.1) , (s - ..t2 ), • •• (s - ..t.)]}= 0
sl- AI= I-v i- AI= (s - ..t, )(s - ..ti} ... (s - 1,.) = 0

The elements of the diagonal matrix A are the roots of the characteristic equation isl - Al= IsI - A I= 0
and are, therefore, the eigenvalues of the system.
The elements of such a transformation matrix M can be found in the following way.
m,, 111,2
11121 "'22
M=
111,11 111n2 ftlnn
M-'AM =A
MM-'AM =MA
AM=MA
All A,2 A,. m,, T/112 ,n,n m,, llln 111,,, ..t, 0 0 0
Ai, Ai2 Ai. m2, 11122 m211 lll2 1 111 12
"½. 0 ~ 0 0
=
0 0 0
A., An2 Ann ltlnl tnn2 111,m lll,, 1 111n2 ,n,m 0 0 0 ..t,.
Equate columns on both sides. Equating the 2nd column we can write
All A.2 A.. 11112 11112
Ai, A12 A2n 11112 11122

= Al
A., An2 All,, 111n1 111n2
In the same manner equating the ith column we can write
All A,2 A,. 11111 11111
Ai, A22 ~,, 111.i, lllv

=..t, (9.27)
A., A.2 A•• 111,,_, 111,.,

Chapter 9: Eigenval ues and Eigenvectors and their Applications
(9.28)
where m' is Lhe itb colwnn of M .

l ,m' - Am' = 0
(9.29)
Equation (9.29) can be solved to find the ith column of d1e transfom1ation matrix M . Observe diat
associated widi each eigenvalue there is a corresponding colwnn or vector for die transformation
matrix M . These vectors are d1e eigenvectors of die system. The transformation matrix M diat
diagonalizes die system matrix A is called the modal matrix. The state variable z is called canonical
wlnen the coefficient matrix becomes diagonal.
Example9.4
Find die modal matrix diat will transforms die coefficient matrix A in Exan1ple 9.1 into a diagonal
matrix. Verify that the modal matrix transfom1s A into a diagonal matrix.
Solutio11
We already have die diree eigenvalues as
J., = 1.618, ~= I and Ai =-0.618
l[ l[x l[ l [xl[ l
The corresponding eigenvectors are
21 31
X11
x, = -0.5809
0.7604 , x 22 = 0.8018
-0.5345 and x32 = 0.8817
-0.1684
2
[
X13 0.2904 X23 -0.2673 X33 -0.4408
The modal matrix is
M=
-0.5809 0.80 I8 0.8817
0. 7604 -0.5345 -0.1684
[ 0.2904 -0.2673 -0.4408
l
For verification purpose, multiply the following.
M-'AM
1
M- AM =
[ --0.5809 0.8018 0.8817
0. 7604 -0.5345 -0. I684
0.2904 -0.2673 - 0.4408
n 1 - 2 4 ][ --058~ 0.8018 0.8817
l 2
0
I
I -I
0.7604 -0.5345 -0. 1684
0.2904 -0.2673 -0.4408 l
[ 24914 1.5397 4 3946 ][ 1 - 2 4 -0.5809 0.8018 0.88 1' ]
M-'AM = 3.7417 0 7.4833 1 2 I 0.7604 -0.5345 - 0.1684
-0.627 1.0145 - 3.9099 0 l -1 ][ 0.2904 -0.2673 -0.4408
4.0311 2.4914 7. 1106 l [ -0.5809 0.8018 0.8817 l

M-'AM = 3.7417 0 7.4833 0.7604 - 0.5345 -0. 1684
[ 0.3875 - 0.627 2.4165
l
0.2904 -0.2673 - 0.4408
1.618 o o
M-'AM = 0 1 0
[ 0 0 -0.618
M-'AM= A is a diagonal matrix.

•
Example9.S
Find the modal matrix for the system in Example 9.2 with a system matrix A.
Solution
A=
[ -l
0 -4
- 13 l ! l
1 +5 - I 0
111- A I= 0 1 + 4 - I =0
13 - I 1
(9.30)
Solving Equation (9.30), we get A, = - 0.5607, Ai = - 2.3389 and-½ =-6.1004
[ - 2.9376 -0.5625 ....,, l

adj(\I- A)= - 13.0409 - 2.4969 4.4533
-44.8518 - 8.5876 15.3162
adj(1i - A)=
[ -4 6634
- 2.2327
09546
- 12.4099 - 5.9415 2.5403
- 20.6 143 - 9.8696 4.2 198
l
11.7133 -6.0487
>ij(.l,1- A) • [ - 12.8897
o.,,,,
6.6562 - 1.0911
l
27.074 - 13.9808 2.29 18
All columns of the adjoints are non-zero ones. Therefore, we may pick any column as an eigenvector.
Let us pick the third column from each adjoint.
The eigenvector for A, = -0.5607 is
[
11
xXn
x, 3
l[ l= 1.003 I
4.4533
15.3162
l[ l
The eigenvector for A.,- =-2.3389 is
~
X21 0.9546
X22 = 2.5403
[
X23 4.2} 98
[
X 31
X32
X33
l[ l
The eigenvector for ,ti =-6.1004 is
0.99[5
= -l.0911
2.2918
Note d1at the eigenvectors determined above are not normalized ones. i[f necessary, they can, however,
be normalized.
The modal matrix widJout nomializing the eigenvectors is
l
[ 1.0031 0.9915
0.9546
M= 4.4533 2.5403 -1.0911
15.3162 4.2198 2.29 18
l
[ --05601 0 0
M-'AM= 0 - 2.3389 0
0 0 -6.1004
The modal matrix widJ nom1alizing the eigenvectors is
[ 0.0628 -0.1903 0.3639

M,, = 0.2786 -0.5063 -0.4004
0.9583 -0.8411 0.841 l
l
[ --05601 0 0
Ms-'AM s = ~ - 2.3389 0
0 -6.1004
•
9.3 Solving Systems of Differential Equations
First-order Differential Equations

Eigenvalues and eigenvectors can be utilized to solve sets of linear first-order differential equations.
Consider the following three by three system:
j , = Y, - y1 +2y3
Y2=3y, +4yl
Y1 =2y, + Y2
We can write this set of equations as Y= AY
where
l[ l [ l
The eigenvalues of A are I .303, - 2 .303 and 2. The corresponding eigenvectors are:
~.172 ~.557 o
0.891 , ~.467 and 0.894
[ 0.420 0.687 0.447
The modal transformation matrix is:
M=
~.172 ~.557
0.89 1 ~.467 0.89~
[ 0.420 0.687 0.447
l
M diagonalizes A and setting Y = MZ transfonns Y = AY into Z =AZ
where
A=M-'AM
1.303 0
A= 0 - 2.303
[ 0 0
Therefore, Z = AZ can be written as

i., = 1.303;:,
t 2 = - 2.303¼
t1 =2z1
These equations can be solved easily.
z, -- ae•=•
- be-1..303,
Z2 -
Therefore.
ae1.303,
z= be- 1.303,
l
Since Y=MZ
3
-0.172 -0.557 O l [ ae'.30 '
Y= 0.891 -0.467 0.894 be-1 ·:,:n,
[ 0.42 0.687 0.447 ce 2'
In tenns of the components, lhe general solution is
y , = - 0.172ae'-303' - 0.557be-1 303'

y2 = 0.891ae1.3031 - 0.467be-2 = +0.894ce
1 21
y3 = 0.42ae'·303' +0.687be-i.303' +0.447ce0
The arbitrary constants a, band c can be found if initial condition is given. Given an initial condition,
Y (0) = [ 0 0 Ir we have the following.
-0.I72a-0.557b =0
0.89 la - 0.467 b + 0.894c = 0
0.42a +0.687b+0.447c= J
Therefore. constants a , band care - 3.228, 0.997 and 3.738 respectively.
Y, = 0.555euoJ, - 0.555e-1 303'

Y1 = - 2.869e'-303' - 0.465e-2·303' + 3.333e2'
y3 = - !.35el3°3' + 0.684e-2·303' + l.667e0
Problem Set 9.1

Solve the systems of differential equations using diagonalization of the coefficient malrices.
Second-order Differential Equations

Though not directly. eigenvalues and eigenvectors can be utilized to solve sets of linear second-order
differential equations.
Consider the mass-spring system shown in Figure 9.4 where y1 and y 2 are displacements of masses
m,. and 111.i respectively from equilibrium positions. The spring constants are k, and k 2 as shown and
both masses are chosen as I.
k,
111, - - - - - - - - - -
yI
-----
-r-- .
·•
k,
Masses Displaced
Figure 9.4 Mass-spring System

Assume that there are no damping and external forces acting on the sys1em. The motions of the
masses arc governed by the foUowing differential equations.
Y, =- k,y, +k2 (Y1 - y,) = - (k, + k2)Y, +k1Y2
Y2 = - kl (.1•2- Y,) = k2Y, - k1Y1
where y, and y2 are functions of time I and the dots denotes derivatives with respect to time. In matrix
form the equations can be written as
Assume k, = 2 and k 2 = 3. Then the equation in matrix fonn becomes
.. =AY where A= [ - 5 _3 ] .
Y
3 3
Let us try Y =e"'' C as a solution where w wiU be determined and C is an unknown constallt matrix
=
Substituting Y e"''C into the differential equation Y =AY gives ns
(9.31)
Dividing Equation (9.31) bye"'' gives us

AC= 11,2C
or. AC=lC (9.32)
wlnere A=w2
Equation (9.32) shows that C must be an e igenvector corresponding to eigenvalue A for Y =e•·•c
10 satisfy the differential equation Y = A Y . Therefore, the next step would be to solve for the
eigenvalues of A . The eigenvalues are Ai =--0.838 and Ai = - 7.162. Corresponding to 1 1 =--0.838
.
we get ti1e e1genvec1or [ 0.585]
_
0 811
Therefore w, =±J0.838 i, and we get solution of the form
y = a e Jo.8381, [ 0.585 ] + /3 e- Jam 1, [ 0.585 ] (9.33)

1
0.811 0.81 l
Corresponding to 1., = - 7 .162 we get lhe eigenvector [ O. 81 l ] .

- --0.585
Therefore 1112 = ±.J7. 162i, and we g.et solution of the form
y = J7.162/r [ 0.8 Jl ] O - ,'7.162h [ 0.8} I ] (9.34)

2 ye -0.585 + e --0.585
a. p, y, and /j are arbitrary scalars. T hese solutions can also be written as
Y, =[acos(.J0.8381)+bsin(.J0.8381)J[ ~:~~~ ]
(9.35)
Y2 = [ ccos( ✓7.1621 )+d sin{ ✓7.1621 )J[ ~:~~~ ] (9.36)
The general sol ution is:
y = [ 0.585acos( J0.8381 )+ 0.585 bsin( .J0.8381) + 0.81 Iccos( ,J7. 1621) + 0.81 Id sin ( .J7. 1621)
0.81 la cos( .J0.8381 )+ 0.811 bsin ( .J0.8381 )- 0.585ccos( ✓7 .1621 )- 0.585d sin ( ✓7. 1621)
l
In tenns of individual displacement
y, = 0.585acos( ✓0.8381 )+0.585bsin ( ✓0.8381 )+ 0.81 Iccos ( ✓7. 1621 )+ 0.8Ud sin ( ✓7. 1621)
y1 = 0.81 I a cos (.J0.8381 )+ 0.8 1 I bsin ( .J0.8381 ) - 0.585ccos(✓7.1621 ) - 0.585d sin(.J7. 1621)
We can determine the values of a, b. c and d if we know the initial position and initial velocity.
Assllllle the foUowing initial conditions.
Y,(O)
[ y2 (0)
]=[ I ] and [ j , (O)
1.5 j•2 (0)
]=[ 2 ]
3
With these initial conditions. we get a= l .802, b = 3.936, c =--0.067 and d =-0.05.
y, = J.054cos ( ✓0.8381 )+ 2.303siu( ✓0.8381 ) - 0.054cos( ✓7. 1 621 ) - 0.041sin ( ✓7. 1621)
y 2 = l.461 cos ( .J0.838t)+ 3. 192sin {.J0.8381 )+ 0.039cos(✓7. I 62/) +0.029 sin (.J7. 1621)
Problem Set 9.2

Solve the system Y = AY, with A as given below.
L A=[~ ~]. Y(O) =[ ~ l Y(O)=[ ~]
2. A=[ ~ -~ ]. Y(O)=[ ~ ]. Y(O)=[ ~ ]

3. A=[ ~ _: ]. Y(O)=[ ~ l ~ Y(O)=[ ]
4. •·[ I :H Y(O)•[ ll il Y(O)•[
Definitions:
Symmetric Matrix: A symmetric marrix is a square matrix that satisfies AT = A
The eigenvalues of a real symmetric matrix are real.
Orthogonal Mmri:r: A real matrix A is called orthogonal if its transpose AT coincides with the
.
inverse A-'.1.e., AT= A-'.
Am orthogonal matrix has the following properties.

1. The rows (columns) of an orthogonal matrix are orthogonal in pairs.
I.aaa;k = 0 for i ,t: j and

k=I
I,aklakf = 0 for i *j
k=-1
2. The sum of the squares of the elements of each row (column) of an orthogonal matrix is
equal to unity.
" .
I,a,! = I,a! = I
3. The determinant of an orthogonal matrix is equal to ± I.
4. The transpose and the inverse of an orthogonal matrix are also orthogonal matrices.
Orthogonal Matrices and Real Quadratic Forms

" n
A real quadratic form in x,, x 2 , .. · , x,, can be written as L, L,ayx1x 1 , where a,y's are all real numbers.
J=I l=I
This is a polynomial expression similar to 6x~ +4x1x 1 - x: and may arise in many areas including
physics and analytic geometry. In many cases we want to write this polynomial expression in a more
compact form, specifically in self-terms 011I y. This requires a coordinate transfonnation.
The first step towards a coordinate transformation is to arrange the polynomial expression into a matrix
• •
expression. The polynomial expression, I, I,agx,x 1 will include sq11are tenns a11 x~ , a22xi, · · ·, a..x~
}=I /:I
and product (mixed) terms.
( afl + aJI )xrt 1 , i *j and can be written as a matrix product XT AX, where A is a real symmetric
matrix.
x,
X= x2 and the elements of A can be folllld in the following manner.
X
•
A11 =a0 for i= 1, 2 , · ··, 11 and
Alj =(a., +ap)l2, i7"j.
Let 11s consider a quadratic expres sion - 12x~+6x,x 2 + 2xi + 5x,x 3 - 4x2 x 3 + 7xl . This expression
can be written as:
The second step is to find the coordinate system where this quadratic expression can be expressed
in self-terms only.
Theorem: If A is a real symmetric matrix then its modal transfonnation matrix is orthogonal.
Start with the quadratic expression in matrix form, XTAX. Let Q be the modal transformation
matrix that diagonalizes A. The change of coordinate is achieved with the transformation, X = QY.
Therefore,
XTAX=(QY? A(QY)
= YT(QTAQ)Y
= y T(Q-1AQ) Y
•Y'[1: ly I
= 1,i + Ai Yi +·· ·+ 1.Y!
The result, termed as the principal axis 1heorem, illustrates the application of a new axes system
with respect to wb.ich the quadratic form has a particularly simple appearance. The resulting expres-
sion with self-terms is termed as the standard form.
Let us continue with our previous quadratic expression which was written in the following matrix
form.
l
Note that the coefficient matrix A is symmetric.
- 12 3 2.5
A= 3 2 -2
[ 2.5 - 2 7
0.243
Q = 0.934 -0.289
[ 0.263
0.077 -0.967
0.954
0.2 12
0.142
l
The eigenvalues are 2.2 18, 7 .807 and - 13.024 and the corresponding modal transformation matrix is
Note that the transformation matrix. is a normalized one.

The resulting standard quadratic form is 2.218y; + 7.807y: - 13.024Yi·
The coordinate transformation is
x, =0.243y, +o.o,1 1y1 - 0.967y3

X 2 = 0.934y1 - 0.289y2 +0.212y3
X3 =0.263y1 +0.954y2 +0.142y3
The new axes system is related to d1e old axes system in d1e following manner.
y 1 = 0.243x1 + 0.934x2 + 0.263x3

Y2 =0.077x, - 0.289x~ +0.954x3
y 3 =-0.967x, +0.2!2x2 +0.l42x3
Example9.6
Consider the quadratic expression xf - 4x1x2 + xJ. Transform the expression into its standard
fo:rm.
Solution
First write the quadratic form
A=[ - 21 -21 ]
The eigenvalues are 3 and - 1. The modal transformation matrix is Q =[ 0. 7o7 0. 7o7 ].
~.707 0.707
Every 2 by 2 orthogonal matrix Q,=[ ~:;~; ~:;~; ] is a rotation of the plane and represents a
counter-clockwise rotation of 45 Degrees.
Using X = QY, the quadratic expressionXTAX is transformed to its :standard form, 3y: - Yi.
The required transfonnation in coordinate form is
x,= 0.707y,+0.707y2
X2 = - 0.707)'1 +Q.707 y1
The new axes system with respect to the old one is
)'1=0.707x1- 0.707x2
y2 =0.707x, +0.707x2
The equation x: - 4x1x 2 + xJ = K re.presents a conic in the plane. Figure 9.5 shows a contour plot in
its original form. Figure 9.6 shows the conic in its original as well as with respect to the new axes
systems. A contour plot of the standard quadratic form is s hown in Figure 9.7. ■
Figure 9.5 Contour plot of

- 100 0 100 200 300 400 x ,' - 2x,x 1 +x2
, =K
x,
Figure 9.6 Geometrical interpretation of x~ - 2x1x~ + Xi = K
Figure 9.7 Contour plot of x~ - 2x,x2 + .ti = K in standard form

Example9.7
Analyze the conic 6x: - 2x1x 2 + 2xi = 9 utilizing standard quadratic form.
A=[ -I6. - I ]
2
The eigenvalues are 6.236 and 1.764. Using X = QY, the quadratic ex.pression XTAX is transfonned
y:
to its standard form, 6.236 + I. 764 = 9. Yi
? ?
__K_+ Yi =l
1.443 5.102
, '
Y, + Yi
(J.201)2 (2.259)2
In its standard fom1 we can easily recognize the conic to be an ellipse. The intercep1s with Yi and y2
are ±1.201 and ±2.259 respectively.
The modal transformation matrix (which is orthogonal) is
Q =[ 0.973 0.23 ]
-0.23 0.973
Every 2 by 2 orthogonal matrix is a rotation of the plane and Q=[ ~t~! 0.23
_
0 973
] represents a
counter-clockwise rotation of about 39.83 Degrees.

The required transfonnation in coordinate fonn is
x, = 0.973 y, + 0.23 Yi
X2 =-0.23)'i +0.973y2
The new axes system with respect to the old one is
Yi =0.973 x, - 0.23x2
Y2 = 0.23x1 + 0.973x2
Figure 9.8 shows a contour plot of the conic 6x~ - 2xix 2 +2.t; aa 9 in its original form. Figure 9.9
shows the conic in ilS original as well as with respect to the new axes systems. A contour plot of the
conic in its standard quadratic fonn is shown in Figure 9.10. ■
10
•
•
4
-2
-4
-•
-8
- 18
- 0 -8 -<l ... -2 0
• •
Figure 9.8 Contour plot of 6x: - 2x1x 2 + 2xJ = K
Figure 9.9 Geometrical interpretation of 6x: - 2x,x1 + 24 = 9

Figure 9.10 Contour plot of 6x; - 2x1x 2 + 2.rff = K in standard form
Problem Set 9.3

Write the following quadratic expressions as XTAX and then express them into their standard forms.
1. x~ + 2x,x2 + 6xJ
.. + x. 2
2 . x 12 - 4 x,x~ 2
_J
CHAPTER 10
8 DA R ry A u p O8 L Ms-+-----+---+--- ----+--<-t-1
-f.JA~R=fI-A-~ D FFE ENT AL

-.------+-----+~---.-------+---I--- ~,U A'I OlfS
Boundary-value problems are differential equations with conditions imposed at some boundary
and initial points. Boundary-value problems arise in many areas of physics and engineering. In
most cases, boundary-value problems do not have any generalized solutions. Their solutions can be
approached by following the laws of physics. It is, however, very complicated to prove the existence
of generalized solutions to such boundary-value problems.
10.1 Existence of Solutions to Boundary-value Problems

Theorem
Suppose the functions/./,- and J,.
y
in the boundary-value problem
y'=.f(x, y,y'), a~x~b. with y(a)=Y. and y(b)=Yb,
are continuous on the domain

D ={(x, y, y') I a ~X ~b. - oo<y<oo, - oo<y' <=} and
f, (x, y, y') > 0, for all (x, y, y') e D and
a constant M exists such that

l.t,. (x, y, YJI~ M , for all (x, y, y') e D,
then the boundary-value problem has a unique solution. •

Chapter 1 O: Boundary-Value Prob lems and Partial Different ial Equations
The boundary conditions y(a)=Y. and y(b)= Y,, are conditions imposed on y (x). They are called
Dirichlet boundary conditions. In some cases boundary-value problems have conditions on the
derivative of y(x), e.g. y' (a)= a and y'(b) = {3. These boundary conditions on derivative are called
Newmann boundary conditions.
10.2 Shooting Methods

In shooting methods, boundary-value problems are convened to initial-value problems. Shooting
methods can be applied to both linear and non-linear boundary-value problems. Although there is no
guarantee about the convergence of shooting method~, they are easy to implement.
The main step of a shooting method is to guess the unspecified initial value and apply techniques that
are used to nwnerically solve initial-value problems. The solution is then verified at the boundary
point and the deviation is calculated. Based on the magnin1de of the deviation, the unspecified initial
value is adjusted and the steps are repeated until the deviation becomes less than a given tolerance.
10.2.1 Linear shooting method

Solution to a linear boundary-value problem y' = f (x,y, y') can be obtained by determining solutions
to two initial-value problems, each time guessing the unspecified initial value. For given boundary
conditions y(a) = Y. and y (b) = Y;. for a~ x $ b, we solve the two folJlowing initial-value problems.
y';;;; .f (x,y,y'), a $x $ b, with y(a) ;;;;y• and y'(a);;;; p,. and
r' =f(x,y,y'), aSxSb, wilh y (a)=Y.and y'(a) =p2 ,

where p, and p2 are the respective guess values for the unspecified initial value of y' (a).
Let y(b) =q, for y'(b) = Pi and y(b) =q2 for y'(b) = Pi·
The initial value y'(a) = p that will satisfy the boundary value y(b) =Yb can be obtained as
p= Y. (P2 - P1) - (Cf,P2 - P1q2) ( 10.J)

(q2 - q,)
Example 10.1
Solve the following boundary-value problem
y'=2y'+ y - 6cosx - 2sinx, 0~xSn/2 y(0)= l and y (n/2)=2
Compare the results with the exact solution y = cosx+2sinx , 0 Sx $ n/2.
Solutio11
Substitute z, = y and Z2 = y' = z;.
Set the initial conditions as z, (0) = y (0) = l wh.ich is already given.
Let the second initial condition be z2 (0) = y' (0) = 1. If this initial condition does not satisfy the
boundary condition y(n /2) = 2, we will try a different one in the second trial.
Chapter 10: Boundary-Value Problems and Partial Differential Equations
z; = y' = z, + 2z1 - 6cosx - 2sinx

Applying Euler's fonuula, we have the following sequence.
z,.1 = Zu . , + hz1.1-,
Z21 = Zv. , + h ( 'u-, + 2zv. , - 6 cos x,_,- 2 sin x 1_ 1 ).
The initial values in sequence form are

Z1.o =I
Consider IO steps. Therefore, h = n:/ 20. We get

y(0)=l, y' (0)=l, y(n/ 2)=- 6.12508577
The initial condition of ;:1 (0) = y'(0) = I does not satisfy the boundary condition y(n/ 2) = 2.
Therefore, try z1 (0) = y' (0) = 3. We get
y(0)=l, y' (0)=3, y (n/ 2)=13.24464139
ObviotL~ly, none of our two choices of the initial slope satisfied the boundary condition y(n/ 2) = 2.
However, by using Equation ( I0.1 ), we can utilize their solutions at the boundary point to detennine
the required slope.
The required initial slope is
2 (3 - l) - {(-6.12508577) 3- 1(13.24464139)}
p= {13.24464139- (- 6.12508577)}
p = 1.83894685
Therefore, wilh y(O) ;;;; l, y'(O) ;;;; 1.83894685, lhe solutioa to the boundary-value problem is showa
in Table I.
Table 1 Solution to the boundary-value problem using Euler's technique
n X Y. y .,,,.,, Absolute E rror

0 0 l l 0
l 0.15707963 1.288861 I 1.30055727 0.01169617
2 0.31415927 1.55222789 1.56909051 0.01686262
3 0.4712389 1.78269236 l.79898752 0.01629517
4 0.62831853 1.97349789 1.9845875 0.01108961
(Co111inued )
Chapter 1 O: Boundary-Value Problems and Partial Differential Equations
Table 1 Solution to the boundary-value problem using Euler's technique (Co111i1111ed )
II X Y. Yao,, Absolute Error

5 0.78539816 2.1186291 2.121320.34 0.00269124
6 0.9424778 2.21286291 2.20581924 0.00703426
7 1.09955743 2.25170206 2.23600355 0.0156985 1
8 1.25663706 2.23137226 2.21Jl3003 0.02024223
9 1.41371669 2.14853256 2.13181115 0.01672141
IO 1.57079633 2.00000003 2 0.00000003
For better accuracy, we can utilize Runge-Kuna method of order four. For the same step size h = n/ 20
we get
y(O)=l, y'(O)=l, y(ir/ 2)=- 13.48910815
y(O)=l, y'(0)=3, y (n/ 2)=17.491 13667
Therefore. the required initial slope is

2(3 - 1) - {(- 13.48910815)3 - I (17.491 I 3667)}
p {17.491 l3667- (- 13.48910815)}
p= 1.99993452
Therefore. with y (O) = 1, y' (0) = 1.99993452, the solution to the bmrndary-value problem is shown
in Table 2.
Table 2 Solution to the boundary-value problem using Runge-Kutta method of order four
II X Y. Y,XM1 Absolute Error

0 0 I l 0
I 0.15707963 l.30055688 J.30055727 0.00000039
2 0.31415927 1.56909026 1.56909051 0.00000025
3 0.4712389 1.79898788 J.79898752 0.00000036
4 0.62831853 1.98458877 l.9845875 0.00000127
5 0.78539816 2.12132271 2.12132034 0.00000237
6 0.9424778 2.20582267 2.20581924 0.00000343
7 1.09955743 2.23600774 2.23600355 0.00000419
8 l .25663706 2.21 l l3433 2.2 1113003 0.00000430
9 l.41371669 2.]3181443 2.13181115 0.00000328
10 1.57079633 2.00000049 2 0.00000049
•
Theorem
If p(x), q(x) and r(x) are continuous fw1ctio11s on [a, b] and q(x) > 0 on [a, b], then the linear
boundary-value problem of the following form
/' =p(x)y'(x)+q(x)y(x) +r (x), a._,;x._,;b, y(a) =Y,, and y(b) =Yb,always hasasolution. ■
The solution to the original boundary-value problem can be written as a linear combination of two
so lutions as
(10.2)
where
y, (x) is the solution to the initial-value problem
y'=p(x)y'(x) +q (x)y(x) +r(x), a._,;x._,;b, y(.a) =Y. and y'(a) =O, (10.3)
and y 2 (x) is the solution to the initial-value problem

yK=p(x) y'(x)+q(x) y(x), a._,;x._,;b, y(a) =O and y'(a)= l. ( 10.4)
To verify, differentiate the proposed solULion twice.
(10.5)
(10.6)
Since y, (x) is the solULion to (10.3) and y2 (x) is the solution to (10.4), we can substitute them in the
original equation. TI1erefore,
y'(x) = p(x)y; (x)+q(x)y, (x)+ r(x)+ Yb ~~~t) {p(x)y; (x)+q(x)y2 (x)}
YK(x) = p(x){Yt (x) + Yb - Y, (b) y; (x)}+q(x){Y, (x)+ Yb - Y, (b) y (x)}+r(x)

2
Y2 (b) Yi (b)
y'(x) = p(x)y'(x)+q(x)y(x)+r(x)
Example 10.2
yK= 2y' + y - 6cosx - 2sinx, Q._,;x ._,; n/2 y(O) =l and y(n /2) = 2
Compare the results with the exact .solution _v = cosx + 2sinx, 0 ._,; x ._,; n/2.
Solution
Convert the boundary-value problem to t\vo following initial-value problems.
y'=2y' + y - 6cosx - 2sinx. 0,S,x<S,n/2 y(O) =land /(0) =0, ( I0.7)
and y' =2y'+y, O<S,x<S,1tf2 y(O) =Oand y'(O)= I (10.8)
Use Runge-Kuna method of order four, and JO steps to solve the two initial-value problems.
Combine the solutions of (10.7) and ( 10.8) using Equation ( !0.2) to obtain the solution of the
boundary-value problem. Table 3 shows the results of the two initial-value problems and the
boundary-value problem.
Table 3 Results of the boundary-value problem using Equation ( I0.2)
11 X Y. Y uat•r Absolute E rror

0 0 l 1 0
1 0.15707963 1.15598657 1.30055727 0.1445707 1
2 0.31415927 1.28824668 1.56909051 0.28084384
3 0.4712389 1.39623968 1.79898752 0.40274786
4 0.62831853 1.48124125 J.9845875 0.50334626
5 0.785398 16 1.54687614 2. 12 132034 0.57444422
6 0.9424778 1.59985288 2.20581924 0.60596637
7 1.09955743 I .65100251 2.23600355 0.58500104
8 l.25663706 1.71677070 2.21113003 0.4943593 1
9 1.41371669 J.82138245 2. 13181115 0.31042867
LO 1.57079633 2.00000000 2 0.0
The technique based on Equation ( I0.2) is susceptible to round-off errors. Jn some cases better
results can be obtained by applying the technique in reverse order. In reverse order Lhe two ini-
tial-value problems become
y' =p(x)y'(x)+q(x)y(x) +r(x), a<S,x<S,b, y(h) =Yh and y'(b)=O, (l0.9)
y'=p(x) y'(x) +q(x) y(x), a<S,x<S,b, y(b) =O and y'(b) =l. (10.10)
Table 4 shows the results of the two initial-value problems and the boundary-value problem when
solved in the reverse order, again in 10 steps.
Even though the technique applied in the reverse order produced better resul ts than the results
obtained in the normal order, this for many applications may still be unacceptable. For such cases.
you may reduce the step size or apply technique shown in Equation ( 10.1).
Table 4 Results of the boundary-value problem solved in the reverse order
n X Y. Absolute Error
y """'
0 0 I I 0
l 0.15707964 l. 27524851 l.30055728 .02530877
2 0.31415927 1.52001172 l.56909052 .04907880
3 0.47123891 1.72872238 l.79898754 .07026516
4 0.62831855 L.8969l054 1.98458752 .08767697
5 0.78539819 2.02141034 2.12132036 .09991001
6 0.94247782 2.10057931 2.20581925 .!0523994
7 1.09955746 2.13454529 2.23600355 .10145826
8 l.256637 JO 2.12550503 2.2 1113002 .08562498
9 1.41371673 2.0781 l084 2.13I81112 .05370028
JO 1.57079637 2.00000000 2 0.0
Problem Set 10.1

1. The boundary-value problem
y, = 2 y'+ y + 2e, - e,
2
' 0<
_x<
_ 2, y(O)= 0 and y(2)=4.67 l
has the solution y(x) = e 2' - e·'. Use the linear shooting method that uses Equation (JO. I) and
h = 0.2 to approximate the solution and compare the resuJts to actual solution.
2. Repeat Problem l with h = O.l.
y' = y' +2y - cosx - 3sinx, - 1:;;x ::;o, y(- J) = I.8768Jl and y(O)= l
has the solution y(x) =sinx+e·•. Use the linear shooting method that uses Equation (10.2)
and h = 0.1 to approximate the solution and compare the results to actual solution.
4. Repeat Problem 3 with h = 0.05
5 . Solve boundary-value problem
y'= 2y'+3y+2 - 4x - 3x 1 , o::;x::;2, y(O)=Oand y(2)=4
using the linear shooting method that uses Equation (JO. I). Use (a) h=0.2. (b) h=O.l.
6. Solve boundary-value probJ.em
y'=2y'+3y+2 - 4x - 3x 1 , o::;x::;2. y(O)=Oand y(2) =4
using the linear shooting method that uses Equation (102). Use (a) h = 0.2, (b) h = 0.1.
10.2.2 Nonlinear shooting method

Solution to a nonlinear boundary-value problem y• = f (x,y,y') cam be obtained by having solu-
tions, each time improving the unspecified initial value. For given boundary conditions y(a)= Y. and
y( b) = Yh for a:<,; x :<,; b, we solve the two following initial-value problems.
y'= f(x,y,y') . a:<,;x:<,;b. wi th y(a) =Y. and y'(a)=p0 • and
y•= f (x,y,y'), a :<,;x:<,;b, with y(a) = Y and y'(a) = Pt ,
0
where p0 and p1 are the respective guess values for the unspecified initial value of y' (a).
We can then use the Secant method to find an improved guess value for the unspecified initial value
of y' (a) using the following sequence.
{y(b, Pt-,)- Yb}(Pt-,- Pt.i)

k = 2, 3, .. .
y( b, P.t-,)- y(b, P.t-1)
Example 10.3
y'= - y'y+3y3, - 2:<,;x:<,;0, y(- 2)=1/3,andy(O)= I/ 5

1
Compare the results with the exact solution y = - - , - 2 :<,; x :<,; 0.
x+5
Solution.
Convert the boundary-value problem to initial-value problems. Use two initial guess values p 0 and
p, for lhe unspecified slope and use Runge-Kuna method of order four to solve the initial-value
problems.
y' (a) = p 0 , and y' (a) = p1
The corresponding calculated bow1dary values are:

y(b, Pt. J= y(0, p0 ) = y(0, 0.4)=4.60458670, and
y(b, Pt-,)= y(0, p,) = y(0, 0.5) =14.25838038

Now, utilize the calculated boundary values to find an improved g-uess value for the unspecified
slope using Secant method.
, = 0.5 (14.25838038 - 0.2)(0.5- 0.4) 0.35437456

p_ 14.25838038 - 4.60458670
y(0, p1 ) = y(0, 0.35437456) = 3.34885134

{y(b, P2) - Y.}(A - P,)

PJ=P2
y(b, Pi) - y(b, P1)
0 35437456 (3.34885134 - 0.2) (0.35437456 - 0.5) 0.3 1234224
P3 = . 3.34885134 - 14.25838038
and soon.
p 10 = - 0.I5 170578
We can accept p 10 = - 0.15 170578 as the initial slope (unspecified initial value) to provide the
solution to the boundary-value problem with a tolerance of 10· 1 • The results of the boundary-value
problem obtained with this initial value of y'(- 2) = - 0.15 I70578 are shown in Table 5.
Table 5 Results of the nonlinear boundary-value problem using Secant mell1od
II X Y. Y,,,.,, Absolute Error

1 - 2.0 0.33333333 0.33333333 0.00000000
2 - 1.9 0 .31893488 0.32258065 0.00364576
3 - 1.8 0.30594832 0.31250000 0.00655168
4 - 1.7 0.29420082 0.30303030 0.00882948
5 - 1.6 0.28354800 0.29411765 0.01056965
6 - 1.5 0.27386836 0.28571 429 0.0LL84593
7 - 1.4 0.26505896 0.27777778 0.01271882
8 - 1.3 0.25703202 0.27027027 0.01323825
9 - J.2 0.2497123 I 0.26315789 0.0.1344559
IO - 1. 1 0.24303496 0.25641026 0.01337530
II - 1.0 0.23694381 0.25000000 0.013056 18
12 ~-9 0.23139004 0.24390244 0.01251240
13 ~ -8 0.22633101 0.23809524 0.0LL76423
14 ~-7 0.22172936 0.23255814 0.01082877
15 ~-6 0.2 1755229 0.22727273 0.00972044
16 ~-5 0.2 1377086 0.22222222 0.00845137
17 ~.4 0.2 1035953 0.21739130 0.00703177
18 ~-3 0.20729571 0.21276596 0.00547025
19 ~-2 0.20455936 0.20833333 0.00377397
20 ~- 1 0.202 13272 0.20408163 0.00194891
21 0.0 0.20000002 0.20000000 0.00000002
■
Problem Set 10.2

y'=/, - 1$ x$0, y(- J)=2andy(0)=1
2
has the solution y(x)= - - . Use I.be nonlinear shooting method with h=0. land with a tol-
2+x
erance of 10-- to approxima.te the solution and compare the results to actual solution.
2. Repeat Problem l with h = 0.05
2
y• = ¼(y') + / - l-4sin(2x), 0 $x :=; n/2, y(0)= 0 and y(n/2) =0
bas the solution y(x) = sin (2x). Use the nonlinear shooting method with h =0. l and with a
tolerance of lo-- to approximate the solution and compare die results to actual solution.
4. Repeat Problem 3 with h = 0.05.
10.3 Partial Differential Equations

Many physical and engineering problems are defined in space and time. Their models, therefore,
are functions of at least two variables and call for partial differential equations to mamematicaJly
ex.plain dlcir behavior. Temperature distribution and mechanical vibration are some well-known
phenomena that are governed by me laws of physics and arc expressed in me form of partial
differential e-quations. These partial differential equations come wim initial as weU as boundary
conditions. Only a few of these partial differential equations can be solved using analytical tech-
niques. Many of Lhcm can be solved as boundary-value problems. The availability of high-speed
computers has made it practical to use finite difference and finite-element medlods to solve partial
differential equations.
Following are some classical partial differential equations, where II is assumed to be a function of
two or more variables I (time), x and y (spatial coordinates).
one dimensional wave equation
two dimensional wave equation
one dimensional heat equation
two dimensional heat equation

two dimensional Laplace equation
two dimensional Poisson equation
The order of a partial differential equation is the highest order of deriv ative which appears. A partial
differential equation is called linear if the unknown function and the partial derivatives are of the
first degree and at most one of these appears in any given term. Otherwise, the equation is caJJed
nonlinear.
A general Linear partial differential equation of order two in two variables can be expressed as
A11,, +2Bu..,, +Cu,., + D11, +Eu, + Fu= G
where A , B, C, D , £ , F and Gare functions of x and y and

011 i)u i)211
u~ = - , u \l = - and u.n. = - -.
i)x · i)y · i)x i)y
The equation is caJJed homogeneous if G = 0. Partial differential equations are classified into three
groups:
Elliptic PDE: if B 2 - 4AC < 0
Parabolic PDE: if 8 2 - 4AC=0
Hyperbolic: B2 - 4AC>0
Only a very few simple boundary-value problems for linear partial differential equations can be
solved using analytical techniques. The analytical techniques utiJize !he techniques of the separa-
tions of variables, superposition principle and Fourier series expansion. Due to the Limited use of
these analytical techniques, numerical solutions are sought to solve a vast majority of the physical
problems.
Theorem:Superposition Principle
If 111 and 112 are solutions of a Linear homogeneous partial differential equation, then any linear com-
bination of 111 and 112
11=c,11, +c2t½ is also a solution,
where c, and c2 are constants. In addition, if u, and 11 2 satisfy a linear homogeneous bow1dary
condition, then the Linear combination 11 = c ,111 + ~ u2 will also satisfy the boundary condition.
10.4 Solution to PDEs: Method of Separation of Variables

o·ne dimensional wave equation
Let us consider a one dimensional wave equation of a vibrating string. Consider a string is fastened
at x = 0 and at x = L. The string is free to vibrate in a fixed plane. 11 (x,r) denotes the position of the
point x on the string at time I. 11 satisfies the one dimensional wave equation
,iu =c2<'l11
- - (IO.II)
at1 i)x 2
The boundary conditions are: u(0,1) = 0 and tt(L ,t) = 0 for all I> 0.
011
The initial conditions are: u(x,O) = f (x) and (x,O) = g(x) for O< x < L.
cJr
The boundary conditions state that the ends of tbe string are held fixed at all time. The initial condi-
tions give the initial position (shape) of the string f (x), and its initial velocity g (x).
u
u(x, 0) = f(x)
X
0
Figure 10.1 Initial shape of a stretched string
Let us assume that u(x, t) can be ex:presscd as a product of two functions.
u(x,t)=X(x)T(r) ( 10.12)
where X ( x) is a function of x alone and T (t) is a function of I alone. The problem, therefore,
becomes finding X (x) and T (t). Differentiating u(x,t) = X (x) T(t) with respect to x and I we get
Substituting these to Equation (10.11), we get
XT'= c1 X'T.
Dividing by c 1 X T, we get
T' x•
(10.13)
c 2 T= X
The left side of Eqn. ( 10.13) is a function of I alone and the right side is a fllllction of x alone. This
implies that the two variables I and x have been separated. The two sides of Equation ( 10.13) are
two separate functions, one solely a function of I and the od1er solely a function of x. Since r and x
are independent of each other, die only way die two sides become equal if they bodi are constant
and equal.
~• =k and x• =k, where k is an arbitrary constant and called I.he separation constant.
c-T X
X" - kX=0
We have to separate the variables in die boundary conditions using Eqn. (10.12).
X(0)T(I)=0 and! X(L)T(t)=0 for all t>0.
If X (0) ;t: 0 or X ( L) ;t: 0 then T (r) must be zero for all I and, therefore, u is zero. To avoid this trivial
so lution, we set X (0) = 0 and X(L) = 0.
The boundary value problem in X becomes
X" - kX =0, X (G) = 0 and X(L)=0
Let us solve the bow1dary value problem in X first. Assume k is positive. say k = µ 2 with µ > 0.
Therefore, x• - µ 2 X =0 and its general solution is: X (x) = c,eP' + c1 ,,µ,
To satisfy die condition X (0) = 0 and X (L ) = 0, we need both arbitrary constants c, and c2 equal to
be zero. To avoid this, discard k = µ 2 and instead, try if k = - µ 2 would provide a solution.
With k = - p 2, the equation for X becomes
The general solution can be written as
X(x) =c, cos(µx ) +c1 sin(µx).
The condition X (0) = 0 implies that c, = 0. Therefore,
X (x) = c 2 sinµx.
The condition X (L) = 0 implies that c1 sinµL =0. To avoid the trivial solution X = 0, we set c2 = 1
and sin(µL):a:O.
Chapter 1 O: Boundary- Value Prob lems and Partial Different ial Equations
sio(µL)=O implies that ,uL=mr, where II is any integer.

. 111C
Tl1erefore,µ= -
L
11
and, X(x)=X. =sin 1C x, n= 1, 2,3, ... is the solution for X.
L
The separation constan1 is
,
k= - µ - = -
,
(L )
l/1C -
With lb.is separation constant, we now proceed to solve for T.
The general solution of this equation is
lllf ) . ( /11C )
T(t) = a,. cos ( c L t +b. sm c L t where 11 = 1, 2, 3 , ...
1
T~ =a. cosA-,,t+b,, sin A.I where i. =c : and 11 =],2, 3, ...
Combine the solutions of X and T. we obtain a set of solutions for (10. 11) that satisfy the corre-
sponding boundary conditions.
1
u. (x,r) = sin( ~ x ){a,. cosA.r +b. sinA.1 ), 11 = I, 2 , 3 ....
Amy linear combination of the above wiU satisfy Equation (10. J I) and the boundary conditions.
Therefore, by superposition principle. we have an infini1e linear combination as a possible solution
that will satisfy (10.ll), the boundary conditions and the initial conditions.
u(x,r) = I, sin( nn x)(a,. cos A.r + b. sin 1.r) ( 10.14)

n= l L
We now detemtlue the unknown coefficients a,. and b. such that Lile function u(x,t) satisfies the
initial conditions. At I= 0 we get
u(x,O)=f(x)= I,a.siu("1C x), O<x<L ( 10.15)

•=• L
Expression (I 0. 15) is the Fourier series of f (x) valid between O < .)( < L (caJJed half-range expan-
sion). Therefore, the coefficients a,. are the sine coefficients and can ·be determined as
a,.= 2LJf l f(x)sm. (/l7C

0
L x) dx, 11=1,2 ,3, ...
The second initial condition can be ·used to detennine b•. Differentiate the series in Equation ( 10.14)
term by term with respect to t,
dll ~~ sin ("1C

-:;-= - .r)( - A,.a,.sinA.t+A.b.cosA. 1) 0
Q[ IJ-: 1 L
Now sett=0
~ 1=0
= g(x) = ~ . (ll7C
-'-' A,,b.sm
n=I
- x)
L
Silllce this is the half-range expansion of g(x},
2 fL . (/11( )
A- b.= LJ
0
0
g(x)sm L x dx, 11=1 ,2,3, . ..
Therefore,
2 fL •
b. = A.L Jo g(x)sm L x d.x
(/11( )
where 1. "'c n1C as dctennincd before.

L
b. = - 2 . (f/1(
J. Lg(x)sm - x )dx, 11 = I , 2, 3, . ..
crm O L
Example 10.4
A string of length, L=2 is stretched along the x-axis with its both ends fastened. The string is setto
vibrate from rest by releasing it from an initial shape given by f (x) and its initial velocity g(x) = 2.
Assume that the s tring vibrates onl y on a fixed plane, and u(x,1) denotes the transverse d isplacement
at time 1;:;: 0 of the point on lhe string at position x. u satisfies the equation
subject to the bow1dary conditions: 11 (0,1) = 0 and" ( L,t) = 0 for all r > 0.
2x if 0SxS0.5
f(x)=
l I if 0 .5 <xS l.5
4 - 2x if i.5< x $2
Using the analytical solution, graph the shape of the string at 0.1 s, 0.2 s, 0.3 s, 0.4 s, 0.5 s, 0.6 s,
0.7 s, 0.8 s, 0.9 sand I s.
Solution
Frnm the equation
a111 =1 60111 O<x< L, r>O

01 2 ox 2
we get c =4
2 fl f(x)si m(/11C
a,.= LJ L x ) dx,n=l , 2 , 3, .. .
0
f /(x)s in
a.= 10
2 (lllr x )dx,n=l,2,3, ...
2
r sm
b,.= -L- J,
1
• ( -nn x ) dx, 11=1,2,3, ...
2,m O 2
The displacement as a function of position x and timer is
u(x,1) = ; srn - . (r:,,r

L x )( a. cos(CTl1f
Lr ) + b. sin (Cl/7t
Lr ))
u(x,r) = I,sin(nn
•=I 2
x)(a. cos(2nm )+ b. sin(2nm ))
We used the first 24 harmonic components to calculate the displacement. Figure I0.2 shows the
displacement of the vibrating string.
1.A ...
1.1 1.1
0.1 0.1
1!
e OA I OA
I I
Ii
OR
.(),3
.0.1 .o.r
-1R -1R
0.1 sec: 0 .2 sec
•1A
OR 0.2 OA 0.6 0.8 1.0 1.2
Potlllon on the sering
... 1.8 1.8 2,0
.u
OR 0.2 OA 0.8 0.8 1.0 1.2
Podion on th~ string
... 1,6 1.8 2,0
1A u
1.1 1,1 OAsec
0 ,1 0 ,1
J•
0
OA
0.0 It OA
o.o
:t0• .0,3 ~
0
-0,3
.0,1 .0,7
•1R •1/J
e.s11e
.1..
0.0 0.2 OA 0.8 0.8 1.0 1.2
Po:dl)on on the string
... u 1.8 2.0
.1.4
0.0 0.2 0A OJI 0.8 1.0 1.2
Position on the string
... u 1.11 2J)
Figure 10.2 Comi1111ed

IA IA
1.1
o.s sec t .l sec
l
1.1
0,1 0.1
OA
0.0
i•" 0A
0.0
.0.3 } .();3
0
.0.1 .0.1
,1.0 .11)
.u •IA
0.0 0.2 0.A 0.6 0 .8 1.0 1.2 1.A 1.6 1.8 w 0.0 0.2 0.A 1).8 0.8 1.0 1.2 u 1.8 1.8 2.0
Poa-ftlon on the string Po~ on the thing
u u
1.1
e.7 sec 1.1
0.1 0.1
I 0A
I 0.A
.
ff
~
'5
0.0
.0.3 i
0.0
4.3
i3
.0,1 .0.1
-1.0 .,.o 0.1 sec
·1A
0.0 0.2 0.A 0.8 0.8
"' 1.2 1A u 1.8 2.0
-1A
0.0 0.2 0.A G.8 0.9 1.0 1.2 ,. ... 1.8 2 .0
Potfflon on Ole string PotlU.On on the smg
IA u
1.1 1.1
O.T 0 .7
~ 0A ~ 0A
.."e 0.0
~
.! 0.0
0.
i3• .03 :
i3
.().3
.0,1 -0,7
·1.0
0 .1 ••c .... 1 .0 sec
•IA
0.0 0.2 0.A o.e 0.8 1.0 1.2 u 1.8 1.8 2.0
.....0.0 0.2 0.A 0.6 0.8 1.0 1.2 ,. ... 1.9 2.0
Position on lhe string Poulon on the ttrtng
Figure 10.2 Displacement of a vibrating string
Problem Set 10.3

In the one dimensional vibrating string where the two ends are held fixed. determine using the ana-
lytical technique the shape of the string at 0. 2 s, 0.4 s, 0.6 sand 0.8 s for the following conditions.
1. L=l ,f(x) =0.2sin(nx), g(x)=O andc=l/2.
2. L = I, J(x)=sin(nx), g(x) =0 and c =l/ 2.
3. L = I, f (x) = si11(.1rx), g(x) =0 and c = I.
4. L = I, f (x) = 0.2sin(3n x), g(x) =0 and c= L
5. L = I, J(x) = 0.2sin(nx), g(x) =sin(nx) and c = J/2.

6. L = I, f (x) =sin(nx), g(x) =sin(nx) and c = 1/2.
7. L=l, f(x)=sin(.n:x). g(x) = I and c=I.
8. L= l, f(x)=0.2sin(3nx) , g(x)=2 andc=L
9. L=2, g(x)=2,c=l/2and
2X if O$ X $ 0.5
f(x)=
lI if 0.5<x~l.5
4 - 2x if 1.5 < X $ 2
10. L=2,g(x)=l,c=l/2 and
2x if O$ X $ 0.5
/(x)= I
l if 0.5<x$1.5
4 - 2x if l.5< x$2
11. L=2, g(x)=xsin(nx),c=land
2X if O$ X $ 0.5
f(x)=
l1 if 0.5<x$1.:5
4 - 2x if l.5<x$2
12. L= l,g(x) =l,c =land
J2x if0Sx$1
/(x)=14 - 2x if l<x$;2
13. L= l,g(x) = sin(nx),c=land
2x if0Sx$1
f(x)= { 4 - 2x if l<x$;2
One dimensional heat equation

Consider temperature distribution in a uniform bar of length L with insulated lateral surface and no
internal sources of heat, subject to certain boundary and initial conditions.
J U X, (; t)
oo oo
.
- . --
t
X
0 ' I
L
Figure 10.3 Insulated bar with ends kept at zero degree
Let 11(x,1) represent the temperature at point x of the bar at time 1. The ends of the bar are beld at
constanttemperature 0. The initial temperature distribution of the bar is 11 (x,0) = f (x). Find ll(x,1)
for O< x < L , I > 0 where II satisfies the one din1ensional heat equation
011 ? d2 LI
- =C - (I 0. 16)
d/ c)x 2
The boundary conditions are: 11(0,1)=0 and u(L,r) =0 for all r > 0.
The initial condition is: u(x,0) = f (x) for 0 < x < L.
Let us assume that ll(x.r) can be expressed as a product of two functions.
ll (x ,r) = X ( x) T (t) where X ( x) is a function of x alone and T (t) is a function of I alone. Substitute
ll(x,1) = X (x) T(t) in the heat equation.
XT' =c2 X'T
Therefore, to hold the equality, ~, =k and x• =k, where k is the separation constant. With the
c·T X
separation of variables, we get the following two ordinary differential equations.
T' - kc 1 T=0 and x• - k X =0
Separating the variables in the boundary conditions, we get
X (0) f(I) =0 and X (L)T(r) = 0 for all r > 0.

In order to avoid trivial solutions, we need the boundary conditions X (0) = 0 and X ( l) = 0. Thus the
boundary value problem in X becomes
X' - k X =0, X(O)=O and X(L)=0
Note that both boundary conditions. are homogeneous.

We have already solved this in the one dimensional wave equation.
2
k= - µ-= - (
?
L
II n'
)
, 11= 1, 2 , 3, ... and
/l7f
X(x) = X,, =sin L x, 11 = 1, 2, 3, ...
Use k from the solution of X in the differential equation for T.
( J0.17)
The general solution of (10. 17) is T,, (t) = b.e->..•, 11 = I, 2 , 3, ...

2
111t
where A-,,= ( cl )
, n=l , 2,3, ...
Now, combine the solutions of X and T.
( /I 7f
11. ( x,t ) =b..e- A•1 sm
•
L x ) , n=l , 2 , 3, .. .
The solution and the given boundary conditions hold for 11 = I, 2, 3, .. _ Therefore, using superposi-
tion principle, we can write
( 10.18)
The coefficient b.. can be obtained utilizing the initial condition. Seti= 0.
f(x) = u(x,O} = i,b. sin( L x)

n=I
11
1t
Therefore,
2 rl f(x)s .m (111r
b,,= LJ L x ) dx,n=l ,2,3, .. .
0
Example 10.5
Consider the temperature distribution in a uniform bar of length 2.7T (Figure 10.3) with insulated
lateral surface and no internal sources of heat. Let 11(.r,1) represents the temperature at point x of the
bar at time 1. Both ends of the bar is held at a constant temperature of 0° C. The initial temperature
distribution of the bar is u ( x, 0) = 100° C and c = I.
Using the analyt.ical solution, graph the temperature along the length of the bar at 0.5 s, 1.0 s, 1.5 s,
2.0 s and 2.5 s.
Solution
Let us use the first 24 harmonic components.
b.=7r }11• JOOsin (")

O 2 dx.n=l,2.3 , ...
x
11{x,1)= 2).e•J.,
1, sin ( !!.x )
,t:I 2
A. = ( ~
0
J, II= I, 2 , 3, ...
100
90
90
70
-a
$B
60
60
! 40
30
20
10
0
0.00 0.00
Po,ltton on the rod
Figure 10.4 Temperature along the insulated bar •

So far, we have considered the boundary conditions as homogeneous. If both ends of the bar are
held at nonzero temperature, the boundary conditions become nonbomogeneous. We still can apply
the principle of superposition if we isolate the nonhomogeneous bou.ndary conditions and make the
problem a homogeneous one and solve it using the principle of superposi tion and then add to it the
nonhomogeneous boundary conditions in the form of a steady-state solution.
Let one end of the bar held at temperature T, and the other end at temperature ½· If the length of the
bar is L. then at steady-state, the temperature distribution of the bar from one end to the other can
be expressed as a straight line.
Chapter 1 O: Boundary-Value Problems and Partial Different ial Equations
111 (x) is the steady-state temperature at a distance x from die left end of dle bar shown in Figure I0.3.
The temperature at die ends are: H1 (0) =:,; and u, ( L) = Tr
We subtract u1 (x} from the initial temperature distribution to make the boundary conditions homo-
geneous. Therefore, we solve Equation (10. 16) using die approach we already discussed widi
following boundary and initial conditions.
The boundary conditjons: 11(0,t ) =0 and 11(L,1) = 0 for all 1 >0.
The initial condition: 11(x, 0) = /(x) - 111 (x) for 0 < x < L.
If ,112 (x ,t) is die solution with homogeneous boundary conditions, then die solution widi die nonzero
boundary conditions is
11 (x, t )=u, (x) +112 (x,t)
T, - 7; ~ b -).•, . (111r )
u ( x,t ) = T., + - L x+::; ,.e srn L x .
where
Problem Set 10.4

Consider the temperature distribmion in a uniform bar of lengdJ L (Figure I0.3) with insulated lat-
eral surface and no internal sources of heat. Using the first 24 harmonic components of die analytical
solution, determine the temperature along the length of the bar at 0.5 s, 1.0 s, 1.5 s, 2.0 sand 2.5 s
for the following conditions.
1. L=l, 11(0,r)=0, 11(L,t)=0, f(x)=80. c= I.

2. L = I, 11(0,1) = 0, u(l,t) = 0, f (x) = 60x, c = I.
3. L= I, u(0, t)=0, u(L ,1)=0, .f(x)=80sin(x/2), c =I.
4. L = I, 11(0,i) = 0, u(L.1) = 0, f (x) = 80, c = 2.
5. L=l, u(0,r)=0 , u(L,t)=0, f(x)=60x. c=2.

6. L= I, 11(0.t)=0, 11(L,t )=0, /(x) =80sin(x/2), c=2.
7. L=2, u(0,t) =0, tt(L,t)=Oi, c= I,
J60x 0~x ~l
/(x)=l 60(2 - x) l~x~2
8. L= l, 11(0,1)=30, 11(L,1) =60, J(x)=80, c=2.

9. L = I, 11(0,t) = 30, 11 ( L,r) = 40, f (x) = 60x. c =2.
10. L = I , u(0,1)=30, 11(L,t) =50, f (x)=80sin(x/2), c =2.

11. L=2 , u(0,t)=30, 11(L,1)=50, c=l,
J60x O:,x:,l
f(x)=160(2 - .x) J:, x:,2
10.5 Finite Difference Method for the Wave Equation

Let us utilize the one dimens.ional wave equation already considered in the earlier section.
2
II,, =C llx,. O<x<L, t>O
The boundary conditions are: u(0,1) = 0 and u(L ,t) = 0 for all t > 0.
The initial conditions are: u(x,0) = f (x) and 111 (x, 0) = g (x) for 0:, x :=; L.
Let I, denote the step size in x and k denote the step size in t.
h=L/n
Our objective is to approximate the values
11" = 11 ( ilt, jk ), where 0:, i:, 11 and j ;:: 0
lly represents the value of u at the mesh point (ilt, jk) as shown in Figure 10.5.
U1, j l l
I;
Ui.-1.J u,,j fl,+1.,
lli, j-J
I2
-rt I
k
__L X
I I I
0 x, L
~h--j
Figure 10.5 Mesh points in the xt plane.
The time in discrete form is expressed as 11 = jk and the pos ition along the string in discrete form is
ex.pressed as x 1 = ih.
Let ns find discrete forms of the second derivatives u., and 11,,. From calculus we know that
' 11(x+ h,t) - 2u(x,l)+ u(x - h,t) cl II

Iun
A~
,
Ir
a_,. 2 (x, t) and
lim u(x,t+k) - 2u(x,t)+11(x,1-k) iJ211( )
t~ k1 i)12 x,r
If k and I, are sufficiently small, we can write
. )- u(x+li,t) - 2u(x,r)+u(x - h ,t)

u..,(x,r - ,
1,-
u,, ( X,I ) _- 11 (x, 1+k) - 2u(x ,t)+u (x, r -

2
k)
k
Therefore, the centered second difference approximation for the second derivatives can be written as
( J0.19)
and
LI,, (iii, jk)"' :1 (u,,J+, - 2111,J + u,.1-, ). ( 10.20)
Substituting these centered second difference approximation into the wave equation, we get
( 10.21 )
c2k2
wheres = - ,- .
1i-
The finite difference solution is stable if O< s < l.
Now we have to discretizc the boundary and initial conditions.
11(0,1)=0 and 11(L,1)=0foral1 r>O
110 _1 =11(0, jk)=O and u.,_1 =11(11h, jk)=0, j > 0
u (x,0)= f (x) and 11.1 (x,O) = g(x) for 0~ x ~ L.
" 1.o =11(ih,0)=/(ih), 0~i~11 (10.22)

U 1,· J·+ 1
...Position...
•
Uj. L, j
•
U 1·+1 ' J.
•
Ui, j-1
Figure 10.6 Mesh points for moving forward in time for the
wave equation
We now use the initial data/and g to obtain the discrete fonn of the second initial co11ditio11. Using
the centered first difference approximation
11(x,1+k) - u(x,1 - k)
111 (x,1 ) "' - ' - - - - ' - - - - - - (10.23)
2k
Setting 1 = 0, we get
u(x,k) -u(x, - k)
g (X ) = 111 ( X, 0 ) "'
2k
11 (x,k) - u(x, - k) =2kg (x)
u(ih.1) - u(ill, - 1)=2kg(ih)
111.1 - 111__ 1 = 2kg ( ih) (10.24)
Set j=O in Eq11. (10.21).

111.1 + u1__ 1 = 2( 1-s)f (ih) +.{f{(i + l)h }+ f {(i- !).h}]
111.1 +111__ 1 = 2/(ih)+s[f {(i + 1)/z }- 2/ (ih )+ f {(i - !)h}] ( 10.25)
Add (10.24) and (10.25) and divide: by 2.v
u11 = f(ih)+kg(iil)+1[J{(i+l)lt} - 2f(ilt)+ /{(i - l)h}] (I0,26)
Example 10.6
Determine the displacement of the vibrating string for L = 2, c = I
2x if O~ X ~ 0.5
g(x)=0
J(x)=
l I if O.5<x~l.5
4 - 2x if l.5<x~2
Solution.
Use h = 0.25 and k = 0.1. Therefore, s = 0. I 6 and 11 = 8.
With s=0. 16, expression (10.21) becomes,
(10.27)
u,., = f (ih)+ kg(il1)+¥J {(i + l)h}- 2/ (ih)+ f {(t' - l)hJ] (10.28)
Using Equations (10.22) and ( 10.28) related to the bolllldary conditions, we get the following.
110 _0 = 0, 11,.0 = 0.5, ''1.o = l, 113_0 = 1, 11 4 _0 = l, 11;.o = 1, u6 _0 = 1, 111 _0 = 0, 118 _0 = 0
110 _1 =0, u,., =0.5, 11 2., =0.96, 113 _1 =l, 114 .1 = 1, 115 _1 =I, 116 _1 =0.96, it.,_ 1 = 0.5, 118., =0
Now we can use Equation (10.27) to generate results for the remaining mesh points.
110.2 = 0, 111.2 = 0.494, l'2.1 = 0.853, 11"3.2 = 0.994, 114.2 = 1, lls.2 =0.994, 116 .2 = 0.853, 117.2 = 0.494. Llg.1 = 0
=0,
110 _3 111•3 =0.466, 112_3 =0.711, U3.3 =0.966, 114 _3 =0.998, 115 .3 =0.966, 116_3 =0.7] (, 111 __1 =0.466,
!/BJ =0
. = 0,
llo 4 11 1.4 = 0.402, -
U, 4 = 0.57, . = 0.902,
L/34 114 '4 = 0.986, 1154
. = 0.902, 116 .4 = 0.57, It,.4 = 0.402,
118 _. = 0 and so on.
Table I0. 7 shows the displacement of the vibrating string at various time. Note that the x-axis is
labelled in discrete position steps along the length of the string.
,.,.~-----------------~ ·•~-----------------~
I
J.... I ....
-
.,.,.0.0 .. •• ·-....,.. . . ..
•• . .,.. , .. ,...
,...
-'..J\o •• •• _,..
..,_
.. .. •.. ... .. •••
,.,.
....
uo
... UHt' I.SM<=
.... ....
i ...
I... I....
...
... ...
.....••
,...
.. •• u .. ... ...
_.,
u ...
. . ... .,.,.
...
•• 20
_~
..• .. . .. ...
liff<
,,.. ... 16.0
,., ,.,
....
I
.
...
J I
... 0.0
•• .. .. .._...,.. ... . ...
.... .. 4 ~• .. .. .. .......... ..,, ... uo . . ...
,_,. ,.., ,.,_
....
:..o .."
. ..,
.... ....
I ... l
! 0.00
i ...,
! ....
.... ...
_,,.
•• 10
•• •• •• •••
S•QtNNftll
... . . ... .....
•• ,. •• . .._,.. ...
u tt• .. 16.0
Figure 10.7 Displace ment of the vibrating string using finite difference technique
As shown in the earlier section, the vibration of the string in tbis example can also be calculated
using the analytical technique. Table 10.8 shows the results obtained from the analytical and discrete
techniques side by side for comparison.
Analytical Techniqu e Finite Djjfference Technique
1.4------------------~
1,1 ...
0.7
....
l:;i..::::'---------------::::.i
i!s .(1.3
! o.ojL-------------c>.f
I ...
4 .7
-1.0
O.S sec
.u.._- -~- ~0.6- ~0.8- ~1.0- 1.2
~ - 1~A - 1,8- - 1.8r---12.0 0.S.scc.
0.0 0..2 OA ·'.J\·'""•- ~..~ -.~.- ~..~ -.~.- -..~.- ,--..~.- -,~..~- - ,...
!
Position on the string t.1>llfll. . (lt,I
1.4------------------~
1.1 ..
'·"
0.7
...
li
t o.ot------------------1
;!s
0.4
4 ,3
J....~t-------i
_.
4 .7
-1.0
02
sec
.u.._--~-~-~-~-~-~------1
~ ~ M ~
1.8
~ 12
Po~M on th.! string
U 1B IB U .. .. u •• •• no wo 110
1.4------------------~ UOr-----------------~ Miff

1.1 1.s sec
....
0.7
....
I
'
c50.
0.4
O.Oj,-=:::---------------:::,,,1
.(),3
I....
•~t-<::--------------,::,,i
4.1
-1.0 ....
.u.._--~-~-~-~-~-~------1
0.0 02 0.4 0.6 OB 1.0 12 1.A 1.11 1B 2.0
Posltion on the string
.. .. 10.o 11.0 14.0 IU
1.4,------------------~
1.1
0,7
2.a sec .....
1
J;:i._--------------------:,1 1
OA
••1c--------------,,;
••
4.7
•l.0
-U +-- - - - ~ - ~ - ~ - ~ - ~ - ~ - ---<
...
0.0 02 O.A 0,$ 0.8 1.0 1.2 1.4 U 1B 2.0 •.,....,...._~,.- -,.---..- -u~ --,..- -,,.
~-~...- --,t,u
Position on the string . , .... Ori
Figure 10.8 Conti1111ed

1, 1 , - - - - - - - - - - - - - - - - ~ ·-~ - - - - - -u ..-, - - - - - ~
1,1 2.5 se c
0 ,7
fe OA
I :: .(J,7
.1.0
-ul - - ~ - ~ ~ - ~ ~ - ~ - - ~ - ~ - - l _J
0.0 0.2 OA 0.6 0.8 1.0 1.2
Po.s.Won on th• string
1A 1.S 1.8 2.0 ·l ~ ~
. _.,
,~.-=••-~,=
.-=...~~ tU WI ..
Figure 10.8 Displacement of the vibrating string using analytical and finite difference techniques
The respective shape of the string at various times obtained by using both analytical and finite
difference techniques are almost identical. The results obtained from the finite difference approach
are in good agreement with the results obtained from the analytical technique.
Problem Set 10.5

In the one dimensional vibrating string where the two ends are held fixed, determine using the
finite difference method the shape of the string at 0. 2 s, 0.4 s, 0.6 sand 0.8 s for the following
conditions. Compare your results: with the results obtained using the analytical technique in
Section 10.4.
1. L = I, f (x) = 0.2sin (irx), g(x) = 0 and c = Lf2.
2. L= I, f(x)=sin(.1rx), g(x) =0 and c=l/2.
3. L = I, f (x) = sin (.1rx), g(x) = 0 and c = 1.
4. L=I, f(x)=0.2sin(3nx) , g(x)=O andc=L
5. L = I, f (x) = 0.2sin(nx), g(x) = sin( rrx) and c = 1/2.
6. L = 1, f (x) =sin(.1rx), g(x) = sin( nx) and c = 1/2.
7. L=I, f(x)=sin(nx), g(x)=I andc=I.
8. L = I, f (x) =0.2sin(3nx), g(x)= 2 and c=L
9. L=2, g(x)=2,c=l/2and
2.T if Q :,; X :,; 0.5

f (x) =
l I
4 - 2x
if 0.5 <x:,; 1.5
if 1.5 < X :,; 2
10. L=2,g(x)=l,c =l/2 and

2x if O ~ X ~ 0.5
f (x) =
jI
4 - lx
if 0.5 < x ~ 1.5
if 1.5 < x ~2
11. L=2, g(x)=xsin(n-x),c=land
2x if O ~ X ~ 0.5
f(x)=
!I
4 - lx
12. L=l, g(x)=l, c=land

if 0.5<x~l.5
if l.5<x~2
2x if0~x~l
J(x)= { 4 - 2x if l< x~2
13. L= l, g(x)=sin(n-x),c=land
f(x)=
l2x if OS .t S I
4 - 2.r .if 1<.t· <
_2
10.6 Finite Difference Method for the Heat Equation

Our one dimensional heat equation and the boundary and initial conditions for the insulated ba:r with
ends held at temperature zero are again
au
- =c -
2 02 11 0<x<L,r>0
il1 ox 1
The boundary conditions are: u(0, 1) = 0 and 11 (L,r) = 0 for all t > 0.
The initial conditions are: 11(x,O) = f (x) for O< x < L.
Finite difference method can be utilized to approximate the values of the unknown function at dis-
crete intervals. The process begins by dividing the independent variables into a discrete grid struc-
rure. Let us dividex (position) into n parts with a step size of h. Similarly. divider (time) into III parts
wi th a step size of k.
x, = ih, i = 0, I, 2, 3, . .. , 11
r1 = jk, j = 0 , I, 2, 3, ... ,m
Our objective is to approximate the values

ulj = u(ili, jk ), where O< i < 11 and j ~ 0
110 represents the value of u at the grid point (i11 , jk).
Frnm calculus.
. 11(x,1+k)- 11(x,1) c)u ) d

Iun --'----'---''--~ --(x.r an
,....o k c)1
lim -u~(x_+
_l_1,~l)~ --2_u~(~x ,_1)~+_1~1(~x_-_h_,t~)
h....O h2
If k and h are sufficiently small, we can write
u(x,r + k) - tt (x,t)
111 ( x, T) "' ---'---'---'---'- and
k
u(x+h,t)- 2u (x,r) + 11(x - lt,t)

II_.., (x,t)"' , .
Ji-
At the mesh point (ih,jk) we have lhe following approximations with respect to the first derivative
and the second derivative.
. . ) 11(ilt,jk+k) - 11(ilt,Jk) I ( )
11, (th, ;k "' k "'k 11, _,+1 - 11,.J
Substituting these discrete approximations into I.he heat equation, we get
I c1
k (" 1.1+1 - u,.1) = h 2 ( ll1+1J - 2u1.1 + u,. 1.1)
c2 k
wlneres= - ,.
Ir
The boundary and initial conditions have to be discretized as well.
11(0,,)=0 and 11(l,1)=0 for all t > '0
110_1 = 11 (0, jk) = 0 and u,,_1 = 11 (nh, jk) =0, j > 0
u(x,0)= f (x) for 0 < x < L
u1_6 =u(ih,0) = f( ih), O:Si::011
To compute u 1.J+ 1 we need the three values 11,. 1.J• u1_1 and u,.,.1 as shown in Figure 4.
U ,,
· J·+ 1
..
Position
...
•
U ·.
I, J
•
U 1·+l. ' J.
Figure 10.9 Mesh points for moving forward in time for the
heat equation
The finite difference approach for heat equation is unstable ifs> 1/2. For stability of the approach
0 < s <S. 1/2.
Example 10.7
A thin bar of length 2 unit is placed in boiling water. After reaching 100° C throughout, the bar is
removed from the boiling water. With the lateral sides kept insulated, s uddenly, at time I = 0, the
ends are immersed in a medium with constant temperature 0° C. Use the fini te difference method to
approximate the solution to d1is problem at 1 = 0.2, 0.4 and 0.6 sec.
Solutio11
Divide the length into IO equal parts and hence h =0.2. Talce a time step of k =0.0 I.
Boundary conditions:
u0 J =11(0, O.Olj)=O and 11, 0 ., =11(1, O.Olj)=O, j>O
Initial conditions:
u,.o =11(O. Ji, 0) = /(O.li) = I 00, 0 ~i ~ 10
The temperature beyond the initial conditions that satisfy the boundary conditions are calculated in
the foUowing manner.
u,., = (11 2 _0 + l4u,_0 + 110 _0 )/16 = (100 + 14 * I 00 + 0) /16 = 93.75
112., = ( 113_0 + 14112_0 + 11,.0 )/4 = (LOO+ 14 *JOO+ 100)/ 16 = 100
118., = JOO, 119 ., =93.75, 11,0., =0
11,.2 = (u 2 ., + 14111.1 + 110 .,)/4 =(JOO+ 14 *93.75+ 0)/16 = 88.281
111.2 = (ll3.1 + 1411-i., + u,., )/16 = (JOO+ 14 * 100 +93.75)/ 16 = 99.609

u3_2 = 100, 11, . 2 = LOO, 115_2 = I 00. u6 _2 = JOO, 117 _2 = 100
11,i,2 =99.609, 119,2 =88.28 1, 11,0.2 =0
U 1.3 = ( l½.2 + 14111,2 + Uo,2)/J6 = (99.609 + 14 * 88.28! + O)/J6 =· 83.472
u2 _3 = ( 1132 + 14112 _2 +u,.2 )/16 = (100+ 14 *99.609+ 88.281) / 16 = 98.926
U3.3 =99.976, 114.3 = 100,115.3 = 100,116.3 =JOO, 117.3 =99.976
11,i,3 = 98.926, 119.3 = 83.472, 11,0.2 = 0
Figure LO. 10 shows the temperature distribution along the length of the rod calculated using the
finite difference method.
120,~ --------------------~
108
96
84
j 72
!: 36
2A
2 3 4 s g 7 8 9 10
Position ('hl
Figure 10.1 0 Temperature ilistribution along tlle: length of

the rod using finite difference technique
Figure I 0.11 shows the comparison between the results obtained using the analytical and finite dif-
ference techniques.
120
0.2sec
108
•
84
i eo12
.'
E
I- 48
"'
■ FJnltt Olfftrtne-t ■ Analytktl
2
, • I
• 7
• • 10
Po,ltion(lht
120
0.4 sec
108
96
84
• 72
I 60
48
"'
2A
■ Finite Oift'efence ■ Ana~eal
2 3
• Positions (thl 6 7 8 9 10
Figure 10.11 Continued

,..,
0,6 Se<:
108
e
..
IMS
72
il
!; 60
....
0.
E
"8
36
24
■ Finite otf'ftrtnct ■ An1tyt:1c11

1 2 3 4 5 s 7 8 9 10
Position (ihl
Figure 10.11 Comparison between the results obtained by analytical and finite difference
technique
Problem Set 10.6

Consider the temperature distributfon in a uniform bar of length L (Figure 10.3) with insulated
lateral surface and no internal sources of heat Using the finite difference method. determine the
temperature along the length of the bar at 0.5 s, 1.0 s, 1.5 s, 2.0 s and 2.5 s for the folJowing con-
ditions. Compare your results with the results obtained using tbe analytical technique discussed in
Section I0.4.
1. L=l , u(O,t)=O, u(L,t)=O, f(x)=80, c=I.
2. L = I, 11(0,t) = 0, u(L.t) = 0, f (x) = 60x, c = I.
3. L = I, u(O,t) =0, 11(L,1) = 0, f (x) = 80sin (x/2), c = I.
4. L = I, 11(0,t) = 0, u(L.t) = 0, f (x) = 80, c = 2.
5. L = l , u(0,t)=0, u ( L,t)=O, f(x)=60x. c=2.
6. L= I, 11(0,t)=0, 11(L,t )=O, f(x)=80sin(x/2), c =2.
7. L=2 , u (O, t)=O, u(L,t)=O, c =l,
f(x)=l 60x o:=;x:=;1

60(2-x) I :=;x :=;2
8. L=l , 11(0.1)=30, u(L,t)=60, f(x)=80, c=2.

9. L=l, 11(0,1)=30, u(L,1)=40, J(x)=60x,c=2.
10. L=l, u(0,t)=30, 11(L,t)=50, /(x)=80sin(x/2), c=2.
11. L=2 , u(0,t)=30, 11(L,t)=50, c=l,
0~x ~ l
/(.x)=
!
60x
60(2 - x) l~x~2
CHAPTER 11
0 TIMIZ Tl N
Optimization techniques are utilized to make decisions in many areas starting from economics to
engineering. The objective of optimization is to make the best decision under given circumstances.
111 most cases the principle objective of optimization is to minimize cost. Starting from its initial
application to design problems, optimization techniques have spread into other areas like invest-
ment, transportation, business, management, pub lic services, operations research to name a few.
The advent of fast and powerful computers have made it possible to solve complex optimization
problems with relative ease. 111 this chapter we will be dealing with optimization problems that can
be expressed in quantitative manner.
Am optimization problem has two main components; an objective function and a set of constraints.
The objective function is a mathematical model that relates the underlying variables associated with
the product or the process. The constraints are mathematical expressions that define the relationship
between certain variables and, in many cases, define the limit imposed on certain variables.
111 this chapter, some foundational concepts related to optimization are discussed first followed by
unconstrained optimization. Constrained optimization techniques are discussed in the later part of
this chapter.
11.1 Convex Sets

A convex set is a set of points such that alJ the points on the straight Jine between any two points of
the set are also contained in the set. This means that the points in a convex set are connected in such
a way that one could move from one point to another without leaving the set. Convex sets are widely
used in the mathematical formuJation and solution of optimization problems. If a and bare points in
a vector space the points on the straight line between a and b are given by
x=Aa+(I - A.)b, O~,l~l
Chapter 11: Optimization
The expression la+ (1 - 1 )b, 0:,; A:,; 1 is called a convex combination of the points a and b.
In a generalized way, if xl' x 2 ,x3 , _ •. , x. arc points in a convex set X, then a point x of the form
x = l,x, + A.iX2 + · ·· + J..x. where A.1 +Ai+···+?... = I and A1 ~ 0, i = L, 2, .... n is called the convex
combination of x 1 ,x2 ,x3 , ••• ,x. and also belongs in X.
11.2 The Mean Value Theorem

Theorem
Let/ be a function which is continuously differentiable on the closed interval [a, b]. Then there
ex.ists at least one point c in (a. b) such that
f'(c)= f(b) - f (a)

■
b- a
The mean value theorem inlplies that the slope off (x) at c as shown in Figure I 1. 1 will be the same
as the slope of the line segment connecting the two points [ a,f (a)] and [ b,.f (b )]. The implication
of mean value theorem is significant if we wam to evaluate f (b) based on the given value off (x) at
a. It tells us that if the point c is found then no other derivative off (x) higher than J'(x) would be
required to evaluate f (b). The concept can be extended to higher order derivatives if we assume that
f (x) itself represents the 11th derivative of a function. c can be expressed as a convex combination
of a and b in the following manner.
c=8a+(l - 8)b O:,; 8 :,;1
Figure 11 .1 Display of The Mean Value Theorem
11.3 Relative Maximum and Minimum

A function f(x) has a relative maximum at x;;;;;a if J(a) is greater than any value immediately
preceding or following. A function J (x) has a relative minimum at x= b if/ (b) is smaller than any
vaJue immediately preceding or following. The 1em1 'relative' in1plies that the value of the function
at points other than a and b may actually be greater than f(a) and smaller than f(b). Figure J 1.2
shows a function J(x) with its relative maximum and minimum .
.fi:x)
0 X
Figure 11.2 Relative maximum and relative minimum
The value of a function at a relative maximum or at a relative minimum is called an extreme value
of the function. The value of x where the function has either a minimum or a maximum is called a
critical value.
11 .4 Optimization of Functions of a Single Variable

If f(x) bas a relative maximum at a point x 0 then in the neighborhood of x 0 , f(x)-5,f(x0 ). Iflhl is
small, then we can write
(11.1 )
Expression (11. l ) can be divided by/, and therefore,
J (x0 +h) - f(x0 ) -5, O h>O ( l l.2)

h
f(xo+h) - f(xo) ~O
h<O ( I 1.3}
h
If we take the limit as lz ➔ 0, then from Equation ( 11.2),
( 11.4)
Similarly from Equation ( 11.3),
clf(x 0)
-'---'--"-'- ~0. (11.5}
dx
The results ( I 1.4) and (11.5) together lead to lhe conclusion lhat
(11.6)
For a relative minimum,
f(xo+h) - f(x 0 ) ~O
h>O (11.7)
Ii
( l l.8)
Therefore, when we take limit as It ➔ 0 and foUow the similar arguments I.bat we did for the relative
maximum, we conclude lhat for a relative minimum,
df(x0 )
0. ( 11.9}
dx
A graphical explanation can be given for the necessary condition for a maximum as shown in
Figure 11.3. The slope, f'(x) of this function is shown at several places on the curve with arrow
marked Line segments. f' (x) is posi.tive on Lhe left-hand side of lhe maximum point and negative on
_/{x)
0 X
Figure 11 .3A function with its slope at and near a maxi-

mum and minimum
Chapter 11: Optim ization
the right-hand side of the maximlllll1 poim. Since J' (x) is continuous, a transition from positive to
negative can only happen if f' (x) becomes zero in betv,een. As shown in the figure, f' ( x) is zero at
x0 indicating that the function f (x) does not increase beyond x 0 • As shown in the figure, a similar
explanation with d1e signs reversed can be given for a minimum poinL
The points where d1e first derivative of a function is zero are called stationary points. The stationary
points are the candidates for relative maximum, minimum and saddle points.
11 .S Sufficient Conditions for an Optjmum Point

Suppose mat J(x) and its first and :second derivatives are continuous at x 0 • Using Taylor's theorem
we can write
If J(x) has a relative maximum at x 0 , then we know from the necessary condition d1at f'(x)=O.
Therefore, Equation (11.10) can be rewritten as
( 11.11)
If f (x0 ) is a maximum point, then it foUows that
( 11.12)
Since /,2 is always positive, the only way the term /i2 /"[ 0x0 + (l - 0)(x0 +It)] can be less than zero
2
iff"[0x0 +( l - 8)(x0 +!i)]<O.
It foUows from d1e continuity of f ' (x) that if f"[0x 0 +(1 - 0)( x 0 + h}]<O, ilien J'(x0 )<O.
Therefore, we conclude mat if f'(.v 0 ) < 0 when f'(x0 ) = O, f (x0 ) is a maximwn.
If f (x0 ) is a minimum poim, ilien
( 11.13}
Following a parallel argument that if f'(x 0 )> 0 when f'(x 0 ) = 0, f (x0 ) is a minimum.
A graphical explanation for die second derivative condition can be given wid1 respect to Figure 11.3.
f' (x) is positive on the left-hand side of the maxin1wn point and negative on the right-hand side of
the maximum point. This means d1at for a maximum, f'(x) decreases as we move from die left side
of the maximum point to the right side of die maximum point. Decrease of f'(x) means its derivative
wid1 respect to xis negative, i.e., f'(x) < 0 in die neighborhood of x 0 including at x 0 •
However, it is possible iliat at x 0 , bod1 the first derivative and die second derivative vanish. We must
then examine the higher order derivatives. Let us start with the following theorem.
Theorem: Assume that / (x) and .its first II derivatives are continuous. Then / (x) has a relative
maximum or minimum at x 0 if and only if II is even, where n is the order of the first non-vanishing
derivative at Xo, The function has a maximum at Xo if f" (xo) < 0 and a minimum if/" (xo) > 0.
Proof
Since f(x) is continuously differentiable and its 11th derivative exists on the interval [x0 , x 0 +h], we
can write the following using Taylor's theorem.
/(x0 +h)= J(x0 ) +lif'(x0 )+ :: f"(x0 )+ ';: /"'(x.,}+···
· · ·+ ( 1i•-•) /''· 1 (x0 ) + It" f" [£Jx0 + (1 - e)(x0 + /1 )]

11 - I I 11!
Let us assume that the first 11 - I derivatives of/ (x) at x 0 vanish, i.e.•
Therefore.
( l l. 14)
We conclude from Equation ( 11.14) that the sign of {f(x0 +!,) - f(x0 )} is the same as the sign of
h" J"[0x0 + (l- 0)(x0 + h)} Since/" (x) is continuous,/"[0x0 + (1 - 6) (x0 + h)] will have same sign
11!
as f" (x0 ). This leads to the conclusion that {/(x0 + h)- f (x0 )} will have the same sign as f" (x0 ) if
11 is even. Because II being even is the only situation where the sign olf !!:... J"[8x0 + (1 - 8)(x., +h)] is
11!
the same as the sign of J"[0x0 +(l - 0)(x0 +h )]. Therefore, when II is even,{/ ( x0 +h ) - f (x0 )} is
positive when /" (x0 ) is positive an.d {/(x0 +/,)- /(x0 )} is negative when/" (x0 ) is negative. This
leads to the conclusion that/ (x0 ) will be a minimum if/" (x0 ) is positive and will be a maxinmm
if r (x.,) is negative.
Let us consider the following function.
y1 = /(x) = x• - 8.x3 +24x 2 - 32x+ 16 =(x - 2)'

2 1
dy, =4(x - 2)'- d y' =12(x - 2)2, d y 1 =24(x - 2)· d•y,4 =24
dx ' d.x 2 ' d.x3 ' dx
. hing d envauve
Tl1e first non-vams . . 1s. -d• Yi,- . 111s
. even and - Yi 1s
d• - . . theretiore, f ( x ) Iias a nun-
. pos1t1ve, .
4
dx dx
imum at x = 2 Figure 11.4 shows the plot of y,.
16.0
13.7
11.3
9.0
6.7
"='-
,I:;.
';::;; •.3
2.0
.0.3
-2.7
~.o
~.00 ·1.00 0.00 1.00 2.00 3.00 4.00
.>:
Figure 11.4 Plot of y1
Consider another function.

3 1
y1 = f (x) =- x• +12x - 54x +l08x - 81 = - (x - 3)'
' 3 •
dy1 =-4(x - 3}3; d-y_,2 = -12(x - 3)2; d ):1 = - 24(x - 3); d ~2 = - 24
dx dx- dx dx
The first non-vanishing derivative is d' J~i . 11 is even and d' )~2 is negative, therefore, J(x) has a
dx dx
maximum at x = 3. Figure I 1.5 shows the plot of y1 .
6.0
-3.7
-13.3
-23.0
-32.7
v
'.::; ◄2.3
,62.0
-61 .7
.71,3
-t11.0
-1.00 -ll.13 0.75 1,63 uo 3..38 4..25 5.13 6 .00
Figure 11.5 Plot of y2
Consider the following function where the first non-vanishing derivative is odd.
3
y3 = J(x) =x3 - 6x 2 + 12x - 8= (x - 2)
dyJ =3(x- 2)1; d2 y3 =6(x - 2); d lyJ =6

dx dx2 dx3
The first non-vanishing derivative is the third derivative. That means II is odd and, therefore, we have
ne ither a maximum nor a minimum. Figure 11.6 shows the plot of y3 =x3 - 6x 1 +12x-8 =(x - 2)3.
Notice a point of inflection at x = 2.
100.0 ·
77.8
65.6
33.3
11.1
~
,::::; ·11.1
-33.3
,65.6
-77.8
-100.0
-8.00 ◄,.oo -2.00 0.00 2.00 8.00
Figure 11 .6 Plot of y 3
It ,can be noticed that in Figure 11 .6 the slope on the left of the poiI11t of inflection is positive, then
at the point of inflection the slope becomes zero and then on the right of the point of inflection the
slope is again positive. This is the characteristic of a point of inflection that the slope around it does
not change sign. The point of inflection (saddJe point) is a stationary point that is neither a relative
maximum nor a relative minimum.
Problem Set 11.1

For the following functions, find aU stationary points. Determine if ithe stationary points are maxi-
mizing, minimizing or saddle points.
1. f (x) =(x-l)(x - 2)-1 3
2. f(x)= - x 2 - x+25
3. f (x) = x 3 - 3x2 - 6x+8
3
4. J(x)=(x - 3)
5. f(x)=(x +s/
6. f(x)=(x - 4/
7. J(x)=(x - 2)1
8. f (x) =x• - 7x3 +9x 2 + 27x -54
9. f (x) =x• - l2x3 +54x 2 -J08x+98
10. f (x) = - x6 + !8x; - l35x 4 + 540x 3 -1215x 2 +1458x - 529
11.6 Global Maximum (Minimum) of One Variable

In many applications we are interested in the global maximum (minimum). If the function is con-
tinuous within an interval, lhen find all lhe relative maximums (minimums) wilhin the interval first.
However, it is quite possible that the values of the function at the boundaries of the interval could
be greater (less) than any of the relative maximun1s (minimums). Therefore. we should evaluate the
function at the boundaries as wel I.
Consider that f(x) is a continuous function in the interval a~ x ~ b. We are interested in finding the
global maximum of f(x) in [a, b ] .
Determine the derivative of the function and then find the roots of f'(x) = 0. Assume that the roots are
x,, x 2 , x 3 , ... , x•. Exclude any roots that are not in [ a, b ]. Determine /(x, ), J(x 2 ). f(x 3 ), .......... ,
J(x.) if the they are in [ a, b]. Now, Determine f(a) and f(b). If none of the roots are wilhin
[a , b ], lhen the global maxinlun1 i.s the larger of f(a) and f(b). If all or some roots are in [a , b ),
then the global maximum is the largest of J(a ), f( b ), f(x, ), f(x 1 ) , J(x3 ), .......... , f(x. ).
Example 11 .1
Find the global maximum off (x) in [ 0, 2.5).
f (x} = 0.25x" - 2x 3 +55x 2 - 6x+4
Solution
f'(x)=x 3 - 6x 2 + I lx - 6
Solving f'(x) = x 3 - 6x2 + I Lx - 6=0 we gel x, = I, x2 =2 and x 3 =3

Ignore x 3 as it is not in [o, 2.5]. Therefore, we evaluate f(x,) and /(x2 ).
The global maximum is
Max[f (0)=4. f (!) = 1.75, f (2) =2, f (2.5) = 1.891]= 1.891
11.7 Optima of Convex and Concave Functions

A convex function /(x) is a continuous fw1ction over a closed interval aSxSb that satisfies the
conrution that
( 11.15)
for any x,, x 2 in [a,b] and all 0, 0$8$1.

The inequality (I 1.15) suggests that any point on the line segment connecting (x"f(x 1 }) and
(x~, f(xi)) will be on or above the convex curve, f(x) as shown in Figure 11.7. If a functionf(x)
bas a second derivative in [ a, b ], then a necessary and sufficient condition for it to be convex is that
Figure 11.7 A convex function

Chapter 11 : Optimization
the second derivative is positive in [ a , b]. It should be mentioned at this point that the sum of two
or more convex functions is also a convex function.
Theorem: Let f(x) be a convex function over a closed interval, a sx $b. Then any relative mini-
mum of f(x) in this interval is also the absolute or global minimum of f(x) over the interval.
The global maximum of a convex function J(x) over a closed interval, a$x $ b will be either J(a)
or f ( b) or both.
A concave fw1ction /(.r) is a continuous function over a closed interval a $x $b that satisfies the
condition that
f (Bx, +(1 - 8)xd ~8/(x, )+(i - 8)/(xJ (11.16}
for any xi' x 1 in [ a, b] and alJ 0, 0 s 8 $ l.

The inequality (1 1. 16) suggests that any point on the line segment connecting (x, ,J(x, )) and
(x:1, J(x1 )) will be on or below the concave curve, /(x) as shown in Figure J 1.8. If a fuuctionf(x) has a
second derivative in [ a , b ], d1en a necessary and sufficient condition for it to be concave is that the
second derivative is negative in [a, b]. It should be mentioned at thfa point that the sum of two or
more concave functions is also a concave function.
!
.f{x) ·····J--···· L
j.• ...
I
L r- .L..
! .• -
••• j •••••••••
! '
!~········1··················l· ~ ~
. ... . .:--- . . -••·· ··i:....... ij....-..L

.
. . .-·-····:. . . . !.........;...........
i
X
0
Figure 11 .8 A concave function
Theorem: Let f(x) be a concave function over a closed interval, as x $ b. Then any relative maxi-
mum of f(x) in this interval is also the absolute or global maximum of f(x) over die interval.
The global minimum of a concave function f(x) over a closed interval, a$ x $ b will be either f(a)
or f ( b) or bod1.
Problem Set 11.2

1. Detemline if f(x) = x 2 + 2.Sx - 12.5 is a convex or concave function in [ -6, 8] and then find
the global maximum of the function in [ -6, 8].
2. Detennine if /(x) = - x 2 - 2.5x + 29.5 is a convex or concave function in [-8, 8] and then
find the global minimwn of the function in [ -8, 8].
3. Find U1e global maximwn of the function f ( x) = x 3 - l.5x 2 - 22.5x + 50 in [ -6, 6].
4. Find the global minimum of the function f(x) = x 3 - 2x 2 - 23x + 60 in [ -6, 6].
5. Find the global maximum of the function /(.r) = x 5 - x' - 67x 3 + l2 !x 2 + I026x - 2520 in
[-8, 7 ]-
6. Find the global mininmm of the fwlction f (x) = x 5 - 3x 4 - 52x3 + I56x 2 + 576x - 1728 in
[-7, 7]-
11.8 Search Methods for Functions of One Variable

Search methods can be used to find the maximum or minimum of functions of one variable that are
unimodal. Although search methods are straightforward to implement, in many cases, can be com-
putationally very burdensome. However, with the availability of fast computers, search methods can
be employed with relative ease.
11.8.1 Exhaustive search method

In exhaustive search method, the entire domain [ x 0 , x.] of a function is searched for the maximum
(minimwn). If a predetermined interval is provided then we will limit our search within this interval.
The domain is divided into equally spaced subintervals. The function is evaluated at the boundaries
of these intervals. The boundary where the function exhibits a maximum value is our current maxi-
mum. Further search will be needed to improve the current maximum. To do this, we will select the
two subintervals, one on the immediate left and the other on the immediate right of the boundary
that has the current maxin1wn. These two subintervals now constitu~e our new zone of search. T llis
zone is divided into smaller subintervals and the function is evaluated at the new boundaries and the
current maximum is updated. The process will continue until no further improvement with respect
to a preselected tolerance can be made.
Figure 11 .9 shows the subintervals for the search process. Assume that after the first iteration, the
current maximum is located at x1• Then in the second iteration the search is carried in [ x 1_ 1 , X;+ i ]. The
zone in the second iteration is divided into subintervals smaller than the first i1eration. Superscripts
are used to indicate the i1eration number.
x,, x, Xi Xi- I X; X f'!' 1
./ '
,, 0 XO x•,, I
x,
• x~ X i- I I
II I I I I I
•x• x'i I
0 x.
Figure 11.9 Subintervals for exhaustive search
Let us use an example to illustrate the steps.

Consider the following llllimodal flllllction
625
J(x)= ? x 2 in (0,300]
(x- +625)
Divide the given interval into SO subintervals. The width of eacb subinterval is 6. Evaluate the func-
tion at the boundaries.
f(xi) =f(O)=O
J( xn = f (6) = 0.0086
f(x~) = /(12)=0.0127
J (xf) = J(l&)= 0.01 2s
f (xn = /(24)= 0.0104
f (xf0 )= f (60)= 0.0021
f (xf0 ) = f (300) = 2.283x 10-5

After the first iteration, the current maximum is f ( xn = f (12) = 0.0127. In the second iteration, the
search is carried in [ 6, 18]. Let us divide the interval into 4 subintervals. Tbe width of each subin-
teirval is now 3. Evaluate the function at the new boundaries. Some bow1daries will be the same as
in the first iteration. We can utilize their values directly in the second iteration. We get the following
vaJues in the second iteration.
f (x~ )= /(6) =0.0086

f(x:)=/(9)=0.0 113
f (x!) = /(12)=0.0127
f (x~) =/(IS)= 0.013
J (x!) = /(! 8) = 0.0125
After tlle second iteration, the current maximum is f(x!) = /(15) = 0.013. If further improvement is
necessary, we will carry out a tllird iteration in [12,18] by making the subintervals even smaller than
Lhe second iteration. The iterations will stop when the absolute difference between tlle successive
maximum will be less than or equal to a given tolerance.
11.8.2 Fibonacci method

In Fibonacci method the nwnber of trials II has to be specified. The larger the number of trials the
narrower wiU be the zone of uncertainty in which the maximwn (minimum) of the function wiU be
located. The boundaries at which the function is evaluated arc decided by utilizing Fibonacci num-
bers. The Fibonacci numbers (F;) are defined in the foUowing manner.
F;, =F; =l. and

Fn = f',,_ + F,,_
1 2, II= 2, 3, 4, ...
The above sequence gives the Fibonacci numbers as: l, I, 2, 3, 5, 8, 13, 2 1 ....
We assume a unimodal function f ( x) in [a , b].
In the first trial. two boundaries are determined such that they both are same distance away from the
two ends of the interval. Once the number of trials II is known, the distance is calculated as
d' =(Fn-1)d
F o
"
where
The boundaries are at x, =a+d; and x 2 =b - d;.

The two boundaries placed between a and b divide the interval into three zones. Using the unimo-
daJity property of the function, a part of the interval is discarded, leaving a smaller interval where the
maximum is located. In the next trial, a new boundary is determined such that the new one and the
existing one from the previous trial. are both the same distance away from the two ends of the new
interval and thus dividing the interval into three zones. Again a part of the interval is discarded and
the process continues nntil the last trial. The middle point of the final interval is taken as the solmion
where the function has the maximum.
Let us use the previous example in Section 11.8. J to illustrate the steps.
Consider the foUowing unimodal function
f(x) = ,625x . [ Q,300]

2 ID
(x- +625)
Find the maxin1un1 using Fibonacci method w ith 11 = 7.
a=0, b=300
d., =b- a=300- 0=300
d, =( F,,_2 )do= ( Fs ) {300) =(.!_)(300) = 114.286

F,, F1 21
x, =a+d1 =0+ 114.286= 114.286

x2 =b - d, =300 - I 14.286=185.714
f(x,)=3.813xl0~
f (x2 ) = 9.4 I3x 10-5
Since/(x,)>.f(xi), the discarded zone is [x2 • b)= [l85.714, 300].

The new interval is [ a , x2 ] = (0, 185.714).
The localion of the new boundary x 3 should be such that its distance from the endpoint a should be
same as the distance between x, and x 2 •
Therefore, x 3 =a+(x1 - x,)=0+(1:85.714- 114.286) =71.429
Since f (x3 ) > f (x,), the discarded zone is [ x,, x 2 ] = [114.286, 185.7 [4]
The new interval is [ a, x,] = (0, 114.286)
The location of the new boundary .t:4 should be such that its distance from the endpoint a should be
same as the distance between x 3 and x,.
Therefore, x 4 =a+(x, - x3 )= 0+(114.286- 71.429)=42.857
Since f ( x,) > f ( x3 ), the discarded zone is [x3 , x,]=[7 1.429, ll4.286]
The new interval is [ a, x 3 ] = [ 0, 7 i.429].
Amdsoon.
The maximum is 0.013 located at .x-7 = 14.286.
Problem Set 11 .3
For the following unimodal functions in their respective intervals, find the maximum (minimum)
utilizing the Exhaustive and F ibonacci search methods.
?
L /(x)= 2
x- in [-9,9]
X - 89
125
2. /(x)=<+ in[ -6,6]
x- + 225
3. f (x) = 3x" +1250 in

4
[-6, 6]
x +51 2
4. /(x) = x',++512
X
1250 in [-6, 6]
5. f(x)= , X in[0,500]
x-+512
625
6 . f (x) 2
x in [-500. 0]
X +J024
11 .9 Optimization of Functions of Several Variables

Consider an n-dimensional Euclidean space R ". x=(x, ,x2 ,x3 , • •• ,x,,f indicates a point or vector
in R" and /( x) is a function of multiple variables commonly expressed as J(x" x2 ,x3 , ••• , x.).
A function .f(x ) is convex over some convex set X in R • if for any two points x, and x 2 in X and for
aJJ 0, 0 ~ e ~ I the iucquaJjty ( 11.17) holds.
(11.17)
In a similar manner a function f( x) is concave over some convex set X in R" if for any two points
x, and x2 in X and for all 0, 0 ~ e ~] the inequality (I 1.18) holds.
(I 1.18)
Let/ ( x) is a continuous function with continuous first- and second-order partial derivatives over an
open convex set X in R", then utilizing Taylor's Theorem, we can state that for any two points x, and
x2 = x, + h in X. there exists a 0, 0 ~ e ~ I such that
(11.19)
provided lb Iis small.

Vf is the gradient vector defined as
vi -(aJ
- ax,
a1 ...
dX2
a1JT
ax,,
H is the Hessian matrix off (x) and is defined as an 11x11 matrix of second-order partial derivatives
off (x).
a21 a12
llf
ax: ax, ax2 ax,ax.
·,pf a21 ,Pf
H = ilx2 ih1 ax; ilx,- axn
a21 a21 a21

ax.ax, ax. ax2 ax;
Assume that a minimum or maximum occurs at x = x0• For single variable cases we used the condi-
tion that tbe first-order dcrivaLive be zero at tbe point of a maximum or minimum. We can utilize this
condition for multivariable cases as well. However, instead of just one first-order derivative, we will
have multiple first-order partial derivative for multivariable cases.
f( x0 + h)= J( x0 ) + VJ( x0 t h +½hTH [B x + (J - 8)(x + b)] b

0 0
aJ( xo)
0 i = ], 2, 3, .... n
ax,
- T
Therefore, Vf( x0 ) h =0
( I I.20)
Similar to the single variable case in Equation (11.1 4), the sign of {f(x0 + h) - /(x0 )} depends
on tbe sign of hTH[ 8 x0 + (J - 8)(x0 + h )] b. Again due to the continuity of the second-order par-
tial derivatives, hTH [x0 ]h will have the same sign as hTH[0 x 0 +(1 - 8)(x0 +b )] h. Therefore,
{/ (x0 + h) - f ( x0 )} wil l be +ive if b TH ( x0 Jb is +ive and -ive if hTH [ x0 ) h is -ive.
Consequently, f( x) will be a minimum at x0 if hTH [ x0 ) h is positive, and conversely, f (x) will be

a maximwn at x0 if hTH (x0 ] h is negative.
Using matrix theory, a quadratic form bTH (x 0 Jb will be positive if and only if the Hessian is a
positive definite matrix. Similarly hTH (x0 Jh will be negative if and oaly if the Hessian is a neg-
alive definite matrix. A positive definite matrix is a symmetric matrix whose eigenvalues are aU
positive.
Example 11.2
Determine the maximum of
Solution
l[,l
of( x)
ax,
[VJ( x)r =
aJ( x) [-,,.• 1•
= -6~2 +24 = 0
ax?
lJJ(x )
3
2x 0
ax3
iJ1 f (x) _8 iJ2J(x) 0 a2 f (x) 0

i)x: i>x,ax2 ax, ax3
cl J(x) -6 a .r(x)
2
0
iff(x)
-2
ax12
l
ax1ax3 dX1l
-8 o o
H (x0 )= 0 -6 0
[ 0 0 -2
We have to prove whether H ( x0 ) is: positive definite or negative definite.
yrH(x0 ) y =- 8y~ - 6yi - 2yJ which is clearly less than zero for any YT = [ y, y2 y3 ] provided
y .t O. Therefore, x! = [ 2 4 0 ] is a maximizing point.

Note that the Hessian matrix in this example is a diagonal matrix with all negative elements. A diag-
onal matrix with all negative elements is a negative definite matrix. A symmetric matrix is negative
definite if its principle determinants. are nonzero and alternating in sign with the first being negative.
If one or more of its principle determinants are zeros d1en it is called a negative semidefinite matrix.
In a more concise fonn we can state that a symmetric matrix M is negative definite if yTMy < 0 for
all y ;c 0, and negative semidefinite if yTMy:;; 0 for all y.
Example 11 .3
Determine the minimum of
f (x)= f (x,, x 2 ,.l'3 )=2x~ +3x; +4x: - 8x, - 12x2 - 24x3 + 11 0
Solutio11
ll l
at(x)
i)x,
c)J(x) 4x, -- 12
8 O
= 6x1 = 0
dX2 [ 8x - 24
3
0
i>f (x)
i)x3
iJ1 f (x) 4
i'P f (x) =O if /(x) 0
i)x: i_lx, i_lx2 dX1dXl
·,)1 f (x) 6 di J(x ) 0 i)2 f (x)
8
ax~ dX2dXl clx;
l
4 0 0
n(.,)-[ 0 6 0
0 0 8
We have to prove whether H (x 0 ) is positive definite or negative definite.
yrH(x0 )y = 4i +6y: +8yJ which is clearly greater than zero for any YT= [ Y, Y2 y3 ] provided
y '# 0. Therefore, x! = [ 2 2 3 ] is a minimum point.
Note that the Hessian matrix in this example is a diagonal matrix with all positive elements. A diag-
onal matrix with aJJ positive cleme'llls is a positive definite matrix. A symmetric matrix is positive
definite if its principle determinants are aJJ positive and positive semidefinite if they arc alJ nonnega-
tive. In a more concise form we can state that a symmetric matrix M is positive definite if yTMy > 0
for alJ y #! 0, and positive semidefinite if yTMy ~ 0 for all y.
Problem Set 11.4

For the following functions, determine their respective maximum (minimum) and with the help of
the hessian matrix prove if the functions maximize or minimize at the stationary points.
1. / (x,. x 2 ,xl) =12x, +x2+ 2Exl - x{ - 2xJ - xi
2. f (x,. x2,xl) =x, + 7x2 +x3 +xf + 2.ti +xi
3. J(x,. x2,x3 ) = 3x1 - l 7x2 + x3 - Sx; - 2xi - Xi
4. f (x., x 2 ,x 3 ) =x1 +5x1 + 7x1 +4x~ + 2xi +3xi
11.10 Multivariate Grid Search Method

As mentioned before, search based optimization techniques became attractive due to the availability
of fast computers. In multivariate grid search method the search region is divided into a grid struc-
ture. The function is evaluated at each aode of this grid structure. The movement from one node to
another is made in the direction of increase (decrease) of the function. This is relatively an inefficient
but straightforward method. The method is useful as long as the dimensionajjty is not too great.
The steps involved with multivariate grid search method can be listed in the following manner.
1. Divide the n-dimensional search region into a grid structure by selecting grid size
=
!1xI , i I, 2, .. . , 11 for each variable x1, aI ~ x I ~ b1 , i 1, 2, ... , 11. =
2. Select a starting point based on an algorithm or some other condition. This starting point will
be replaced later with the current best point.
3. Evaluate the function, f (x) at the 3" -1 surrounding points.
4. From the current best point and its surrounding 3" - 1points, select the point with the greatest
in1provement in J (x). This point, becomes the new current best point.
5. Repeat steps 3 and 4 until no further improvement is possible.
6. Reduce the grid size by halving or some other process and return to step 3. This process of
grid size reduction is continued until some pre-specified tolerance between the current best
and the previous best in / ( x) is reached.
Figure 11. IO shows the grid structrure in a two dimensional case. The starting point is labeled as
A. The 3 2 - 1 points surrounding A arc numbered I to 8. If the contours are in the direction of the
,
I If
' '\
/ I
.
I I I I'\. ! \
I ll - ~ '\ ,- 1\
._ f-
- -- -BJ
~
g
I
~ \
\
\ \
\
\
\ 'f
~
._ -
I in t- )_ I
'
- 'SI' ' ~
\ ·1- \
ct
"'
~
~
©\
......
,._,__
I'...
I
I
I
I /
,J
'L J
,
-....
' ' \ '\ 'i- '\!- " '\..,
-~,y
I
I ., ~
,, I
lL ~ .: -<" ./
~ l \• ,_ j '- t-.... I ~
/ ~
7 ~ 6
Figure 11 .1O
' i-... 1'-..
Two-dimensional grid search
I
function increasing towards B, then if we are maximizing, we would choose point 3 as our next
starting poim. Successive choices arc shown as circled points. When we come close to the point B
we would have to reduce the grid size in order to get as close as desired to the maximal point.
11.11 Univariate Search Method

In a univariate search method, the function f(y 1,y2 , • •• ,y,, .. .,y,.) is maximized (minimized}
wid1 respect to one variable at a time keeping other variables fixedl at dieir previous values. This
approach substantially lessens the computational burden. The search procedure stans from a point
x6 = (y~, y~ ,, .. ,yf, ... , y~f within the feasible space. The next point x 1 is delermined by performing
a maximization (minimization) with respect to the first variable.
x 1 = x0 + il,a,
where a, = [1, 0, 0, . ... 0 rand il, is a scalar such that f (x0 + il,a,) is maximized (minimized).
Similarly point x, is detennined by performing a maximization (minimization) wid1 respect to the
kth variable.
x, = x,_, + l,a,
where a, = [ 0, 0, .. ., 1, . .. , 0 r and 1, is a scalar such that f (x._1 + 1,a,) is maximized (minimized).
The steps are repeated until ll,I for each variable becomes less than a tolerance.
Figure 11.11 shows a univariate search approach for a two-dimensional case. The univariate approach
works well for functions that have Less interaction among the variables.
x,
Figure 11 .11 A univariate search in a two-dimensional case
Chapter 11 : Optimization
Example 11.4
Minimize f (x,, x1 ) =4x, + 7x2 +x,x2 - x~ - Xi
Solutio11
Let us start at(x1 ,x2 )=(7, 7).
Fix x 2 = 7 and solve the problem Min f (x,, 7) with respect to x,.
Min J(x,, 7) occurs at x, =5.5

Next, we minimize f(5. 5, x2 ) with respect to x 2 •
Min J(S.5 , x 2 ) occurs at x 2 =6.25

Au each s tage we only solve a one-dimensional optimization probJem. The results are shown in
Table I.
Table 1 Results of a univariate search
Point Function Minimized Variable Folllld Best Current Point
X0 =(7, 7) f(x., 7) x, =5.5 (5.5, 7)
X1 = (5.5, 7) /(5.5. xJ x 2 =6.25 (5.5, 6.25)
~ =(5.5, 6.25) f(x.,6.25) x, =5.125 ( 5.125, 6.25)
X3 =(5.125, 6.25) /(5.125, x 2 ) x 2 =6.0625 (5.125, 6.0625)
x. = (5.125, 6.0625) f ( x,, 6.0625) x, =5.0313 (5.0313, 6.0625)
x5 = (5.03 I3, 6.0625) f (5.0313, X1) ~ =6.0156 (5.03 13, 6.0156)
x 6 =(5.0313, 6.0156) f (x., 6.0156) x, =5.0078 (5.0078, 6.0156)
X7 = (5.0078, 6.0J 56) /(5.0078,x1 ) ~ =6.0039 (5.0078, 6.0039)
Ks= (5.0078, 6.0039) f (x., 6.0039) x, =5.002 (5.002, 6.0039)
X9 = (5.002, 6.0039) /(5.002, ·t"i) ~ =6.001 ( 5.002, 6.001)

11.12 Gradient Methods: Directional Derivatives

Gradient methods follow the direction along which a function increases in order to find the maxi-
mum. In multidimensional cases, we can find this direction with the help of directional derivatives.
The directional derivative of f( x) at x 0 in the direction vis
D f(x )=limit /(xo+lv) - /(xo)

v O ).➔0 A,
The derivative off (x) with respect to a direction v can be expressed in terms of partial derivatives
in the following manner:
lvl =l ( 11.21)
where x 1 is tbejth component of x and 11 is

1
tbejth component of the unit vector v.
The direction v along which the rate of change of /(x) at a point x0 is a maximum, can be found by
solving the following.
maximize td
J=I
f ( Xo) VJ
dX1
•
slibjectto g (v) = I, vJ = L
J=I
The Lagrangian function can be written as:
(I 1.21)
Differentiating ( I5) we get
j =l,2,3, ... , 11 ( I 1.23)
aF =1 -L," Vj, =O
- ( I 1.24)
ail J=I
Fr,o m Equation ( 11.23) we can write
j =l, 2, 3, .. . ,11 ( 11.25)

Substituting Equation ( 11.25) into Equation (11.24) we have
1- • _ I [cJf(xo)]l =0 ( I 1.26)
L,
}=I
4A1 ax J
2
A =¾IVJ(x0 }i2 {I 1.27)
A =±.!.IVf (x0 )I ( I 1.28)

2 .
Substituting Equation ( I 1.28) into Equation ( 11.25) we get
( 11.29)
The plus sign is for the rate of maximum increase off (x) and the minus sign is for the rate of max-
imum decrease.
11.13 Direction of the Steepest Descent (Ascent)

In multidimensional cases in R • we are dealing with II number of variables. Finding the miuin1Um
(maximum) of a function in R" can be computationally very challenging. One would like to find
the minimum (maximum) of a function in multidimensional space fo the fewest number of steps.
This can be accomplished by finding a direction of the steepest descent (ascent) and taking as many
steps as possible along that direction without overshooting. Let us explain this with the help of the
fo1Jowing unconstrained example. T he function chosen for this example is intentionally kept simple
so that we may explain the concept in an effective way.
Example 11.S
Minimize f(x.,x 1 )=(l.Sx, - .r2 )? +(x1 - 3}2 + 7
Solution
The gradient is:
( 11.30)
Let us start our descent from a point P0 = (6, 7). At P0• f (x 1, x2 ) = 27. The gradient vector of
f{x,,x 1 ) at P0 is:
V0 = 4.5(6) - 3(7) ] = [ 6 ] or 6x +4x,

1
[ - 3(6)+4(7) - 6 4 -
The unit grailient vector, v0 would be v0 = 0.832x1 +0.555x2 • Since the objective is to minimize, we
have to move in the opposite direction of the unit gradient vector.
Vector -v O extends one unit ilistance from the point P0 (Figure l l. 12) in the direction of steepest
dc:scenl. Therefore, we should move in the direction of -v0 • The question, however, is how many
nwnber of units in the ilirection of- vO? Because, with too many s teps we may surpass the minimum.
Let us take an arbitrary number of U!Jlits in the direction of -v0, say IO units. This wiU place us at the
point P, as shown in Figure l l.12.
P, = 6x, + 7x2 -1 0(0.832x, +0.555x 2 )

= - 2.32lx1 +L453x2 or. (- 2.321, 1.453)
At P,. f(x 1,x2 )=33.735. It is obvious d1at by moving from Fo to P, we have moved too far. Instead
of decreasing, we actually increased the value of the fw1ction. Therefore, we have to find a way to
correct our course. Which we can do by determining the gradient vector of .f(x,, x1 ) at P,.
=[ 4.5(- 2.321) - 3(1.453) ]=[ - 14.801 ] or -1 4.801.t +6.774x

V1 1 2
- 3(- 2.321)+4(1.453) - 6 6.774
The unit gradient vector, v, would be v, =--0.909x1 +0.416x2
..
.... . u
x,
.. •
Figure 11 .12 Gradient direction

Since in our previous step we have passed the minimum, we will take a shorter step this time along
the direction of -v 1, say 2 units. We will move from P, to a new point P2 .
P2 = - 2.32Jx, + 1.453x2 - 2(-0.909x, + 0.416x2 )=-0.502x, + 0.62lx2
With our most recent move. the value of the function bas decreased compared to the previous step.
We will continue wid1 the steps until we reach the minimum. However, we need some technique to
determine how far to move at each step without moving too far from the minimum.
11.13.1 Steepest descent (ascent} method: one variable at a time

It is desirable to follow the direction of a gradient vector in such a way that we can avoid unneces-
sary steps. Steepest descent (ascent) utilizes gradient without moving too far from die minimum of
a function. Let us use the function used in the previous example to illustrate the method.
Example 11 .6
Solutio11
The gradient of f(x,,x 1 ) is:
ilf
Vf(x" x1 )=
ax,
"c}f
il.x1
=[
4.Sx, - 3x2
- 3x1 +4X1 - 6 ]
The gradient vector off (xi, x 1 ) at Po= (6, 7) is:
The partial derivative of f(x 1 , x1 ) at P0 =(6, 7) iudicates that the function, f(x, , x1 ) is increasing
at a rate of 6 units per unit increase in x 1 and increasing at a rate of 4 units per unit increase in x 1 •
Therefore, in order to move away from P0 = (6,7) it would be advantageous for us to decrease
both x 1 and x 1 . Since we are permitted to change only one variable, we will select die variable that
Chapter 11: Optim ization
will produce a greater contribution towards the minimization of f(x,.x 2 ). At poim P0 =(6, 7), we
will have a greater decrease in / (x,, x 2 ) with a unit decrease in x, than with a unit decrease in x2 •
Therefore, hold x 2 at 7 and increase x, by some increment. Next, determine by how much should
we decrease x,?
Let us decrease x, by subtracting an increment of r. The oew value for f (x,, x 1 ) will be
f(6 - r, 7)=2.25r2 - 6r+27 (I 1.31)
We will select t such that it causes the expression (I 1.31 ) to be the minimwn. At 1 = 1.333 the expres-
sion (I I.3 I) becomes the minimum. Therefore, our new point P, is
P, =(6- t, 7)=(4.667, 7)
At P, Lhe gradient off(x,, x 2 ) is:
'iJJ
ax,
V, =
'iJJ =[ ~]
dX2
The partial derivative of f(x,,x 2 ) at P, =(4.667, 7) indicates that the function, f(x,,x 2 ) remains
unchanged with respect to x, but is increasing at a rate of 8 units per unit increase in x 2 and. We,
therefore, decrease x 2 by I io the next move. The new value for f (x,. x 2 ) will be
f (4.667, 7 - ,)=2t 2 - 7.999r+23 ( I I.32)
We will select I such that it causes the expression ( 11.32) to be the maximum. At 1 = 2 the expression
(I 1.32) becomes the minimum. Therefore, our oew point P2 is
P1 =(4.667, 7 - 2)=(4.667,5)
We can continue to find new points in this fashion. Each successive iteration will yield less improve-
ment in the value of f (x,, x 2 ) and we can stop when the improvement is less than or equal to a
pre-specified tolerance.
11.13.2 Steepest ascent (descent) method: multiple variable at a time

It is clear from our previous discussions that die gradient of die objective function / ( x) at any point
xis a vector io the direction of the greatest local increase in f (x). Therefore, one would proceed in
the direction of die gradient in order to arrive at the maxinmm point. If the search is for d1e minimum
point, d1en one has to proceed in the negative direction of the gradient. Therefore at x(•>, the direction
of the steepest ascent is:
( I 1.33)
where
vl•>is a wlit vector in the direction of the steepest ascent and
VJ(x1'>) is the gradient vector off (x) at xl•J_
In search of the maximum, therefore, the transition from x<•Jto x<>+•J will be as following:
(I 1.34)
where A<•l is a scalar multiplier.

The question is what should be the size of At•>·? There are three general methods available for the
selection of A<•>. In one method, a fixed value is selected for A. In the second method, a variable
vaJue for A is selected. The size being gradually decreased as one moves closer to the maximum
(minimum) point. lo the third method, the objective function is maximized (nlinimized) with respect
to A in order to move from x<•l to x<•.a>_ln this section, the third melhod is discussed.
The objective function / (x) can be maximized with respect to A by solving the following equation.
df ( x<•> +iv<•>)
0 ( I 1.35)
dl
Let us assume that / (x) is a quadratic function. The hessian for a quadratic function remain
unchanged. By utilizing Taylor"s expansion, f (x), therefore, can be written as:
( I 1.36)
By inserting x<•·•> - x<•>= Avt•> in Equation ( 11.36) we get
( 11.37)
Differentiating Equation ( 11.37) with respect to A we get
( I 1.38)
From Equation ( 11.38), A can be expressed as:
[v1 (x<•1 )r (v11

' )
( I 1.39)
(v<•lr H (v<•>)
At this stage, let us discuss an interesting feature of the maximization (minimization) of a quadratic
function. The next search direction is onhogonal to the current search direction. In other words,
Vf (xl•••>) is onhogonal to v<'>. This can be demonstrated in the following manner.
If f (x) is a quadratic functio°' it can be expressed as:

f (x) = a+ xTb+-½xTHx
For a quadratic function, elements of the Hessian matrix, H are either constant or zero.
The gradient of f (x) is
V/(x)= b+Hx
Therefore,
( 11.40)
Substitute Equation (11 .40) into Equation ( 11.38).
[ b + Hx<•l r( v<•l ) + ,1,<•l ( v<•> )TH( v<•>) = 0 ( ll.41)
Introducing x <t+iJ -xl•I for ,1. t•Jv <•J im Equation (11 .4 I) we get
[ b + Hxl') r( vlkl) +A.(>) ( vC•J rH( vl•l) = 0
[ b+ Hx<•>r (v 1•>) + (x(k+I) - xl•>f H(v1•>) = 0
The Hess.ian matrix, H is a symmetric matrix. Therefore, HT= H. Amd, therefore we can write
Ta.king transpose,
[ vl•> r [ bT +[x(k+l)r Hr= 0

or [ vl•) rVJ( x<•+I)) = 0 ( ! 1.42)
Eq uation ( I 1.42) indicates that the gradien t at x<•+i) is orthogonal to the previous search
direction, vl'l.
Example 11.7
Solution
Iteration l
We start from xt J
0
=[ ~ ].
Vf(x(Ol )=[ ~.5x 1 - 3x1 ]=[ 6 ]
- 3.t 1 +4x~ - 6 4
Unit vector in the direction of the gradient is v<0> =[ 00.555

-
832
].
H=[ ~~ -! ]
A,(O) =([ 6 ]" [ 0.832 ])([ 0.832

4 0.555 0.555
]T[ 4.5 - 3 ][ 0.832
-3 4 0.555
])-I= -4.573
x(1) +
=x<o> A-(Olv(o) =[ 6
7
]+ 4 _573 [ 0.832 ]
0.555
=[ 2. I95 ]
4.463
A(Ol is negative, meaning we are moving towards the minimizi ng point by going in the opposite
direction of the gradient vector.
After the first iteration the value of the function is I0.5 I 2.
Iteration 2
Um.t vector ID
. th e directton
. o fthe grawent
..1: •
1s V'11) = [-0.555]
_ •
0 832
H=[ - 3 4.5 - 3 ]
4
A.(I) =([ - 3.512 ]T[ -0.555 ])([ -0.555 ]T[ 4.5 - 3 J[ - 0.555 ])-I=-0.9 [5
5.268 0.832 0.832 -3 4 0.832
x i?) = xl•l +A!'lv!•l =[ 2. 195 ]+o_915 [ -0.555 ]=[ 2.702 ]

4.463 0.832 3.702
After the second iteration, the value of the function is 7 .617.
After the 7w iteration.
x111 = xl6l + A(6lvC~> = [ 2.022 ] + 0 _025[ 0.832 ] = [ 2.00 I ]

3.022 0.555 3.008 .
The minimum value of the function, after the 7w iteration is 7.
Problem Set 11 .5
Utilizing (a) multivariate grid search, (b) univariate search method, (c) steepest ascent (descent)
- one variable at a time and (d) steepest ascent (descent) - multiple variable at a time find the maxi-
mum (minimum) of the following functions.
1. f (x,, xi) =(x, - 3x2 }2 + (2x, - 5}2 + 13, 0 ~x, ~ IO and O~x2 ~10
2 2
2. /(x1 , x2 ) =(2x, - 3x2 ) + (4.:r2 - 7) - 5, - 5 ~x, ~5 and - 5 ~x2 ~5
11 .14 Constrained Optimization

Equality Constraints
Am optimization problem with equali ty constraints (often referred to as hard constraillfs) can be
specified as
minimize z =J( x)=J(x, ,x 2 , ••• ,x.) (11.43)
subject to g, ( x) =g1 (x,, x2 , ••• , x,.) =b1

(11.44)
i=L,2, ... ,111
The Lagrangian function corresponding to Equations (11.43) and ( 11.44) is
F(x.l)=J(x)+ I,11 [b1 - g1 (x)] ( 11.45)

f=J
The necessary conditions for a point [ X•• ,r r to maximize F ( X, A) are,
k =1,2, ... , II (11.46)
i = 1, 2, ... , Ill ( 11.47)
Example 11.8
• •
llllillJJllZe
• 2
;: = 5x1 + 3x 22
subject to x , + 2x 2 = 16
Solution.
First, fonn the Lagrangian function as:
oF(x, A)
( I 1.48)
ax,
(1 1.49)
_a F_('--x_
,A.c...) - 16 - x - 2x =0 ( 11.50)
dA. ' 2
Solving Equations ( 11.48), ( I 1.49) and ( I 1.50) we get
48 160
X = -23 X =-
• I ? 23
Therefore. the corresponding minimum value of z is: 166.957.
Inequality Constraints
Many optimization problems come with inequality constraints (often referred to as soft co,mrai111s).
The general approach is to convert all inequality constraints to equality constraints and fonn a
Lagrangian function and follow the same technique as discussed earlier.
Assume we have the following problem:
maximize J(x,.~)
subject to g(x,. x 2 ) ~ 0
Convert the inequality constraint to an equality constraint by introducing a new variable, s (often
referred to as slack vari.able) in the following manner:
The variable s is squared to ensure that s2is always positive regardless the value of s. With this
modification, the equivalent problem becomes:
minimize J(x1 , x2 )
subject to g(x,.x 2 )- s2 =0
The Lagrangian function is:
We take partial derivatives of the Lagrangian function and equate them to zero.
-
aF -_-
at- 11,-
, i)g -_ 0 ( 11.51)
ax, ax, ax,
( 11.52)
( 11.53)
( 11.54)
Condition ( 11.54) states that if 2,ls = 0, either A or s or both equal to zero. Ifs is zero then from
condition ( 11.53) we can write that g (x, , xi)= 0.
Multiply both sides of condition ( 13.d) bys. We get
AS1 =0
( 11.55)
The essential conditions, then are (I 1.51), (I 1.52), (I 1.55) and the original inequaliry constraint.
These conditions are known as Kuhn-Tucker conditions for the optimization of a function subject to
an inequality constraint.
The Kuhn-Tucker conditions are summarized in the following:

a. aJ - ,log =0
ax, ax,
b. of - ,lag =0
dX1 dX2
C. ,!,g(x,.x2 )=0
d. g ( Xi, X2 ) ;:: 0
Example 11.9
minimize f(x,, x"2)=2x, +x,x1 +3x2
subject to x~ + x 1 ;:: 3
Solutio11
The inequality constraint can be rewritten as:
Referring to the Kuhn-Tucker conctitions we can write
a. 2+x1 - 2Ax1 =0
b. 3+x1 - A.=0
First, we will assume that the constraint will be satisfied. In that case we will then set ,l = 0 and solve
(a) and (b) for x 1 and x 2• We get x, = - 3 x 1 = - 2. We check 10 see if the constraint is satisfied. In
!his case lhe constraint is satisfied and, therefore, x1 =- 3, x2 =- 2 will be our minimum point. The
constrained minimum of the function is --6.
Example 11.10
minimize f(x,, -~) = 2x1 +x1x 2 +3-½
subject to x, + x 1 ;:: 3
Solution.
The inequality constraint can be rewritten as:
g(x,,x2 )=x1 +.t'1; - 3;;::0
Referring to the Kuhn-Tucker conditions we can write

a. 2+x1 - A=0
b. 3+x, - A-=0
c. it(x, +x1 - 3) =0
Like we did in Example 11 .5, first, we will assume that the constraint wW be satisfied. In that
case we wiU then set it= 0 and solve (a) and (b) for x 1 and x2 • We get x, = - 3, x 1 = - 2. With d1ese
vaJues the constraint is not satisfied. Therefore we now set the inequality constraint as an equality
constraim and set A-:/! 0. In that case we wW have to solve (a). (b) and {c) for x,, x 1 and ,l. We get
x, = I, x 1 = 2, it= 4. T he constrained minimum of dle function is I0.
Problem Set 11.6

1. Maxim.ize f(x,, x 2 , x 3 ) = x,x2 x 3 subject to x; + 2xi +xJ = 29.
2. Maximize f(x, , x 1 ) = x, + 2x1 subject to 4x1 + 8x2 = l I.
3. Maxim.ize f(x,, x 2 )=2x, + x,x2 +3x 2 subjectto x; + x 1 =3
4. Minimize f ( x,, x 1 ) = 2x1 + x 1x 1 + 3x2 subject to x, + x 2 <:: 9

600
5. Minimize f(x,, x 1 )= ~ +30x1 + 15x 1x2 + 15x subject to - 250~0
~~ ~~
_J
Kendall E. Atkinson, An lmroduc1ion to Numerical Analysis, John Wiley & Sons, lnc., 1978
Claude McMillan, Jr. , Mathematical Programming, 2ndE, John Wiley & Sons, Inc., 1975
William E. Boyce and Richard C. DiPrima, Elementary Differential £q11aiiom and Bo11ndary Value
Problems, 9E, John Wiley & Sons, Inc., 2009
Singiresu S. Rao, Applied Numerical Methods for Engineer and Sciemists, Prentice Hall, 2002
Richard L. Burden, J. Douglas Faires and Annette M. Burden, Numerica.f Atwlysis, IOtbEd,
CENGAGE Learning, 2015
Robert Schilling and Sandra L. Harris, Applied Numerical Methods f-or Engilleers Using MAII..A/11'
and C, Brooks/Cole Publishing Company, I 999
Thomas L. Hannan, James B. Dabney and Norman J. Richert, Adva11ced Engineering Mathematics
with MAII..A/11', 2nc1Ed, Brooks/Cole Publishing Company, 2000
Kendall Atkinson and Weimin Harn, Elementary Nw11eriwl A,wlysis, 3rc1Ed, John WUey & Sons,
Inc. , 2004
Nakhle Asmar, Parlial D(/ferential Equations and B011nda1y Value Problems, Prentice Hall, 2000
Matheus Grasselli and Dmitry Pelinovsky, Numerical Mathemalics, Jones and Bartlett Publishers,
Inc., 2008
P. Venkataraman, Applied Optimization with Matlab"' Progmmming, John Wiley & Sons, Inc., 2002
Ronald L. Rardin, Optimization In Operations Research, Prentice Hall, 1998
Stephen W. Goode, Differential Equations and Linear Algebra, 2nc1Ed, Prentice Hall, 2000

Nurul A. Chowdhury - Numerical Methods

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Nurul A. Chowdhury - Numerical Methods

Uploaded by

Copyright:

Available Formats

University ofSaskatchewan

Cover image© Shuuerstock, Inc.

Copyright © 2019 by Kendall Hunt Publis.bing Company

All eights reserved. No part of this publkation may be reproduced,

Published in the United States of America

Chapter 1 POLYNOMIAL APPROXIMATIONS AND TAYLOR POLYNOMIALS ....... 1

Chapter 2 SOLUTIONS OF EQUATIONS IN ONE VARIABLE ........................... 11

2.5 Brent's Algorithm ....................................................................................... 33

Chapter 3 SOLUTIONS OF LINEAR SYSTEMS OF EQUATIONS ....................... 47

Chapter 4 ITERATIVE TECHNIQUES FOR SOLVING LINEAR AND

Chapter S INTERPOLATION •...................................•......................•..... 103

Chapter 6 NUMERICAL DIFFERENTIATION AND INTEGRATION ................. 145

Chapter 7 INITIAL-VALUE PROBLEMS FOR ORDINARY

7.3 Backward Euler's Method ........................................................................ 178

Chapter 8 PERIODIC FUNCTIONS AND FOURIER SERIES .......................... 201

Chapter 9 EIGENVALUES AND EIGENVECTORS AND THEIR

Chapter 11 OPTIMIZATION •.....................................•............................ 287

References ...................................................................................................................................... 323

where a 0 , a, , a2 , • • • • a,. are coefficients and x is a variable.

1.1 Taylor Polynomials

Let P, (x)=h0 +h,x

P,'(a)=b, =f'(a) (1.4)

From Equations ( 1.3) and ( I .4), we get

Rearranging Equation (1.5),

To satisfy the conditions, we may write

P;(a) = h, +2b1 a =f'(a) (1.9}

P;(a) = 2b1 = f'(.a) (1.10)

From Equation ( I. LO), b2 = ½/' (a)

From Equation ( 1.8), h0 = f(a) - h,a - b1a 2

h0 = f (a) - a{f'(a) - a · f'(a)} - a

Therefore, the quadratic polynomial ( l.7 ) can be written as

= f (a) - a -J'(a) + ..!._ -a 1 · f"(a)

Pi (.r) is approximately sinx when ,Y is near a. •

P,, (x)= I, (x ~,a)' pi> (a) ( 1.13)

Table 1.1 Taylor polynomial approximations to sinx

X sinx P, ( x ) P2(X) P3(X) lsinx - P. (x )I lsmx-P2 ( x~ lsinx-P3 ( x~

P,,(x)=e"{ I+ (x - a)+, I(x - a) "} =e. £..,

where R. ( x) is called the 11th-degree remainder for f (x) at x =a.

(n/4 )"'' ~ w-s (I.I 9)

f(x)= f(a)+(x - a)f'(a)+ (x - a)2 f'(a)+ (x - a)3 f 3>(a)

1.3 Taylor Series

1.4 Maclaurin Series

Problem Set 1.1

31. /(x)=ln(l+x), a=5

2.1 The Bisection Method

Figure 2.1 Bisection algorithm

IP. - p.J <e P. ;t:O (2.2)

Table 2.1 Results of the bisection method

Table 2.1 Results of the bisection method (Cominued)

II a. b• P. Sign of S ign of Sign of

2.1.1 Error bound in the bisection method

Take 10 base log on both sides.

or, (2 - n) log'° 2 < log111 I o-s

It is evident from Table 2.1 that the root is within l o-s

Problem Set 2.1

2.2 Fixed-point Iteration

Figure 2.2 Intersection of two functions

or. IP- 11= lg(p) - g(r)I (2.4)

g(p) - g(r) g'(,s)

:. g(p) - g(r) = g'(s)(p - r)

i.e., lg(p) - g(rll ~lg'(s)ll<p- r)I

i.e., lg(p) - g(r)I ~ kl(p - r)I

i.e., lg(p) - g(r)I <l(p- r)I (2.5)