Simplified Numerical Analysis

Fourth Edition

Amjad Ali, Ph.D.

Centre for Advanced Studies in Pure and Applied
Mathematics (CASPAM)
Bahauddin Zakariya University (BZU), Multan,
Pakistan

Companion Books:
Laboratory Manual of Simplified Numerical Analysis (C++ Version) https://zenodo.org/record/8264208
Laboratory Manual of Simplified Numerical Analysis (MATLAB® Version) https://zenodo.org/record/8266086
Laboratory Manual of Simplified Numerical Analysis (Python Version)

For codes, please visit: https://github.com/DrAmjadAli11/SimplifiedNumericalAnalysis

Prime Distributor:
NEW BOOKS N BOOKS, Hussain Arcade, Near Trend Super Store, Gol Bagh Market,
Gulgasht Colony, Multan. 061-6511828 , 061-6223080 0333-6110619 (WhatsApp)

Table of Contents
Chapter 1: Preliminary Concepts in Numerical Analysis
1.1 Intro[duction ................................................................................................................................... 1
1.2 Number Systems and Representations ........................................................................................... 9
1.3 The Round-off Error ...................................................................................................................... 14
1.4 The Truncation Error ...................................................................................................................... 30

Chapter Summary ................................................................................................................................... 31

Chapter Excercises .................................................................................................................................. 34

Chapter 2: Solution of a Nonlinear Equation in One Variable

Corridor I: BASICS
2.1 Introduction .................................................................................................................................. 35
2.2 Bracketing Methods ....................................................................................................................... 45
2.2.1 The Bisection Method (or Bolzano Method) ................................................................... 45
2.2.2 The False-Position Method (or Regula-Falsi Method) ..................................................... 55
2.3 Open Methods .............................................................................................................................. 63
2.3.1 The Fixed-Point Iteration Method .................................................................................... 63
2.3.2 The Newton-Raphson Method ......................................................................................... 70
2.3.3 The Secant Method ......................................................................................................... 76

Corridor II: ANALYSIS

2.4 Convergence Analysis ................................................................................................................... 85
The Bisection Method ..................................................................................................... 86
The Regula-Falsi Method ................................................................................................. 88
The Secant Method ......................................................................................................... 90
The Newton-Raphson Method ......................................................................................... 93
The Fixed-Point Iteration Method .................................................................................... 94
2.5 Further Discussions ....................................................................................................................... 98

Corridor III: PROGRAMMING ARCADE

2.6 Algorithms and Implementations ............................................................................................... 105

Chapter Summary ................................................................................................................................. 106

Chapter Excercises ................................................................................................................................ 110

Chapter 3: Polynomial Interpolation

Corridor I: BASICS
3.1 Introduction ................................................................................................................................ 115
3.2 The Newton’s Divided Difference Interpolation .......................................................................... 127
3.3 The Lagrange Interpolation.......................................................................................................... 134
3.4 Deriving the Lagrange Interpolation Formula from the Newton’s Divided-Difference Formula . 143
3.5 Interpolation Formulas for Equally Spaced Nodes ...................................................................... 146
3.6 Hermite Interpolation .................................................................................................................. 156
3.7 Spline Interpolation .................................................................................................................... 159
3.7.1 Linear Spline .................................................................................................................. 159
3.7.2 Quadratic Spline ............................................................................................................ 160
3.7.3 Cubic Spline ................................................................................................................... 163

Corridor II: ANALYSIS

3.8 Error of Interpolation ................................................................................................................... 170

Corridor III: PROGRAMMING ARCADE

3.9 Algorithms and Implementations ............................................................................................... 179

Chapter Summary ................................................................................................................................. 180

Chapter Excercises ................................................................................................................................ 183

Chapter 4: Numerical Integration

Corridor I: BASICS
4.1 Introduction ................................................................................................................................ 187
4.2 The Trapezoidal Rule ................................................................................................................... 189
4.3 The Simpson’s 1/3 Rule ............................................................................................................... 199
4.4 Generalized Closed Newton-Cotes Quadrature .......................................................................... 209

Corridor II: ANALYSIS

4.5 Truncation Error of the Trapezoidal Rule .................................................................................... 219
4.6 Truncation Error of the Simpson’s 1/3 Rule ................................................................................ 229
4.7 Further Discussions ..................................................................................................................... 237
4.8 The Gaussian Quadrature ........................................................................................................... 244

2
 Corridor III: PROGRAMMING ARCADE
4.9 Algorithms and Implementations ............................................................................................... 253

Chapter Summary ................................................................................................................................. 254

Chapter Excercises ................................................................................................................................ 257

Chapter 5: Numerical Differentiation

Corridor I: BASICS
5.1 Introduction ................................................................................................................................ 261
5.2 Finite Difference Approximations of Derivatives using the Taylor Series ................................... 262
5.2.1 First Order Derivatives .................................................................................................. 262
5.2.2 Second Order Derivatives .............................................................................................. 265
5.3 Listing of the Derivative Formulas .............................................................................................. 269

Chapter 6: Direct Linear Solvers

Corridor I: BASICS
6.1 Introduction to Linear Systems .................................................................................................... 275
6.2 Solving Linear Systems using the Gaussian Elimination Method ................................................ 281
6.3 Pivoting Strategies ...................................................................................................................... 285
Partial Pivoting ............................................................................................................. 286
Scaled Partial Pivoting .................................................................................................. 286
Complete Pivoting ........................................................................................................ 286
6.4 The Gauss-Jordan Method .......................................................................................................... 293
6.5 Solving Linear Systems using the LU Factorization Method ....................................................... 296
6.5.1 The Doolittle’s Method ................................................................................................. 296
6.5.2 The Crout’s Method ...................................................................................................... 304
6.5.3 The Cholesky’s Method ................................................................................................. 310

Corridor II: ANALYSIS

6.6 Operation Count Analysis ............................................................................................................ 316
6.7 Matrix Inversion ........................................................................................................................... 317

Corridor III: PROGRAMMING ARCADE

6.8 Algorithms and Implementations ............................................................................................... 319

Chapter Summary ................................................................................................................................. 319

Chapter Excercises ................................................................................................................................ 321

Chapter 7: Iterative Linear Solvers

Corridor I: BASICS
7.1 Vector Norms and Distances........................................................................................................ 325
7.2 Convergence Criteria for Linear Solvers ...................................................................................... 330
7.3 Basic Methods.............................................................................................................................. 331
7.3.1 The Jacobi Method ........................................................................................................ 332
7.3.2 The Gauss-Seidel Method ............................................................................................. 339
7.3.3 The SOR Method ........................................................................................................... 346

Corridor II: ANALYSIS

7.4 Matrix Norms and Conditioning .................................................................................................. 353
7.5 Iteration Matrix and Matrix Form of a Solver .............................................................................. 363

3
 Corridor III: PROGRAMMING ARCADE
7.6 Algorithms and Implementations ............................................................................................... 365

Chapter Summary ................................................................................................................................. 369

Chapter Excercises ................................................................................................................................ 372

Chapter 8: Eigenvalues and Eigenvectors

Corridor I: BASICS
8.1 Basic Definitions and Concepts ................................................................................................... 373
8.2 General Approach of Finding Eigenvalues and Eigenvectors ...................................................... 376
8.3 Some Numerical Methods for Eigenvalues ................................................................................. 383
The Power Method ....................................................................................................... 383
The Householder Method ............................................................................................ 386
The QR Factorization Method ...................................................................................... 388
The Sturm Method ...................................................................................................... 390

Corridor II: ANALYSIS

8.4 Further Discussions ...................................................................................................................... 391
The Power Theorem ..................................................................................................... 391
The Gerschgorin Circle Theorems ............................................................................... 397
The Singular Value Decomposition (SVD) ..................................................................... 398

Corridor III: PROGRAMMING ARCADE

8.5 Algorithms and Implementations ............................................................................................... 402

Chapter Summary ................................................................................................................................. 402

Chapter Excercises ................................................................................................................................ 403

Chapter 9: Numerical Solution of Ordinary Differential Equations (ODEs)

Corridor I: BASICS
9.1 Introduction ................................................................................................................................ 405
9.2 Solving IVPs using Single Step Methods and Multistep Methods ............................................... 408
The Euler Method ......................................................................................................... 409
The Mid-point Method (an RK2 method of Order 2) .................................................. 410
The Modified/Improved Euler Method (an RK2 method of Order 2) .......................... 411
The RK Method of order 4 (RK4) ................................................................................. 412
9.3 Solving IVPs using Predictor-Corrector Methods ........................................................................ 414
The Adams-Bashforth-Moulton Method of Order 4 .................................................... 415
9.4 Solving Systems of ODEs and Higher Order ODEs ....................................................................... 417
Using the Classical RK4 Method ................................................................................... 418
9.5 Solving Linear BVPs using the Finite Difference Method ............................................................ 423

Corridor II: ANALYSIS

9.6 Some Theoretical Concepts and Error Analysis ......................................................................... 431

Corridor III: PROGRAMMING ARCADE

9.7 Algorithms and Implementations ............................................................................................... 441

Chapter Summary ................................................................................................................................. 442

Chapter Excercises ................................................................................................................................ 443

Bibliography .......................................................................................................................................... 447

The End ................................................................................................................................................. 448
4