List of Formulas

True percent relative error

True value − approximation

εt = 100%
True value

Approximate percent relative error

Current approximation − Previous approximation

εa = 100%
Current approximation

Stopping criterion – Terminate computation when ε a < ε s where ε s is a pre-specified

percent tolerance. If this relationship holds, the percent relative error of our results is assumed
to be within the pre-specified acceptable level ε s . Suppose you want your result to be correct
to at least n significant figure then set ε s = (0.5 × 10 2−n )%

Bisection Method
x + xu
xr = l
2

False-Position Method

f ( xu )( xl − xu )
x r = xu −
f ( xl ) − f ( xu )

Simple Fixed-Point Iteration Method

xi +1 = g ( x )

Newton-Raphson Method

f ( xi )
xi +1 = xi −
f ' ( xi )

Secant Method

f ( xi )( xi −1 − xi )
xi +1 = xi −
f ( xi −1 ) − f ( xi )
Gauss Elimination

 a11 a12 a13 : b1  a11 a12 a13 : b1 

Elementary Row Operations
a a 22 a 23 : b2  ⇒ 0 '
a 22 '
a 23 : b2' 
 21 
 a31 a32 a33 : b3   0 0 ''
a33 : b3'' 
''
x3 = b33'' / a33
( )
Backward Substitution
⇒ x 2 = b2' − a 23' '
x3 / a 22
x1 = (b1 − a12 x 2 − a13 x3 ) / a11

Gauss-Jordan

 a11 a12 a13 : b1  1 0 0 : b1( n ) 

 (n) 
Elementary Row Operations
a
 21 a 22 a 23 : b2  ⇒ 0 1 0 : b2 
 a31 a32 a33 : b3  0 0 1 : b3( n ) 
 
x1 = b1( n )
Backward Substitution
⇒ x 2 = b2( n )
x3 = b3( n )

LU Decomposition

Doolittle

 a11 a12 a13   1 0 0 u11 u12 u13 

a a22 a23  = l21 1 0  0 u 22 u 23 
 21
 a31 a32 a33  l31 l32 1  0 0 u33 

 a11 a12 a13   u11 u12 u13 

a a22  
a23  = l21u11 l21u12 + u 22 l21u13 + u 23 
 21 
 a31 a32 a33  l31u11 l31u12 + l32u 22 l31u13 + l32u 23 + u33 

a11 = u11 ⇒ u11 = a11

a12 = u12 ⇒ u12 = a12
a13 = u13 ⇒ u13 = a13
a21 = l 21u11 ⇒ l 21 = a 21 / u11
a31 = l31u11 ⇒ l31 = a31 / u11
a22 = l 21u12 + u 22 ⇒ u 22 = a 22 − l 21u12
a23 = l 21u13 + u 23 ⇒ u 23 = a 23 − l 21u13
a32 = l31u12 + l32 u 22 ⇒ l32 = (a32 − l31u12 ) / u 22
a33 = l31u13 + l32 u 23 + u33 ⇒ u 33 = a33 − l31u13 − l32 u 23
Crout

 a11 a12 a13  l11 0 0  1 u12 u13 

a a22 a23  = l21 l22 0  0 1 u 23 
 21
 a31 a32 a33  l31 l32 l33  0 0 1 

 a11 a12 a13   l11 l11u12 l11u13 

a a22  
a23  =l21 l21u12 + l22 l21u13 + l22l23 
 21
 a31 a32 a33  l31 l31u12 + l32 l31u13 + l32u23 + l33 

a11 = l11 ⇒ l11 = a11

a21 = l12 ⇒ l 21 = a 21
a31 = l13 ⇒ l31 = a31
a12 = l11u12 ⇒ u12 = a12 / l11
a13 = l11u13 ⇒ u13 = a13 / l11
a22 = l21u12 + l22 ⇒ l 22 = a 22 − l 21u12
a32 = l31u12 + l32 ⇒ l32 = a32 − l31u12
a23 = l21u13 + l22u 23 ⇒ u 23 = (a 23 − l 21u13 ) / l 22
a33 = l31u13 + l32u 23 + l33 ⇒ l33 = a33 − l31u13 − l32 u 23

Cholesky

 a11 a12 a13  l11 0 0  l11 l21 l31 

a a22 a23  = l21 l22 0   0 l22 l32 
 21
 a31 a32 a33  l31 l32 l33   0 0 l33 

 a11 a12 a13   l112 l11l 21 l11l31 

a   
 21 a22 a23  = l 21l11 l 21 + l 222
2
l 21l31 + l 22 l32 
 a31 a32 a33  l31l11 l31l 21 + l32 l 22 l312 + l322 + l332 

a11 = l112 ⇒ l11 = a11

a21 = l21l11 ⇒ l 21 = a21 / l11
a31 = l31l11 ⇒ l31 = a31 / l11
a22 = l212 + l222 ⇒ l 22 = a22 − l 212
a32 = l31l21 + l32l22 ⇒ l32 = (a32 − l31l 21 ) / l 22
2 2 2
a33 = l + l + l
31 32 33 ⇒ l33 = a33 − l312 − l322
Gauss-Seidal

b1 − a12 x2 − a13 x3
x1 =
a11
b − a21 x1 − a23 x3
x2 = 2
a22
b −a x −a x
x3 = 3 31 1 32 2
a33

Linear Regression

y = a 0 + a1 x

where,

n∑ xi yi − ∑ xi ∑ yi
a1 =
n∑ xi2 − (∑ xi )
2

a0 = y − a1 x

Polynomial Regression

y = a0 + a1 x + a2 x 2

where,

(n )a0 + (∑ xi )a1 + (∑ xi2 )a2 = ∑ yi

(∑ x )a + (∑ x )a + (∑ x )a = ∑ x y
i 0
2
i 1
3
i 2 i i

(∑ x )a + (∑ x )a + (∑ x )a = ∑ x
2
i 0
3
i 1
4
i 2
2
i yi

Newton’s Divided Difference Interpolating Polynomials

f n ( x ) = b0 + b1 ( x − x0 ) + ... + bn ( x − x0 )( x − x1 )...( x − xn −1 )

where,

b0 = f ( x0 )

b1 = f [x1 , x0 ]

b2 = f [x2 , x1 , x0 ]

.
.

b1 = f [xn , xn−1 ,..., x1 , x0 ]

i xi f ( xi ) First Second Third

0 x0 f ( x0 ) f [x1 , x0 ] f [x2 , x1 , x0 ] f [x3 , x2 , x1 , x0 ]
1 x1 f ( x1 ) f [x2 , x1 ] f [x3 , x2 , x1 ]
2 x2 f (x2 ) f [x3 , x2 ]
3 x3 f ( x3 )

Lagrange Interpolating Polynomials

n
f n ( x ) = ∑ Li ( x ) f ( xi )
i =0

where,
n x − xj
Li ( x ) = ∏
j =0 xi − x j
j ≠i

Numerical Differentiation

Forward finite divided difference

First Derivative

f ( xi +1 ) − f ( xi )
f ' ( xi ) =
h

Second Derivative

f (xi + 2 ) − 2 f (xi +1 ) + f (xi )

f '' (xi ) =
h2

Backward finite divided difference

First Derivative

f ( xi ) − f ( xi −1 )
f ' ( xi ) =
h
Second Derivative

f ( xi ) − 2 f ( xi −1 ) + f ( xi − 2 )
f '' ( x i ) =
h2

Centered finite divided difference

First Derivative

f ( xi +1 ) − f ( xi −1 )
f ' ( xi ) =
2h

Second Derivative

f ( xi +1 ) − 2 f ( xi ) + f ( xi −1 )
f '' ( x i ) =
h2

High-Accuracy Differentiation

Forward Finite Divided Difference

First Derivative

− f (xi + 2 ) + 4 f (xi +1 ) − 3 f (xi )

f ' ( xi ) =
2h

Second Derivative

− f (xi +3 ) + 4 f (xi + 2 ) − 5 f (xi +1 ) + 2 f (xi )

f '' (xi ) =
h2

Third Derivative

− 3 f (xi + 4 ) + 14 f (xi +3 ) − 24 f (xi + 2 ) + 18 f (xi +1 ) − 5 f (xi )

f ''' (xi ) =
2h 3

Fourth Derivative

− 2 f (xi +5 ) + 11 f (xi + 4 ) − 24 f (xi +3 ) + 26 f (xi + 2 ) − 14 f (xi +1 ) + 3 f (xi )

f '''' (xi ) =
h4

Backward Finite Divided Difference

First Derivative

3 f (xi ) − 4 f (xi −1 ) + f (xi −2 )

f ' (xi ) =
2h
Second Derivative

2 f (xi ) − 5 f (xi −1 ) + 4 f (xi −2 ) − f (xi −3 )

f '' (xi ) =
h2

Third Derivative

5 f (xi ) − 18 f (xi −1 ) + 24 f (xi −2 ) − 14 f (xi −3 ) + 3 f (xi −4 )

f ''' ( xi ) =
2h 3

Fourth Derivative

3 f ( xi ) − 14 f ( xi −1 ) + 26 f ( xi −2 ) − 24 f ( xi −3 ) + 11 f ( xi −4 ) − 2 f (xi −5 )
f '''' ( xi ) =
h4

Centered Finite Divided Difference

First Derivative

− f (xi + 2 ) + 8 f (xi +1 ) − 8 f (xi −1 ) + f (xi −2 )

f ' (xi ) =
12h

Second Derivative

− f (xi + 2 ) + 16 f (xi +1 ) − 30 f (xi ) + 16 f (xi −1 ) − f (xi −2 )

f '' (xi ) =
12h 2

Third Derivative

− f (xi +3 ) + 8 f (xi +1 ) − 13 f (xi +1 ) + 13 f (xi −1 ) − 8 f (xi −2 ) + f (xi +3 )

f ''' (xi ) =
8h 3

Fourth Derivative

− f ( xi +3 ) + 12 f ( xi +1 ) − 39 f ( xi +1 ) + 56 f ( xi ) − 39 f (xi −1 ) + 12 f (xi −2 ) − f (xi +3 )

f '''' ( xi ) =
6h 4

Numerical Integration

Trapezoidal

f ( a ) + f (b)
I
= (b − a )
2
Multiple-Application of Trapezoidal Rule
n −1
f ( x 0 ) + 2 ∑ f ( xi ) + f ( x n )
I = (b − a ) i =1

2n

Simpson’s 1/3 Rule

f (x0 ) + 4 f (x1 ) + f (x2 )

I ≅ (b − a )
6

Multiple-Application of Simpson’s 1/3 Rule

n −1 n−2
f ( x0 ) + 4 ∑ f ( xi ) + 2 ∑ f (x ) + f (x )
j n

I ≅ (b − a )
i =1, 3, 5 j =2, 4, 6

3n

Simpson’s 3/8 Rule

f (x0 ) + 3 f (x1 ) + 3 f (x2 ) + f (x3 )

I ≅ (b − a )
8

Runge-Kutta Methods

First-Order Runge-Kutta Method (Euler’s Method)

yi +1 = yi + f ( xi , yi )h

dy
where f (x, y ) =
dx

Second-Order Runge-Kutta Methods

Heun’s Method / Heun Method with a Single Corrector

Predictor: yi0+1 = yi + f ( xi , yi )h

Corrector: yi +1 = yi +
f ( xi , yi ) + f xi +1 , yi0+1 (
h
)
2

dy
where f (x, y ) =
dx

or simply
 1 1 
yi +1 = yi +  k1 + k 2 h
2 2 

where

k1 = f ( xi , yi )
k 2 = f ( xi + h, yi + k1h )

Midpoint Method

yi +1 = yi + k 2 h

where

k1 = f ( xi , yi )
 1 1 
k 2 = f  xi + h, yi + k1h 
 2 2 

Ralston’s Method

1 2 
yi +1 = yi +  k1 + k 2 h
3 3 

where

k1 = f ( xi , yi )
 3 3 
k 2 = f  xi + h, yi + k1h 
 4 4 

Third-Order Runge-Kutta Methods

1
yi +1 = yi + (k1 + 4k 2 + k3 )h
6

where

k1 = f (xi , yi )
 1 1 
k 2 = f  xi + h, yi + k1h 
 2 2 
k3 = f (xi + h, yi − k1h + 2k 2 h )
Fourth-Order Runge-Kutta Methods

1
yi +1 = yi + (k1 + 2k 2 + 2k3 + k 4 )h
6

where

k1 = f (xi , yi )
 1 1 
k 2 = f  xi + h, yi + k1h 
 2 2 
 1 1 
k 3 = f  x i + h, y i + k 2 h 
 2 2 
k 4 = f ( x i + h, y i + k 3 h )

Fifth-Order Runge-Kutta Methods

1
yi +1 = yi + (7k1 + 32k3 + 12k 4 + 32k5 + 7k6 )h
90

where

k1 = f (xi , yi )
 1 1 
k 2 = f  xi + h , y i + k1h 
 4 4 
 1 1 1 
k 3 = f  xi + h , y i + k1h + k 2 h 
 4 8 8 
 1 1 
k 4 = f  xi + h, yi − k 2 h + k3 h 
 2 2 
 3 3 9 
k5 = f  xi + h, yi + k1h + k 4 h 
 4 16 16 
 3 2 12 12 8 
k 6 = f  xi + h, yi − k1h + k 2 h + k3 h − k 4 h + k5 h 
 7 7 7 7 7 

Laplace Equations

Ti +1, j + Ti −1, j + Ti , j +1 + Ti , j −1
Ti , j =
4

Overrelaxation

Ti ,new
j = λTi ,Current
j + (1 − λ )Ti ,oldj
Heat-Conduction Equation

Explicit method,

(
Ti l +1 = Ti l + λ Ti +l 1 − 2Ti l + Ti −l 1 )
∆t
where λ = k
(∆x )2
Implicit method

− λTi −l +11 + (1 + 2λ )Ti l +1 − λTi +l +11 = Ti l

∆t
where λ = k
(∆x )2