Numerical Differentiation

Dr. Md. Masud Rana

Professor, EEE,
Rajshahi University of Engineering & Technology
Rajshahi-6204

1
 Contents
• Forward, Backward, Central Difference
• Richardson Extrapolation

2
 Introduction
• Calculus is the mathematics of change. Because engineers must
continuously deal with systems and processes that change, calculus is
an essential tool of engineering
• Standing in the heart of calculus are the mathematical concepts of
differentiation and integration:

y f ( xi  x)  f ( xi )

x x
dy f ( xi  x)  f ( xi )
 x lim 0
dx x
b
I   f ( x)dx
a
3
 Introduction
• Integration and differentiation are closely linked processes. They are in fact
inversely related.
• Types of functions to be differentiated or integrated:
1. Simple polynomial, exponential, trigonometric (solved analytically)
2. Complex function (solved numerically)
3. Tabulated function of experimentally data (solved numerically)

•Why Numerical Calculus Is Needed

•In this case two problems may arise
•The function is very difficult to solve analytically but easy in numerically. Let an example
1 1
 a  x 2 1  b  x 
2
  b  x  dx   a  x b  x  ( a  b )sin 
 ab 


That is very difficult to solve analytically.

•In many case experimental tabulated data may be given for integration or differentiation that is easy to calculate numerically.

4
 Noncomputer Methods for
Differentiation and Integration
• The function to be differentiated or integrated will
typically be in one of the following three forms:
– A simple continuous function such as polynomial, an
exponential, or a trigonometric function.
– A complicated continuous function that is difficult or
impossible to differentiate or integrate directly.
– A tabulated function where values of x and f(x) are
given at a number of discrete points, as is often the case
with experimental or field data.

5
 Applications
• Differentiation has so many engineering applications (heat transfer,
fluid dynamics, chemical reaction kinetics, etc)

• To calculate the Centre of Mass, Centre of Gravity and Mass Moment

of Inertia of a sports utility vehicle. To calculate the velocity and
trajectory of an object, predict the position of planets, and understand
electromagnetism

6
 Taylor’s Expansion

1
f ( x  h)  f ( x)  f ( x)h  f ( x)h 2  O(h 3 )
2
1
 
f ( x ) h  f ( x  h)  f ( x )  f ( x ) h  O ( h )
2 3

2
f ( x  h)  f ( x ) 1
f ( x)   f ( x)h  O(h 2 )
h 2

7
Forward Difference Formula for f (x)

f ( x  h)  f ( x )
f ( x)  error  O (h)
h

Geometrically
f (x)

f (x)
f ( x  h)  f ( x)

x xh
8
 Backward Difference Formula for f (x)
Similarly
1
f ( x  h)  f ( x)  f ( x)h  f ( x)h 2  O (h 3 )
2

Geometrically

f ( x )  f ( x  h)

f ( x) 
f (x)

h f ( x)  f ( x  h)
error  O(h) h

xh x x
9
 Central Difference Formula for f (x)
1 1 1
f ( x  h)  f ( x)  f ( x)h  f ( x)h 2  f ( x)h 3  f ( 4 ) ( x)h 4  O ( h 5 )
2 6 4!
1 1 1
–) f ( x  h)  f ( x)  f ( x)h  f ( x)h 2  f ( x)h 3  f ( 4 ) ( x)h 4  O (h 5 )
2 6 4!
1
f ( x  h)  f ( x  h)  2hf ( x)  f ( x ) h 3  O( h 5 )
3
1
2hf ( x)  f ( x  h)  f ( x  h)  f ( x)h 3  O (h 5 )
3

10
 Central Difference Formula for f (x)

Geometrically
f ( x  h)  f ( x  h)
f ( x) 
2h f (x)

f (x)
error  O(h 2 ) f ( x  h)  f ( x  h)

2h

xh x xh x

11
 Example
f ( x)  x 3 ( f ( x)  3x 2 , f (1)  3)

Calculate f’(1) using FD, BD, CD

12
 Example (cont)
FD:
f (1.1)  f (1) error  0.31
h=0.1 f (1)   3.31
0 .1  error  h
f (1.05)  f (1) error  0.1525
h=0.05 f (1)   3.1525
0.05
BD:
f (1)  f (0.9) error  0.29
h=0.1 f (1)   2.71
0.1
f (1)  f (0.95)
h=0.05 f (1)   2.8453 error  0.1547
0.05
CD: f (1.1)  f (0.9)
f (1)   3.01 error  0.01  error  h2
h=0.1 0 .2
f (1.05)  f (0.95)
h=0.05 f (1)   3.00250 error  0.00250
0.1
13
 Example (cont)
• Remarks:
– FD, BD, CD each involves 2 function calls, 1
subtraction, and 1 division: same computation
time
– CD is the most accurate (hence, the most
recommended method)
– However, sometimes, CD cannot be applied

14
 Forward Difference Formula for f (x)
1 1
f ( x  2h)  f ( x)  f ( x)2h  f ( x)(2h) 2  f ( x)(2h) 3  O (h 4 )
2 6
1 1
–2） f ( x  h )  f ( x )  f ( x ) h  f ( x ) h 2
 f ( x ) h 3
 O ( h 4
)
2 6
1 4 1
f ( x  2h)  2 f ( x  h)   f ( x)  f ( x)h 2 [2  2  ]  f ( x)h 3 [  ]  O (h 4 )
2 3 3
f ( x)h 2  f ( x  2h)  2 f ( x  h)  f ( x)  f ( x)h3  O(h 4 )

f ( x  2h)  2 f ( x  h)  f ( x )
f ( x) 
h2
error  O(h) 15
 Backward Difference Formula for f (x)
1 1
f ( x  2h)  f ( x)  f ( x)2h  f ( x)(2h) 2  f ( x)(2h) 3  O (h 4 )
2 6
1 1
–2） f ( x  h )  f ( x )  f ( x ) h  f ( x ) h 2
 f ( x ) h 3
 O ( h 4
)
2 6
1 4 1
f ( x  2h)  2 f ( x  h)   f ( x)  f ( x)h 2 [2  2  ]  f ( x)h 3 [  ]  O (h 4 )
2 3 3
f ( x)h 2  f ( x  2h)  2 f ( x  h)  f ( x)  f ( x)h3  O(h 4 )

f ( x )  2 f ( x  h)  f ( x  2h)
f ( x) 
h2
error  O(h) 16
 Central Difference Formula for f (x)
1 1 1
f ( x  h)  f ( x)  f ( x)h  f ( x)h 2  f ( x)h 3  f ( 4 ) ( x)h 4  O ( h 5 )
2 6 4!
1 1 1
+) f ( x  h)  f ( x)  f ( x)h  f ( x)h 2  f ( x)h 3  f ( 4 ) ( x)h 4  O (h 5 )
2 6 4!
1 ( 4)
f ( x  h)  f ( x  h)  2 f ( x )  f ( x)h 2  f ( x)h 4  O(h 6 )
12

f ( x  h)  f ( x  h)  2 f ( x )
f ( x)  2
 O ( h 2
)
h

Similar remark on the selection of FD|BD|CD applies for f”(x)

17
More Accurate FD Formula for f (x)

1
f ( x  h)  f ( x)  f ( x) h  f ( x)h 2  O (h 3 )
2
1 f ( x  2h)  2 f ( x  h)  f ( x )
f ( x  h)  f ( x)  f ( x)h  h 2 [ 2
 O ( h )]  O ( h 3
)
2 h
1 1
 f ( x)  f ( x)h  f ( x  2h)  f ( x  h)  f ( x )  O ( h 3 )
2 2
1 3
 f ( x  2h)  2 f ( x  h)  f ( x )
f ( x)  2 2  O(h 2 )
h

 f ( x  2h)  4 f ( x  h)  3 f ( x)
f ( x)   O(h 2 )
2h
18
 More Accurate FD Formula (cont)

• Better accuracy can be achieved using this formula

• But, it involves more computations:
– 3 function calls, two +/–, one division
• Trade-off:
– More computation is the price you paid for better
accuracy
• Similar idea applies to more accurate BD formula

19
 Richardson Extrapolation
Idea:
• exact • Use different h to
= computed+ error estimate the truncation
error
• The truncation error is • Use extrapolation to
of the form: chk get more accurate
– where c is some result
constant

20
 Richardson Extrapolation (cont)
• Example: CD for f’(x)
f ( x  h)  f ( x  h)
f ( x)   O(h 2 )
2h

• Using Different h (h1, h2)：

exact  R1  c1h12
 R2  c2 h22

• c1 and c2 could be different

21
 Richardson Extrapolation (cont)
• If c1  c2  c
R1  ch12  R2  ch22  exact
R2  R1
 
c h  h  R2  R1
2 2
c 2
1 2
h1  h22
R2  R1 2 R2  R1
exact  R2  2  h  R   h 2

h1  h2 2  h1 
2 2 2 2 2

h2  2  1
 h2 
R2  R1
 exact  R2 
  1
h1 2
h2
22
 Example
• f(x) = x3. Use CD with Richardson
extrapolation to compute f’(1)
h1  0.1 f (1)  3.01  R1
h2  0.05 f (1)  3.0025  R2

3.0025  3.01 Magic?

exact  3.0025  2
 3.0000 Coincidence?
 0.1 
  1
 0.05 
23
 Revisit CD Formula for f (x)
1 1 1 1
f ( x  h)  f ( x)  f ( x)h  f ( x)h 2  f ( x) h 3  f ( 4 ) ( x)h 4  f ( 5) ( x)h 5    
2 3! 4! 5!
1 1 1 ( 4) 1 (5)
–) f ( x  h )  f ( x )  f ( x ) h  f ( x ) h 2
 f ( x ) h 3
 f ( x ) h 4
 f ( x ) h 5
 
2 3! 4! 5!
2 2
f ( x  h)  f ( x  h)  2hf ( x)  f ( x)h 3  f ( 5) ( x)h 5    
3! 5!
f ( x  h)  f ( x  h) 2 2
 f ( x)   f ( x)h 2  f ( 5) ( x)h 4  
2h 3! 5!

Change notation:
f ( x)  F (h)  a1h 2  a2 h 4  O(h 6 )
24
 Error Analysis
F (h)  f ( x)  a1h 2  a2 h 4  O(h 6 )
2 4 6
h h  h h
F ( )  f ( x)  a~1    a~2    O 
2  2  2  2

Assuming a1  a~1 and a2  a~2

Eliminate a1 to get better accuracy

25
 Error Analysis (cont)
2 4 6
h h  h h
4 F ( )  f ( x)  a1    a2    O 
2  2  2  2
–) F (h)  f ( x)  a1h 2  a2 h 4  O(h 6 )
h  1
4 F ( )  F (h)  3 f ( x)  1  4   a2 h 4  O (h 6 )
2  16 
1 h  1
f ( x)  4 F ( )  F (h)  a2 h 4  O (h 6 )
3 2  4

O(h4)

26
 Revisit Previous Example
• f(x) = x3. Use CD with Richardson
extrapolation to compute f’(1)
• a2 involves f(5)(x), hence, the exact solution
is no surprise.

27
 Remark
• How much effort did we use to get this level
of accuracy?
– F(h): f(x+h), f(x-h); one –, one 
– F(h/2): f(x+h/2), f(x-h/2); one –, one 
– R.E.: two , one –

28
Summary (Textbook p.373)

29
30
 http://earendil.ath.cx/content/academic/numeralysis1.doc

‫אקסטרפולציית ריצ'רדסון לביטול איבר השגיאה המוביל‬

 
F  h0   a0   ai x pi
F  h0   a0   ai x pi pi

i 1 i 1

F  h0    p1 F  h0 
F  h0   
 1 p1

F  h0    F  h0 
p 1 

1 p 1
 a0  
i2
ai p p
x i i

31