Introduction to Numerical Method and Preliminaries 1/23

Introduction to Numerical Method

Muhammad Irfan
School of Electrical Engineering and Computer Science (SEECS)
National University of Sciences and Technology,
Islamabad, Pakistan

Lecture (1 and 2)

January 31, 2022

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 2/23

Outlines of Lecture

This lecture contains following outlines.

• Introduction to Numerical Method

• Mathematical Preliminaries

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 3/23

Main Objectives

This course is designed to achieve the following objectives.

• The objectives of studying this course is to make the students
familiar with the ways of solving complicated mathematical
problems numerically.
• Describing and understanding of the several errors and
approximation in numerical methods.
• The understanding of several available solutions of equations in
one variable.
• The explaining and understanding of the several available
methods to solve the simultaneous equations.
• The studying of Curve Fitting and Interpolation.
• To find derivative and integration numerically.
• To find the solution of 1st and 2nd order ODEs numerically.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 4/23
Preliminaries

Numerical Method
Definition
It is the branch of mathematics which deals with the solution of
mathematical problem, having initial information/ conditions in
numerical form and results are required in numerical form. The
technique applied to achieve the results are called numerical
methods.

This course has some distinct features. It can give the numerical
solution of the problems.
•Which have no analytical method for their solution.
•The analytical methods exists but are very cumbersome to use.
•The results up to desired accuracy are obtained rapidly.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 5/23
Preliminaries

Error Analysis

Analysis of error involved in calculations is important topic in

numerical method. In any experimental study, how much more
accurate our experimental results would need to be to ensure that
an error should be minimum.

Definition
Error is a term used to denote the amount by which an
approximation fails to equal the exact solution.

Error = Exact solution − Approximation

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 6/23
Preliminaries

Sources of Error

Numerically computed solutions are subject to certain errors.

Mainly there are three types of errors.
1. Inherent errors 2. Truncation errors 3. Round Off errors

Inherent Errors (Experimental):

Errors arise due to assumptions made in the mathematical
modeling of problems. Also arise when the data is obtained from
certain physical measurements of the parameters of the problem
i.e. errors arising from measurements.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 7/23
Preliminaries

Sources of Error (Cont.)

Truncation Errors:
Often an approximation is used in place of an exact mathematical
procedure. For instance consider the taylor series expansion of sin x
is
3 5 7
sin x = x − x3! + x5! − x7! + .....
Practically we cannot use all of the infinite number of terms in the
series for computing the sin x. we usually terminate the process
after the certain number of terms. The error that results due to
such a termination or truncation is called truncation error.
Example:
Expand f (x) = e x
x2 x3 x4 x5
f (x) = e x = 1 + x + + + + .....
2! 3! 4! 5!
Muhammad Irfan Introduction to Numerical Method and Preliminaries
Introduction to Numerical Method and Preliminaries 8/23
Preliminaries

If series is truncated after 3rd term,

x2 x3
ex ≈ 1 + x + +
2! 3!
0.04
e 0.2 ≈ 1 + 0.2 + = 1.22
2
Truncation error will be,
xi x3 x4 x5
Σ∞
i=3 = + + .....
i 3! 4! 5!
put x =0.2
0.008 0.0016
= + = 0.00133 + 0.00066 = 0.133 × 10−2 +
6 24
0.0066 × 10−2

truncation error = 10−2

Muhammad Irfan Introduction to Numerical Method and Preliminaries
Introduction to Numerical Method and Preliminaries 9/23
Preliminaries

Sources of Error (Cont.)

Round Off Errors:

Errors arising from the process of rounding off during
computations. These are also called ’Chopping’ i.e. discarding all
decimals from some decimals on. e.g. (a)the irrational number π
equals approximately 3.14, rounded to two decimal places or three
significant figure.
Chopping :
Rounding a number by chopping amount to dropping the extra
digits. Hence the given number is truncated. Suppose we are using
a calculator with a fixed word length of four digits. Then the
truncated representation of the number 72.32451 will be 72.32 the
digits 452 will be dropped.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 10/23
Preliminaries

Sources of Error (Cont.)

Symmetric Round Off Errors:

In the symmetric round-off method the last retained significant
digit is rounded up by 1 if the first discarded digit is greater or
equal to 5.
e.g. x= 72.918671 , y = 18.63421 be two given numbers to be
rounded off to five digit numbers.
x = 72.919 , y =18.634

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 11/23
Preliminaries

Relative Error

Definition
If a is an approximate value of a quantity whose exact value is a
then relative error r of a is defined by

|a−a| ||
| r |= =
| true value | |a|

Example:√
Consider
√ 2 = 1.414213.... upto four decimal places then
2 = 1.4142 + errors
| error | =| 1.4142 − 1.41421 | = 0.00001, taking 1.4142 as true
value or exact value
Hence, r = 0.00001
1.4142
Muhammad Irfan Introduction to Numerical Method and Preliminaries
Introduction to Numerical Method and Preliminaries 12/23
Preliminaries

Absolute Error

Definition
If a is an approximate value of a quantity whose exact value is ’a’
then the difference =a − a is called absolute error of ’a’.

Example: If a = 10.52 is an approximation to a = 10.5 then the

error is  = 0.02.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 13/23
Preliminaries

Error Bound

Definition
It is a number β for a such that | a − a |≤ β or |  |≤ β.

Example:
√ If β = 10−4 and a = 1.4142 is an approximation to
a = 2 = 1.4142135... then the error is
 =| a − a |= 0.0000135 ≤ β.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 14/23
Preliminaries

Error
Definition
Local Error:
The Local error is the error introduced during one operation of the
iterative process. (This is the error after first step.)

1 = x(to ) − x

Definition
Local Error:
Global error measures the difference between the computed
solution at a given time and the true solution of the initial value
problem that we are trying to solve.

n = x(tn ) − x
Muhammad Irfan Introduction to Numerical Method and Preliminaries
Introduction to Numerical Method and Preliminaries 15/23
Preliminaries

Accuracy and Precision

Definition
Accuracy:
Accuracy means how close are our approximations from exact
value.

Definition
Precision:
Precision mean how close are the measurements obtained from
successive iterations.
or
Precision is how consistent results are when measurements are
repeated. Precise values differ from each other because of random
error, which is a form of observational error.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 16/23
Preliminaries

Accuracy and Precision (Con)

Definition
Example:
If you take measurements of the mass of a 50.0 gram standard
sample and get values of 47.5, 47.6, 47.5, and 47.7 grams, your
scale is precise, but not very accurate. The average of your
measurements is 47.6, which is lower than the true value. Yet,
your measurements were consistent. If your scale gives you values
of 49.8, 50.5, 51.0,and 49.6, it is more accurate than the first
balance but not as precise. The average of the measurements is
50.2, but there is a much larger range between them. The more
precise scale would be better to use in the lab, providing you made
an adjustment for its error. In other words, it’s better to calibrate
a precise instrument than to use an imprecise, yet accurate one.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 17/23
Preliminaries

Numerical Algorithm
Definition
A complete set of procedures which gives an approximate solution
to a mathematical problem.

Definition
Criteria for a good method
1) Number of computations i.e. Addition, Subtraction,
Multiplication and Division.
2) Applicable to a class of problems.
3) Speed of convergence.
4) Error management.
5) Stability.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 18/23
Preliminaries

Numerical Algorithm (Cont.)

Definition
Stable Algorithm:
Algorithm for which the cumulative effect of errors is limited, so
that a useful result is generated is called stable algorithm.
Otherwise Unstable.

Definition
Numerical Stability:
Numerical stability is about how a numerical scheme propogate
error.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 19/23
Preliminaries

Numerical Algorithm (Cont.)

Definition
Numerical Iteration Method:
A mathematical procedure that generates a sequence of improving
approximate solution for a class of problems i.e. the process of
finding successive approximations.

Definition
Algorithm of Iteration Method:
A specific way of implementation of an iteration method, including
to termination criteria is called algorithm of an iteration method.
In the problem of finding the solution of an equation, an iteration
method uses as initial guess to generate successive approximation
to the solution.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 20/23
Preliminaries

Numerical Algorithm (Cont.)

Definition
Convergence Criteria for a Numerical Computation:
If the method leads to the value close to the exact solution, then
we say that the method is convergent otherwise the method is
divergent.i.e. limn→∞ xn = r

Definition
Why we use numerical iterative methods for solving
equations?
As analytic solutions are often either too tiresome or simply do not
exist, we need to find an approximate method of solution. This is
where numerical analysis comes into picture.

Muhammad Irfan Introduction to Numerical Method and Preliminaries

Introduction to Numerical Method and Preliminaries 21/23
Preliminaries

Convergence of Iterative Method

Definition
Local Convergence:
An iterative method is called locally convergent to a root, if the
method converges to root for initial guess sufficiently close to root.

Definition
Rate of Convergence of an Iterative Method:
Suppose that the sequence (xk ) converges to ’r’ then the sequence
is said to converge to ’r’ with order of convergence ’a’ if there exist
a positive constant ’p’ such that
limk→∞ |x|xk+1 −r |
k −r |
k+1
α = limk→∞ α = p
k
Thus if α = 1, the convergence is linear. If α = 2, the convergence
is quadratic and so on.Where the number a is called convergence
factor.
Muhammad Irfan Introduction to Numerical Method and Preliminaries
Introduction to Numerical Method and Preliminaries 22/23
Preliminaries

Convergence of Numerical Method

Definition
CONDITION OF A NUMERICAL PROBLEM :
A problem is well conditioned if small change in the input
information causes small change in the output. Otherwise it is ill
conditioned.

Example:
2x + y = 4, 2x + 1.01y = 4.02 has the exact solution
x = 1, y = 2
If we do small change in the equation 2x + y = 3.8,
2.02x + y = 4.02
It is easily verified that the exact solution of this system is
x = 11, y = −18.2.
This solution differs greatly from the solution of the first system.
Both these systems are said to be ill-conditioned.
Muhammad Irfan Introduction to Numerical Method and Preliminaries
Introduction to Numerical Method and Preliminaries 23/23
Preliminaries

Application of Numerical Method

•Estimation of ocean currents

•Modeling combustion flow in a coal power plant
•Modeling of airflow over air plane bodies
•Electromagnetic
•Design and analysis of control systems for air craft
•Large scale shock wave physics code development
•Network simulation
•Train and traffic signal
•Weather prediction
•Car safety enhancement

Muhammad Irfan Introduction to Numerical Method and Preliminaries