Numerical Analysis (MATH-254)

Fall 2021
Lecture No. 2 (Basic Concepts of Numerical methods and Analysis)

Focus

SU
- Numerical Analysis vs Numerical Methods

- Some Fundamental/Basic notions and concepts of Numerical Methods

K
a. Numerical Methods

A numerical method is an algorithm that takes numbers as input and pro-

duces numbers as output. For instance, Quadratic formula, Matrix Multipli-
cation, Cramer’s rule, etc.
M
Numerical methods refer to “methods” (like algorithms) that can be used
to solve certain mathematical problems (like ODE’s or PDE’s, integrals and
functions evaluation, solving algebraic
R ∞ and transcendental equations) in a
ah
−x2
“numerical” fashion. e.g., x = cos x, 0 e dx, etc.

b. Numerical Analysis
Sh

Numerical analysis is a set of techniques used to prove that a numerical

method approximately solves a given problem.

Numerical analysis refer to using the developed methods to “analyze” a par-

ticular physical problem of interest.
r.

Numerical analysis is concerned with all aspects of the numerical solution of

D

a problem, from the theoretical development and understanding of numerical

methods to their practical implementation as reliable and efficient computer
programs.

Concluding Remark: A numerical method is the actual procedure we im-

plement to solve a problem. For example, finite difference or finite element

1
 methods for solving PDEs. As the name suggests, numerical analysis ana-
lyzes these methods and tells us how accurate they are. Obviously it’s a little
more complicated, but that’s the basic gist. I doubt we will have a course
that looks at one and not the other though, so we wouldn’t really stress the
difference at this point.

c. Important Features or Aspects of Numerical Algorithms/Methods:

SU
1. Stability: Numerical analysts are concerned with stability, a concept
referring to the sensitivity of the solution of a problem to small changes in
the data or the parameters of the problem. Consider the following example.
The polynomial

K
p(x) = x7 − 28x6 + 322x5 − 1960x4 + 6769x3 − 13132x2 + 13068x − 5040

The roots of this polynomial are 1, 2, 3, 4, 5, 6, and 7. When the coeffi-

M
cient -28 is changed to -28.002 then two roots become imaginary. Such a
polynomial p(x) is called unstable or ill-conditioned with respect to the root-
finding problem. Ideally, numerical methods for solving problems should not
be sensitive to changes in the data. Moreover, the formulation of the original
problem should be stable or well-conditioned.
ah

2. Efficiency or Cost: Numerical analysts are generally interested in mea-

suring the efficiency (or “cost”) of an algorithm. For example, the use of
Sh

Gaussian elimination to solve a linear system Ax = b containing n equations

3
in n unknowns having a unique solution will require approximately 2n3 arith-
metic operations. Numerical analysts would want to know how this method
compares with other methods for solving this problem (e.g., Cramer’s rule,
Gauss-Jordan elimination method (requires approximately n3 operations),
etc.).
r.

3. Accuracy It refers to the absolute or relative error of an approximate

D

value of a quantity with its exact value.

4. Reliability It measures how often (as a percentage) the software (method

or algorithm) fails, in the sense that the true error is larger than what is re-
quested/desired.

d. Precision Precision refers to the number of digits used for arithmetic,

2
 input, and output. In mathematics, numbers have infinite precision, but
numerals (representations of number) have finite precision. One third ( 13 ) can
not be represented precisely in decimals or it would require an infinite number
of digits, which is impossible. Finite precision is decimal representation of
a number which has been rounded or truncated. Finite precision arithmetic
is important in the field of computing because computers can only work with
(operate) numbers of fixed sizes (finite lengths).

SU
e. Applications of Numerical Methods/Analysis:

(1) Numerical analysis and mathematical modeling are essential in many ar-
eas of modern life. Sophisticated numerical analysis softwares are commonly

K
embedded in popular software packages (e.g., spreadsheet programs) and al-
lows fairly detailed models to be evaluated, even when the user is unaware
of the underlying mathematics. Attaining this level of user transparency re-
quires reliable, efficient, and accurate numerical analysis softwares, and it
M
requires problem-solving environments (PSE) in which it is relatively easy
to model a given situation. PSEs are usually based on excellent theoreti-
cal mathematical models, made available to the user through a convenient
graphical user interface. Best known of these PSEs is MATLAB, a commercial
ah
package that is arguably the most popular way to do numerical computing.
Two popular computer programs for handling algebraic-analytic mathemat-
ics (manipulating and displaying formulas) are Maple and Mathematica.
Sh

(2) Computer-aided engineering (CAE) is an important subject within en-

gineering, and some quite sophisticated PSEs have been developed for this
field. A wide variety of numerical analysis techniques is involved in solving
such mathematical models. The models follow the basic Newtonian laws of
mechanics, but there is a variety of possible specific models, and research
r.

continues on their design. One important CAE topic is that of modeling

the dynamics of moving mechanical systems, a technique that involves both
ordinary differential equations and algebraic equations (generally nonlinear).
D

The numerical analysis of these mixed systems, called Differential-Algebraic

Systems, is quite difficult but necessary in order to model moving mechani-
cal systems. Building simulators for cars, planes, and other vehicles requires
solving differential-algebraic systems in real time.

(3) Another important application is atmospheric modeling. In addition to

3
 improving weather forecasts, such models are crucial for understanding the
possible effects of human activities on the Earth’s climate. Many types of
numerical analysis procedures are used in atmospheric modeling, including
computational fluid mechanics and the numerical solution of differential equa-
tions. Researchers strive to include ever finer detail in atmospheric models,
primarily by incorporating data over smaller and smaller local regions in the
atmosphere and implementing their models on highly parallel supercomput-

SU
ers.

(4) Modern businesses rely on optimization methods to decide how to allo-

cate resources most efficiently. For example, optimization methods are used
for inventory control, scheduling, determining the best location for manufac-

K
turing and storage facilities, and investment strategies.

f. Scientific Notation

M
It is defined as a standardized way to represent any real number b as the
product of a real number and an integer power of 10 as follows.

b = a × 10±n , 1 ≤ a < 10, n ∈ N

ah
One purpose of scientific notation is for scientists to write very large, or very
small, numbers with ease.
Sh

Examples:

– Avogadro Number or number of atoms (ions, molecules, or electrons) in

one Mole of a substance = 6.02214076 × 1023 (9 s.d.)

– Mass of earth = 5.9763 × 1024 kg (5 s.d.)

r.

– Mass of an electron = 9.1093837 × 10−31 Kg (8 s.d.)

D

– Distance between the moon and the earth = 3.844 × 105 Km (4 s.d.)

g. Significant Digits Significant figures are the number of digits in a value,

often a measurement, that contribute to the degree of accuracy of the value.
We start counting significant figures from the first non-zero digit in the num-
ber starting from the extreme left. The importance of significant figures is for

4
 one person to tell another person both the measurement of something AND
the level of accuracy of that measurement.

There are 3 rules for significant digits in a number or measurement.

(1) The nonzero digits (1 to 9) are always significant.

SU
(2) Any zeros between two non-zero significant digits are significant.

(3) Leading zeros are NEVER significant and trailing zeros are significant
ONLY when appear in decimal form.

K
Alternatively, use the scientific notation representation of a given number. It
is the easiest way to determine the number of S.D. in a measurement. The
term significant figures refers to the number of important single digits (0
through 9 inclusive) in the coefficient of an expression in scientific notation.
M
The number of significant figures in an expression indicates the confidence or
precision with which an engineer or scientist states a quantity.

Examples: 10079 (5 SD), 0.02084 (4 SD),1000 (1 SD), 1000.00 (6 SD), etc.

ah
h. Major Steps in Numerical Analysis

There are 3 main steps: (1) Problem formulation, (2) Development of Nu-
Sh

merical Method/Algorithm, and (3) Programming (flow chart and coding in

an algorithmic language/environment)

Next Lecture: Types and Sources of Errors, Graphical Method for Locat-
ing Roots of Nonlinear Equations
r.
D

5
 Numerical Analysis (MATH-254)
Spring 2021
Lecture No. 3 (Types and Sources of Errors, Graphical Method for
Locating Roots of Nonlinear Equations)

Focus

SU
- Types and sources of errors

- Root Location Methods for Nonlinear Equations: M-I. Graphical Method

K
Errors in Numerical Methods: Errors, their size, and their analytic forms
are a fundamental concern in numerical analysis. When approximating the
solution of a problem, it is prudent to understand the nature of the error in
M
the computed solution. Moreover, understanding the form of the error allows
creation of extrapolation processes to improve the convergence behavior of
the numerical method.

Definition (Error of Approximation): Suppose an approximation of an

ah
exact value α is p, then the error of approximation e is defined as e = α − p.
If α > p then the error e is positive and if α < p then the error e is negative.
Sh

Definition (Absolute Error): In many situations the sign of the error is

not known and might be even irrelevant. Therefore, we define absolute error
or magnitude of error as |e| = |α − p|

Definition (Relative Error): The relative error (RE) is defined as

r.

|α − p|
RE = , |α| =
6 0.
|α|
D

1
Explanatory Example: Let α = 3 and p = 0.333, then
1
Absolute Error = |α − p| = × 10−3
3
and
|α − p|
Relative Error = = 10−3
α
6
 Note: Generally, RE is a better measure of the extent of error than the ab-
solute error. However, it is not defined when the actual/exact value is zero.

Absolute vs Relative Error: The absolute error tells us how much exactly
the measured value differs from the actual value, or it is the difference b/w
what we have measured and what we should have measured which is not a
very useful information in practical applications. On the other hand, the RE

SU
tells us whether the error is big or small, compared to the actual/real value
which is more useful information. For instance, two students measure two
objects with a meter stick. One student measures the height of a room and
gets a value of 3.125 meter with ±0.001 meter (±1 mm) as the range of uncer-
tainty (error) in the measurement. The other student measures the height of

K
a small cylinder and measures 0.075 meter with the same uncertainty range.
Obviously, the overall accuracy of the ceiling height is much better than that
of 7.5 cm long cylinder. The comparative accuracy of these measurements
can be determined by looking at their REs.

RE in ceiling height = 0.001

3.125 = 0.032
M
0.001
RE in cylinder height = 0.075 = 1.333
ah
Clearly, the RE in measurement of the ceiling height is considerably smaller
than that in the measurement of cylinder length/height even though the
amount of AE is the same in each case.
Sh

Definition: Percentage (% age) Error The percentage error (PE) is

defined as
|α − p|
PE = × 100%, |α| =
6 0.
|α|
r.

Sources of Errors The common sources of errors are: Human Errors, Ma-
chine/Apparatus errors, Truncation/Chopping Errors, and Round-off Errors
D

Numerical Solutions of Nonlinear Equations in ONE Variable: In

the first major topic of this course, we will find the approximate solutions of
the following types of nonlinear equations in ONE variable.

Nonlinear Polynomial Algebraic Equations: A nonlinear algebraic equa-

7
 tion (a polynomial equation of degree n) is defined as:
an xn + an−1 xn−1 + · · · + a1 x + a0 = 0, an 6= 0; n > 1,
where an , an−1 , · · · a1 , and a0 are constants. For example, the following equa-
tions are nonlinear.
x2 + 5x + 6 = 0, x3 = 2x + 1, x200 − 2x + 1 = 0.

SU
Nonlinear Non-Polynomial Algebraic Equations: The nonlinear Al-
gebraic Equations where power of the unknown variable is not a positive
integer number are called non-polynomial. For example, the following non-
polynomial equations are nonlinear.

K
√ 2 2
x−1 + 2x = 1, x + x = 2, x3 + + 4 = 0.
x
Transcendental Equations: An equation which involves trigonometric
M
functions, exponential functions and/or logarithmic functions is called a tran-
scendental equation. For example, all the following transcendental equations.
These equations are always nonlinear.
x = cos x, ex + x − 10 = 0, x + ln x = 10, 2x + tan x − 10 = 0.
ah
Definition (Root of a Nonlinear Equation): Assume that f (x) is a
continuous function. A number α for which f (α) = 0 is called a root of the
equation
Sh

f (x) = 0,
or a zero of the function f (x).

Steps for Finding Root(s) of a Nonlinear Equation: The process of

finding the approximate values of the roots of a nonlinear equation f (x) = 0
r.

can be divided into the following two steps:

D

(S-I) Location of the roots

(S-II) Computation of the values of the roots with the specified degree of
accuracy.

(a) Location of Roots: A root α of f (x) = 0 will be located in the interval

[a, b] if f (α) = 0 for a < α < b. To locate the roots of nonlinear equations,

8
 we can use graphical method and an analytical method called the tabulation
method.

M-I. Graphical Method for Location of Roots of f (x) = 0

– Sketch/Draw the graph of the function y = f (x)

SU
– The abscissas (x-coordinate) of the points where graph intersects/crosses or
touches the x-axis are the required roots of the nonlinear equation f (x) = 0.

– Sometimes, we approximate the root(s) of f (x) = 0 by separating the terms

of the equation into two groups; one of them is written on the left-hand side

K
of the equation and the other on the right-hand side, that is, the given equa-
tion can be represented as f1 (x) = f2 (x). In this case, the abscissas of the
points of intersection of the graphs of y = f1 (x) and y = f2 (x) are the roots
of the nonlinear equation.
M
– The manual sketching the graphs of of the functions y = f (x) is not an
easy task. However, with the help of softwares like MATLAB, Maple, etc.
we can draw the graphs very easily.
ah
Example 1: Given x2 + 4x + 2 = 0. Locate the roots of this equation.
Sh

2
Graph of f(x)=x +4x+2
20

Graph of f(x)=x 2, g(x)=-4x-2

15 25

20
10
y=f(x)

15
y=f(x), y=g(x)

5
r.

10

0 5

0
D

-5
-5 -4 -3 -2 -1 0 1 2 3 -4 -3 -2 -1 0 1 2 3 4 5
x x

(a) Graph of f (x) = x2 + 4x + 2 (b) Graph of f (x) = x2 and g(x) = −4x − 2

Figure 1: Location of Roots by Graphical Method

Example 2: Given x3 − 3x2 + 2 = 0. Locate the roots of this equation.

9
 3 2
Graph of f(x)=x -3x +2

3 2
Graph of f(x)=x and g(x)=3x -2
30
0

25

-2
20
y=f(x)

y=f(x) and y=g(x)

-4 15

10
-6

SU
-8
0

-10
-5
-1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5 3
x x

(a) Graph of f (x) = x3 − 3x2 + 2 (b) Graph of f (x) = x3 and g(x) = 3x2 − 2

Figure 2: Location of Roots by Graphical Method

K
Next Lecture: Root Location Methods for Nonlinear Equations: M-II.
Tabulation Method

M
ah
Sh
r.
D

10