MA 572 Numerical Analysis

Lecture 3
(August 5, 2021)

MA572 1 / 11
Propagation of Errors

Error in summation
Many numerical methods, especially in linear algebra, involve summations. Consider the
computation of the sum
Xm
S= xj (1)
j=1

with x1 , x2 , . . . , xm being floating-point numbers.

Define
S2 = fl(x1 + x2 ) = (x1 + x2 )(1 + 2 ) (2)

making use of
xA ω̂yA = fl(xA ωyA )

and
fl(x) = (1 + )x.

Using recurrence relation,

Sr+1 = fl(Sr + xr+1 ), r = 2, . . . , m − 1.

MA572 2 / 11
Propagation of Errors

Then
Sr+1 = (Sr + xr+1 )(1 + r+1 ). (3)

The quantities 2 , . . . , m satisfy

−β −t+1 ≤  ≤ 0, chopped fl(x)

or
1 1
− β −t+1 ≤  ≤ β −t+1 , rounded fl(x)
2 2
depending on whether chopping or rounding is used.

MA572 3 / 11
Propagation of Errors

Let us expand the first few terms in (3)

S2 − (x1 + x2 ) = 2 (x1 + x2 )
S3 − (x1 + x2 + x3 ) = (x1 + x2 )2 + (x1 + x2 )(1 + 2 )3 + x3 3
u (x1 + x2 )2 + (x1 + x2 + x3 )3
S4 − (x1 + x2 + x3 + x4 ) = ((x1 + x2 )2 + (x1 + x2 )(1 + 2 )3 + x3 3 )(1 + 4 )
u (x1 + x2 )2 + (x1 + x2 + x3 )3 + (x1 + x2 + x3 + x4 )4 .

In above, we have neglected the product terms i j since they are of smaller magnitudes.

By induction
X
m
Sm − xj u (x1 + x2 )2 + · · · + (x1 + x2 + · · · + xm )m
i =1
= x1 (2 + 3 + · · · + m ) + x2 (2 + 3 + · · · + m )
+x3 (3 + · · · + m ) + · · · + xm m . (4)

MA572 4 / 11
Propagation of Errors

By induction
From (4), it can be concluded that the best strategy for addition is to add from the smallest to
the largest to have the minimum error.

MA572 5 / 11
Stability in Numerical Analysis

A good number of mathematical problems have solutions which are quite sensitive to small
computational error such as rounding errors. To deal with this, the concept of stability and
condition number are used.

The condition number of a problem is closely related to the maximum accuracy that can be
attained in the solution when using finite length numbers and computer arithmetic.

In general, we wish to use numerical methods that have no greater sensitivity to small errors than
was of the original problem.

Here we limit our discussion to the problems that have the form of an equation

F (x, y ) = 0. (5)

MA572 6 / 11
Stability in Numerical Analysis

Here x is the unknown variable to be evaluated and the variable y is the data on which the
solution x depends.

Equation (5) may represent different kinds of problem such as

(A) F may be a real-valued function of the real variable x, and y may be a vector of coefficients
present in the definition of F ,
(B) The equation may be an integral or a differential equation, with x as an unknown function
and y a given function or given boundary values.

The problem (5) is said to be stable if the solution x depends in a continuous way on the variable
y . This means that if {yn } is a sequence of values approaching y in some sense, the associated
solution values {xn } must also approach x in the same way.

Equivalently, if we make smaller changes in y , it must lead to corresponding smaller changes in x.

Stable problems are also called well-posed problems.

If a problem is not stable, then it is called unstable or ill-posed.

MA572 7 / 11
Stability in Numerical Analysis

For practical purposes, there are many problems that are stable but which still pose a great
difficulty as far as numerical computation are concerned. To deal with this difficulty, we introduce
a measure of stability called a condition number.

The condition number attempts to measure the worst possible effect on the solution of x of (5)
when the variable y is perturbed by a small amount.

Let δy be a perturbation of y and let x + δx be the solution of the perturbed problem

F (x + δx, y + δy ) = 0. (6)

Define
||δx||/||x||
K (x) = sup , (7)
δy ||δy ||/||y ||
where || · || denotes a measure of size.

The supremum in (7) is taken over all small perturbations δy for which the perturbed problem (6)
will still make sense.

MA572 8 / 11
Stability in Numerical Analysis

Problems that are unstable lead to K (x) = ∞.

The number K (x) is called the condition number for (5). It is a measure of the sensitivity of the
solution x to small changes in the data y .

If K (x) is quite large, then there exists small relative changes δy in y that lead to large relative
changes δx in x.

But if K (x) is small, say K (x) ≤ 10, then small relative changes in y always lead to
correspondingly small changes in x.

Since numerical calculation always involve a variety of small computational errors, we do not want
problems with a large condition number.

MA572 9 / 11
Rootfinding for nonlinear equations

Many physical problems lead to algebraic equations which may be linear or nonlinear. Linear
equations may not be difficult to solve but in many cases, solving nonlinear equations require
more efforts and they may not be solved analytically. In that case, approximate solutions may be
sought by using some numerical methods.

One frequently occurring problem in applied mathematics or engineering is finding one or more
roots of an equation of the form
f (x) = 0. (8)

In most cases, explicit solutions may not be available and we must satisfy ourselves in finding a
root to any specified degree of accuracy. The numerical methods for finding the roots are called
iterative methods.

We first take up iterative methods for solving (8) when f (x) is any continuously differentiable
real-valued function of a real variable x.

MA572 10 / 11
Rootfinding for nonlinear equations

The iterative methods for such general class of equations requires knowledge of one or more
initial guesses x0 for the desired root α of f (x). An initial guess can usually be found by using the
context in which the problem first arose. Otherwise, a simple graph of y = f (x) will suffice for
estimating x0 .

Another important problem that will be discussed is to find one or more roots of a polynomial
equation
p(x) ≡ a0 + a1 x + · · · + an x n = 0, an 6= 0. (9)

To illustrate the idea of an iterative method for solving equation (8), let us consider the following
equation:
1
f (x) ≡ a − = 0, (10)
x
for a given a > 0.

MA572 11 / 11