Errors

Objectives: Errors may arise from such things as inaccurate measurements of a physical
quantity (not an accurate enough measuring device) and/or the fact that not all
numbers can be accurately represented in a floating point system. We will look
at methods for characterizing and analyzing errors.
Definition If p ∗ is an approximation to p, then the absolute error is given by |p − p ∗ |, and
|p − p ∗ |
the relative error is given by , p ≠ 0. [In many situations we do not
|p |
|p − p ∗ |
know p (it may be what we are trying to find) so we will probably use ,
|p ∗ |
p∗ ≠ 0
The absolute error refers to the number of digits after the decimal point.
The relative error refers to the number of digits regardless of the position of the decimal
point.
e.g. Suppose we approximate π = 3.14159265... by π ∗ = 3.1416.
The absolute error is
|π − π ∗ | = 0.00000735 = 7.35 × 10 −6 .
The relative error is
|π − π ∗ |
= 0.00000735 = 0.0000023 = 2.34 × 10 −6
|π | 3.14159265

e.g. Suppose we approximate π = 3141.59265... by π ∗ = 3141.6.

The absolute error is
|π − π ∗ | = 0.00735 = 7.35 × 10 −3 .
The relative error is
|π − π ∗ |
= 0.00735 = 0.0000023 = 2.34 × 10 −6
|π | 3141.59265

A) Significant Digits
(Definitions and examples as found in various texts.)
a) Gerald & Wheatley, Applied Numerical Analysis:
Another term that is commonly used to express accuracy is significant digits. that is, how
many digits in the number have meaning. ...
A more formal definition of significant digits follows.
1. True value = d 1 d 2 ...d n d n+1 ...d p , and
2. Approximate value = d 1 d 2 ...d n e n+1 ...e p
where d 1 ≠ 0 and with the first difference in the digits occurring at then + 1st digit. We
then say that (1) and (2) agree to n significant digits if |d n+1 − e n+1 | < 5. Otherwise, we
say they agree to n − 1 significant digits.
Example 0.7 Let the true value = 10/3 and the approximate value = 3.333.
Then the absolute error = 0.000333... = 1/3000;
the relative error = 1/3000/10/3 = 1/10000;
the number of significant digits = 4.
Comment: It is difficult to understand the significance of this example. They do not appear
to make any use of the definition.
b) Atkinson, An Introduction to Numerical Analysis:
In place of relative error, we often use the concept of significant digits. We say x A has m
significant digits, with respect to x T if the error x T − x A has magnitude less than or equal
to 5 in the m + 1st digit of x T , counting to the right from the first nonzero digit in x T .

Examples
(a) x T = 1 x A = 0.333 |x T − x A | = 0.00033
3
Since the error is less than 5 in the fourth digit to the right of the first nonzero digit in x T ,
we say that x A has three significant digits with respect to x T .
(b) x T = 23.496 x A = 23.494 |x T − x A | = 0.002
x A has four significant digits with respect to x T , since the error is less than five in the fifth
place to the right of the first nonzero digit in x T . Note that if x A is rounded to four places,
an additional error is introduced and x A will no longer have four significant digits.
(c) x T = 0.02138 x A = 0.02144 |x T − x A | = 0.00006
The number x A has two significant digits, but not three, with respect to x T .
c) Burden and Faires, Numerical Analysis:
The number p ∗ is said to approximate p to t significant digits (or figures) if t is the largest
nonnegative integer for which
|p − p ∗ |
< 5 × 10 −t
|p |
The reason for using relative error in the definition is to obtain a continuous concept.
Consider the numbers 1000, 5000, 9990, and 10, 000. In order that p ∗ approximate 1000
to four significant figures by this definition, p ∗ would have to satisfy
|p ∗ − 1000 |
< 5 × 10 −4
|1000 |
This requires that 999.5 < p ∗ < 1000.5 and agrees with the intuitive definition of
significant digits (that is, do p and p ∗ round to the same value for those digits).
d) Mathews, Numerical Methods Using Matlab:
Definition 1.8. The number p ∗ is said to approximate p to d significant digits if d is the
largest positive integer for which
|p − p ∗ | −d
< 10 = 0.5 × 10 1−d (mistake in earlier edition - was 0.5 × 10 −d )
|p | 2

Errors 2
Example 1.15
Determine the number of significant digits for the approximations in Example 1.14.
|x − x ∗ | −2
• If x = 3.141592 and x ∗ = 3.14, then = 0.000507 < 10 . Therefore, x ∗
|x | 2
approximates x to two significant digits. (Definition corrected - example not.)
|y − y ∗ | −5
• If y = 1, 000, 000 and y ∗ = 999, 996, then = 0.000004 < 10 . Therefore, y ∗
|y | 2
approximates y to six significant digits. (Corrected.)
|z − z ∗ | −0
• If z = 0.000012 and z ∗ = 0.000009, then < 10 . Therefore, z ∗ approximates
|z | 2
z to one significant digit. (Corrected.)
Comment: The latter two are the same and are what we will use.
Using the previously stated methods in the other texts, we have
Gerald & Wheatley

True = 1000.00 = d 1 d 2 d 3 d 4 .d 5
Approx = 999.51 = e 1 e 2 e 3 e 4 .e 5
The first digit that differs is the first one, so |d 1 − e 1 | = |1 − 0| = 1 < 5. Because n = 0, we
have 0 significant digits -
or
True = 1000.00 = d 1 d 2 d 3 d 4 .d 5 d 6
Approx = 1000.49 = d 1 d 2 d 3 d 4 .e 5 e 6
The first digit that differs is the fifth one, so |d 5 − e 5 | = |0 − 4| = 4 which is less than 5 so
with n = 4, we have 4 significant digits.
Atkinson
x T = 1000.0 and x A = 999.51 so that |x T − x A | = 0.49 . Because this is less than or equal to
5, and m + 1 = 5, we have 4 significant digits -
or
x T = 1000.0 and x A = 1000.49 so that |x T − x A | = 0.49 . As above.
Mathews
|p − p ∗ | |1000 − 999.51 |
p = 1000 and p ∗ = 999.51 so = = 0.49 = 0.49 × 10 −3
|p | 1000 1000
Because this is < 0.5 × 10 1−4 we have 4 significant digits.

|p − p ∗ | |1000 − 1000.49 |
p = 1000 and p ∗ = 1000.49 so = = 0.49
|p | 1000 1000
As above.
|π − π ∗ |
In our absolute/relative error example = 2.34 × 10 −6 so that we have 6 significant
|π |

Errors 3
digits (in theory). Because we have only 5 digits in the approximation, we can have only 5
significant digits. The 6 significant digit approximation would be 3.14160 with the 0 being
significant.
We might be inclined to say that we have 4 decimal place accuracy. This is not, in general, a
useful concept.
If we approximate 100π by 314.16 we have
|100π − 100π ∗ | = 0.000735
so we seem to have 2 fewer decimal places even though
|100π − 100π ∗ | |π − π ∗ |
=
|100π | |π |
i.e. the same number of significant digits.

B) Sources of Error
1. Blunders;
2. Rounding error (due to finite precision arithmetic);
3. Truncation error (polynomial instead of infinite series, Riemann sums etc.)

It should never be assumed that having a computer makes knowledge of analysis unnecessary
- if anything it is more important.
For example, we know that ∑ 1 n diverges, but if, you try to sum it on a computer, it will
probably appear to converge. i.e. we can not depend on the fact that the computer gives an
answer as a proof.

C) Propagation of Error
Objectives: to consider how the error of a result changes as we carry out arithmetic
operations.
Consider x and y. These are exact values but are represented by approximations x ∗ and y ∗ , so
that x = x ∗ + e x and y = y ∗ + e y .
i) Sum/Difference
x ∗ ± y ∗ = x − e x  ± y − e y  = x ± y − e x ± e y .
100 + 0.5 + 200 + 0.5 = 300 + 1
Worst case - absolute error
|e x ± e y | ≤ |e x | + |e y |.
- relative error
|e x ± e y | |x | |e x | |y | |e y |
≤ +
|x ± y | |x ± y | |x | |x ± y | |y |
The important thing to note about the relative error is that if |x ± y | is small then the relative
error can become quite large. This is one of the most dangerous situations - subtractive

Errors 4
cancellation. We will see an example of this in the data fitting section.
ii) Product
x ∗ y ∗ = x − e x  y − e y  = x y − x e y − y e x + e x e y
We neglect the second order term e x e y so that
x y − x∗ y∗ = x ey + y ex
The absolute error is |xe y + ye x | ≤ |xe y | + |ye x |. This may be large if x or y is large.
e.g. 1000000 + 0.51000000 + 0.5 = 10 12 + 2000000 × 0.5 + 0.25
The absolute error is 1000000.
|xe y + ye x | |e | |e |
The relative error is ≤ y + x . For large values of x and/or y this will be
|xy | |y | |x |
small (even though the absolute error is large).
In the above, the relative error is 10 −6 which is fairly small.
For small values of x and/or y the error can become large.
iii) Quotient
ex
x ∗ = x − e x  = x 1 − x
y∗ y − e y  e
y 1− y
y
e e
= x 1 − exx 1 + y + ... = x 1 + y − exx + ...
y y y y
xe e
= x + 2 − x y
y y y
The error is bounded by
|x ||e y | |e |
2
+ x .
y |y |
Although we may hope for an average error, we must allow for the possibility of worst case
behaviour.

As we continue to perform more and more operations (+, −, ∗, /) we can expect that the
accuracy will decrease.

If it turns out that our algorithm suffers severely from roundoff error, we should look for a
different algorithm.

Example Suppose we wish to estimate the error in evaluating

fx, y, z = x 2 + y/z
with x = 1.23 + e x , y = 2.34 + e y , z = 3.45 + e z . We assume that
e x , e y , e z < 0.005. i.e. they are rounded values. The error in multiplying to get
x 2 is
xe x + xe x = 2 ∗ 1.23 ∗ 0.005 = 0.0123.
The error in dividing to get y/z is

Errors 5
 |y ||e z | |e |
2
+ y = 0.005 2.342 + 1 = 0.00243 .
z |z | 3.45 3.45
The total error would appear to be
< 0.0123 + 0.00243 = 0.0147 3.
While this is the theoretical error based on accurate arithmetic, the true error
may affected by floating point considerations.

Consider the way in which 0.1 is represented in the ,

0.1 = 14 + 15 + 18 + 19 + ⋯
2 2 2 2
This will not be exactly 0.1 so additional error is introduced.
If you test for 3.0*(1.0/3.0)==1 you could get a false.
If the number of digits in a result is greater than the precision of the floating point
representation then further round-off errors will occur.

Example Use 3 digit rounding arithmetic to calculate 133 − 0.499. Compute the relative and
absolute errors.
Exact result is 132.501. The approximate result is 133.
Absolute error |132.501 − 133| = 0.499.
Relative error 0.499 = 0.003766
132.501

D) Stability

Another concern is what happens for small changes in input data. If the output undergoes
only small changes for small changes in the input data, the algorithm is stable, otherwise it is
unstable.

Definition Suppose that E n represents the growth of an error  after n subsequent

operations. If |E n | ≈ Cn, where C is a constant independent of n, the growth of
the error is said to be linear (stable).
If |E n | ≈ k n  for some k > 1, the growth of error is called exponential
(unstable).

Errors 6