Chap 2 Sec 2

Matrix Theory
––Errors
School of Mathematical Sciences
Teaching Group
Textbook:
Ilse C. F. Ipsen, Numerical Matrix Analysis: Linear Systems and Least Squares. SIAM,
2009.
Reference books:
▸ Fuzhen Zhang. Matrix Theory-Basic Results and Techniques, Second Edition.
Springer, 2011.
▸ Roger A. Horn and Charles A. Johnson: Matrix Analysis. Cambridge University
Press, 1985.
▸ Gene H. Golub and Charles F. Van Loan: Matrix Computations, Third Edition.
Johns Hopkins Press, 1996.
▸ Nicholas J. Higham. Accuracy and Stability of Numerical Algorithms, Second
Edition. SIAM, 2002.
▸ Y. Saad. Iterative Methods for Sparse Linear Systems, Second Edition. SIAM,
Philadelphia, 2003.
Matrix Theory Errors
Maintained by Yan-Fei Jing
Absolute and Relative Errors
Outline
Floating Point Arithmetic
Conditioning of Subtraction
Matrix Theory Errors - 2/18

Introduction
There are two techniques for measuring error of an approximation:

▸ absolute error
▸ relative error

Introduction

▸ absolute error
▸ relative error

Introduction

▸ absolute error : gives how large the error is when compared to
the correct value;
▸ relative error

Introduction

the correct value;
▸ relative error : gives how large the error is relative to the
correct value.

Introduction

the correct value;
correct value.
To quantify ill-conditioning, we need to assess the size of errors.

Introduction

the correct value;
correct value.
To quantify ill-conditioning, we need to assess the size of errors.

Example
He has g = 1011 dollars in his bank

Someone has y = 10 dollars in his account.
bank account.

Example

Someone has y = 10 dollars in his account.
bank account.
The bank makes a mistake and subtracts 20 dollars from every account.

Example
Someone has y = 10 dollars in his He has g = 1011 dollars in his bank

bank account. account.
His account has a negative balance of g = 1011 − 20

He still has ̃ dollars.
̃y = −10 dollars.
The account is overdrawn, and
all kinds of bad consequences ensue.
Example
Someone has y = 10 dollars in his He has g = 1011 dollars in his bank

bank account. account.
His account has a negative balance of g = 1011 − 20

̃y = −10 dollars. He may not even notice the error.
The account is overdrawn, and
all kinds of bad consequences ensue–
In debt.
Example

account.
Someone has y = 10 dollars in his
bank account.
g = 1011 − 20
He may not even notice the error.
His account has a negative balance of
̃y = −10 dollars. How can we express mathematically
The account is overdrawn, and that the bank’s error is much worse
all kinds of bad consequences ensue– for Someone than for Bill Gatez?
In debt.
Example analysis
Compare the error to the balance in the account!!!

Example analysis

▸ For Someone:
y −̃y
= 2,
y
▸ For Bill Gatez:
g −̃g
= 2 ⋅ 10−10 ,
g

Example analysis

▸ For Someone:
y −̃y
= 2,
y
showing that the error is twice as large as Someone’s original
balance.
▸ For Bill Gatez:
g −̃g
= 2 ⋅ 10−10 ,
g
so that the error is only a tiny fraction of his balance.

Example analysis

▸ For Someone:
y −̃y
= 2,
y
balance.
▸ For Bill Gatez:
g −̃g
= 2 ⋅ 10−10 ,
g
⇒ It is clear that the bank.s error is much more serious for Someone
than it is for Bill Gatez.

Example analysis
▸ For Someone:
y −̃y
= 2,
y
balance.
▸ For Bill Gatez:
g −̃g
= 2 ⋅ 10−10 ,
g
▸ A difference like y − ̃
y measures an absolute error;

Example analysis
▸ For Someone:
y −̃y
= 2,
y
balance.
▸ For Bill Gatez:
g −̃g
= 2 ⋅ 10−10 ,
g
▸ A difference like y − ̃
y measures an absolute error;
y −̃y y −̃y
▸ A ratio y
or ̃
y
measure a relative error.

Definition

Definition
▸ If the scalar ̃
x is an approximation to the scalar x , then we call
x∣
∣x − ̃
an absolute error.

Definition
x∣
∣x − ̃
an absolute error.
▸ If x ≠ 0, then we call
∣x − ̃x∣
∣x ∣
is a relative error.

Definition
x∣
∣x − ̃
an absolute error.
▸ If x ≠ 0, then we call
∣x − ̃x∣
∣x ∣
is a relative error.
▸ If ̃
x ≠ 0, then
x∣
∣x − ̃
x∣
∣̃
is also a relative error.

Analysis of relative error

A relative error close to or larger than 1 pmeans that an
approximation is totally inaccurate.


▸ Suppose that
∣x − ̃x∣
≥ 1,
∣x ∣
then
x ∣ ≥ ∣x ∣,
∣x − ̃
which means that the absolute error is larger than the
quantity we are trying to compute.


▸ Suppose that
∣x − ̃x∣
≥ 1,
∣x ∣
then
x ∣ ≥ ∣x ∣,
∣x − ̃
which means that the absolute error is larger than the
quantity we are trying to compute.
▸ If x = 0, however the approximation 0 ≠ ̃
x is, the relative error
is always
∣0 − ̃x∣
= 1.
∣x ∣

Significant digits

Significant digits

Significant digits
A relative error can give information about how

many digits two numbers have in common.

Significant digits

As a rule of thumb, if
≤ 5 ⋅ 10−d ,
∣x − ̃x∣
∣x ∣
then we say that the numbers x and ̃
x agree to
d decimal digits.
Significant digits

≤ 5 ⋅ 10−d ,
∣x − ̃x∣
∣x ∣
x agree to
d decimal digits.
Significant digits
▸ If x = 1 and ̃
x = 1.003,
then
= 3⋅10−3 ≤ 5⋅10−3 ,
∣x − ̃x∣
∣x ∣
so that x and ̃
x agree
to three decimal digits.

≤ 5 ⋅ 10−d ,
∣x − ̃x∣
∣x ∣
x agree to
d decimal digits.
Significant digits
▸ If x = 1 and ̃
x = 1.003,
then
= 3⋅10−3 ≤ 5⋅10−3 ,
∣x − ̃x∣
∣x ∣
so that x and ̃
x agree
to three decimal digits.
A relative error can give information about how ▸ The number x = 1 and
x = 0.997 also agree to
̃
three decimal digits
because
= 3⋅10−3 ≤ 5⋅10−3 .
∣x − ̃x∣
≤ 5 ⋅ 10−d ,
∣x − ̃x∣
∣x ∣
∣x ∣
x agree to
d decimal digits.
Outline

Floating point
An early electromechanical
programmable computer, the Z3,
included floating-point arithmetic
(replica on display at Deutsches
Museum in Munich).

Floating point
Museum in Munich).
In computing, floating point: a

method of representing an approxi-
mation of a real number in a way
— that can support a trade-off be-
tween range and precision.

Floating point
▸ A number is, in general,
represented approximately to a
fixed number of significant
digits (the significand) and
scaled using an exponent;
(the base for the scaling is
normally two, ten, or sixteen).
Museum in Munich).

Floating point
An early electromechanical ▸ A number that can be
programmable computer, the Z3, represented exactly is of the

following form:
(replica on display at Deutsches significand × baseexponent ,
Museum in Munich).
where, significand ∈ Z, base ∈
N, exponent ∈ Z.
Floating point
An early electromechanical ▸ A number that can be
programmable computer, the Z3, represented exactly is of the

following form:
(replica on display at Deutsches significand × baseexponent ,
Museum in Munich).
where, significand ∈ Z, base ∈
N, exponent ∈ Z.
method of representing an approxi- For example , 1.2345 =
exponent
mation of a real number in a way ª
−4
— that can support a trade-off be- 12345 × 10 .
tween range and precision. ´¹¹ ¹ ¸¹¹ ¹¶ ¯
significand base
Floating point arithmetic
Many computations in science and engineering are carried out in

floating point arithmetic,
where all real numbers are represented by a finite set of floating
point numbers.


point numbers.
Arithmetic operations on floating point numbers consist of
▸ addition
▸ subtraction
▸ multiplication
▸ division


point numbers.
Arithmetic operations on floating point numbers consist of
▸ addition
▸ subtraction
▸ multiplication
▸ division
▸ All floating point numbers are stored in the same, fixed

number of bits regardless of how small or how large they are.
▸ Many computers are based on IEEE double precision
arithmetic where a floating point number is stored in 64 bits.
Insight
The floating point representation ̂

x of a real number x differs from
x by a factor close to one, and satisfies 1
x = x (1 + x ), where ∣x ∣ ≤ u,
̂
where, u is the “unit roundoff” that specifies the accuracy of

floating point arithmetic.
1
We assume that x lies in the range of normalized floating point numbers,
so that no underflow or overflow occurs.
Insight
The floating point representation ̂

x of a real number x differs from
x by a factor close to one, and satisfies 1
x = x (1 + x ), where ∣x ∣ ≤ u,
̂
where, u is the “unit roundoff” that specifies the accuracy of

floating point arithmetic.
▸ In IEEE double precision arithmetic
u = 2−53 ≈ 10−16 .
1
We assume that x lies in the range of normalized floating point numbers,
so that no underflow or overflow occurs.
Relative perturbation
If x ≠ 0, then
∣x − ̂x∣
= ∣x ∣ ≤ u.
∣x ∣

If x ≠ 0, then
∣x − ̂x∣
= ∣x ∣ ≤ u.
∣x ∣
⇒ Conversion to floating point representation causes relative

errors.

If x ≠ 0, then
∣x − ̂x∣
= ∣x ∣ ≤ u.
∣x ∣
⇒ Conversion to floating point representation causes relative

errors.
▸ A floating point number ̂
x is a relative perturbation of the
exact number x .

Remarks

Remarks
▸ Since floating point arithmetic causes relative perturbations in

the inputs, it makes sense to determine relative—rather than
absolute—errors in the output.

Remarks

▸ As a consequence, we will pay more attention to relative
errors than to absolute errors.

Remarks

▸ As a consequence, we will pay more attention to relative
errors than to absolute errors.
Question: how elementary arithmetic operations are affected
when they are performed on numbers contaminated
with small relative perturbations,
such as floating point numbers.

Outline

Subtraction is the only elementary operation that is sensitive to

relative perturbations.

Example
Analogy of Captain Jack and his

battleship

Example

battleship
To find out how much he weighs,

▸ Jack first weighs the battleship
with himself on it;

Example

battleship

with himself on it;
▸ and then he steps off the
battleship and weighs it
without himself on it;

Example

battleship

with himself on it;
▸ at the end he subtracts the
two weights.

Example

battleship

with himself on it;
two weights.

Example
▸ Vague feeling: this should not

give an accurate estimate for
the captain’s weight.
battleship

with himself on it;
two weights.

Example

battleship x ∶ the weight of the battleship
▸ ̃
plus Jack
with himself on it;
two weights.

Example

battleship x ∶ the weight of the battleship
▸ ̃
plus Jack
▸ Jack first weighs the battleship y ∶ the weight of the battleship
▸ ̃
with himself on it; without Jack

two weights.

Example

x ∶ the weight of the battleship

▸ ̃
Analogy of Captain Jack and his plus Jack —̃
x = 1122339
battleship
y ∶ the weight of the battleship
▸ ̃
To find out how much he weighs, without Jack —̃
y = 1122337
with himself on it;
battleship and weighs it Due to the limited precision of the
without himself on it; scale, the underlined digits are uncer-
▸ at the end he subtracts the tain and may be in error.
two weights.

Example

x ∶ the weight of the battleship

▸ ̃
Analogy of Captain Jack and his plus Jack —̃
x = 1122339
battleship
y ∶ the weight of the battleship
▸ ̃
To find out how much he weighs, without Jack —̃
y = 1122337
▸ Jack first weighs the battleship ▸ Jack computes as his weight
with himself on it; x −̃
̃ y = 2.
battleship and weighs it Due to the limited precision of the
without himself on it; scale, the underlined digits are uncer-
▸ at the end he subtracts the tain and may be in error.
two weights.

Catastrophic cancellation
x −̃
̃ y =2
This difference is totally inaccurate — because it is derived from
uncertainties, while all the accurate digits have cancelled out.

x −̃
̃ y =2
This is an example of “catastrophic cancellation”.

x −̃
̃ y =2
Catastrophic cancellation occurs

x −̃
̃ y =2

▸ when we subtract two numbers that are
uncertain,

x −̃
̃ y =2

▸ when we subtract two numbers that are
uncertain,
▸ and when the difference between these two
numbers is as small as the uncertainties.

THANKS FOR YOUR ATTENTION.
QUESTIONS ?

Chap 2 Sec 2

Uploaded by

Copyright:

Available Formats

You might also like

Chap 2 Sec 2

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chap 2 Sec 2

Uploaded by

Copyright:

Available Formats

Matrix Theory

Absolute and Relative Errors

Floating Point Arithmetic

Matrix Theory Errors - 2/18

There are two techniques for measuring error of an approximation:

Matrix Theory Errors - 3/18

There are two techniques for measuring error of an approximation:

Matrix Theory Errors - 3/18

There are two techniques for measuring error of an approximation:

Matrix Theory Errors - 3/18

There are two techniques for measuring error of an approximation:

Matrix Theory Errors - 3/18

There are two techniques for measuring error of an approximation:

To quantify ill-conditioning, we need to assess the size of errors.

Matrix Theory Errors - 3/18

There are two techniques for measuring error of an approximation:

To quantify ill-conditioning, we need to assess the size of errors.

Matrix Theory Errors - 3/18

He has g = 1011 dollars in his bank

Matrix Theory Errors - 4/18

He has g = 1011 dollars in his bank

Matrix Theory Errors - 4/18

Someone has y = 10 dollars in his He has g = 1011 dollars in his bank

His account has a negative balance of g = 1011 − 20

Someone has y = 10 dollars in his He has g = 1011 dollars in his bank

His account has a negative balance of g = 1011 − 20

He has g = 1011 dollars in his bank

Compare the error to the balance in the account!!!

Matrix Theory Errors - 5/18

Compare the error to the balance in the account!!!

Matrix Theory Errors - 5/18

Compare the error to the balance in the account!!!

Matrix Theory Errors - 5/18

Compare the error to the balance in the account!!!

Matrix Theory Errors - 5/18

Matrix Theory Errors - 5/18

Matrix Theory Errors - 5/18

Matrix Theory Errors - 6/18

Matrix Theory Errors - 6/18

Matrix Theory Errors - 6/18

Matrix Theory Errors - 6/18

Analysis of relative error

Matrix Theory Errors - 7/18

Analysis of relative error

Matrix Theory Errors - 7/18

Analysis of relative error

Matrix Theory Errors - 7/18

Matrix Theory Errors - 8/18

Matrix Theory Errors - 8/18

A relative error can give information about how

Matrix Theory Errors - 8/18

A relative error can give information about how

A relative error can give information about how

A relative error can give information about how

Absolute and Relative Errors

Floating Point Arithmetic

Matrix Theory Errors - 9/18

Matrix Theory Errors - 10/18