ECE257 Numerical Methods and Scientific Computing

Error Analysis

Todays class:

Introduction to error analysis Approximations Round-Off Errors

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Introduction

Error is the difference between the exact solution and a numerical method solution. In most cases, you dont know what the exact solution is, so how do you calculate the error. Error analysis is the process of predicting what the error will be even if you dont know what the exact solution Errors can also be introduced by the fact that the numerical algorithm has been implemented on a computer
John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

Falling Object Velocity

From Numerical Methods for Engineers, Chapra and Canale, Copyright The McGraw-Hill Companies, Inc.
ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4 John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Significant digits

Can a number be used with confidence? How accurate is the number? How many digits of the number do we trust?

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Significant digits

From Numerical Methods for Engineers, Chapra and Canale, Copyright The McGraw-Hill Companies, Inc.
ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4 John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Significant digits

The significant digits of a number are those that can be used with confidence
The digits that are known with certainty plus one estimated digit

Exact numbers vs. measured numbers

Exact numbers have an infinite number of significant digits is an exact number but usually subject to round-off

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Significant digits
49.0 mph - 3 significant digits

87324.45 miles 7 significant digits

From Numerical Methods for Engineers, Chapra and Canale, Copyright The McGraw-Hill Companies, Inc.
ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4 John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Accuracy and Precision

Accuracy is how close a computed or measured value is to the true value Precision is how close individual computed or measured values agree with each other
Reproducability

Inaccuracy/Bias vs. Imprecision/Uncertainty

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Accuracy and Precision

From Numerical Methods for Engineers, Chapra and Canale, Copyright The McGraw-Hill Companies, Inc.
ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4 John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Accuracy and Precision

The level of accuracy and precision required depend on the problem

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Error Definitions

Two general types of errors

Truncation errors - due to approximations of exact mathematical functions

Et = True value - approximation

Round-off errors - due to limited significant digit representation of exact numbers

Et = True value - representation

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Error Definitions

Et does not capture the order of magnitude of error

1 V error probably doesnt matter if youre measuring line voltage, but it does matter if youre measuring the voltage supply to a VLSI chip.

Therefore, its better to normalize the error relative to the value

true error t = 100% true value
John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

Error definitions

Example:
Line voltage
E t = 120V 119V = 1V

t =

Chip supply voltage

1V 100% = 8.33% 120V

E t = 3.3V 2.3V = 1V 1V t = 100% = 30.3% 3.3V

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

Error definitions

What if we dont know the true value? Use an approximation of the true value
approximate error t = 100% approximate value

How do we calculate the approximate error?

Use an iterative approach
Approximate error = current approximation - previous approximation Assumes that the iteration will converge

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Error Definitions

For most problems, we are interested in keeping the error less than specified error tolerance

a < s

How many iterations do you do, before youre satisfied that the result is correct to at least n significant digits?

s = (0.5 x 10 2n )%
ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4 John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Example

Infinite series expansion of ex

x2 x3 xn e x = 1+ x + + +L+ 2! 3! n!

As we add terms to the expansion, the expression becomes more exact Using this series expansion, can we calculate e0.5 to three significant digits?

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Example

Calculate the error tolerance

s = (0.5 x 10 23 )% = 0.05%

First approximation
e 0.5 = 1

Second approximation
e 0.5 = 1+ 0.5 = 1.5

Error approximation
1.5 0.5 a = = 33.3% > 0.05% 1.5
John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

Example

Third approximation 2
e
0.5

0.5 = 1+ 0.5 + = 1.625 2

Error approximation
1.625 1.5 a = = 7.69% > 0.05% 1.625

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4 John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Example
Terms 1 2 3 4 5 6 1 1.5 1.625 1.645833333 1.6484375 1.648697166 Result t 39.3% 9.02% 1.44% 0.175% 0.0172% 0.00142% 33.3% 7.69% 1.27% 0.158% 0.0158% a

True value is 1.6487212707

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4 John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Round-Off Errors

Round-off errors are caused because exact numbers can not be expressed in a fixed number of digits as with computer representations For example, a 32-bit number only has a range from 2147483648 to 2147483647 For larger numbers or fractional quantities, we can use floating point numbers, but we will encounter round-off errors

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Floating-Point

What can be represented in N bits? Unsigned 2s Complement 1s Complement BCD But, what about? very large numbers? 9,349,398,989,787,762,244,859,087,678 very small number? 0.0000000000000000000000045691 rationals irrationals transcendentals 2/3 0 - 2N-1 to to 2N 2N-1 - 1
N

-2N-1+1 to 2N-1-1 0 to 10N/4 - 1

2
e
John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

Recall Scientific Notation

decimal point 23 exponent Sign, magnitude 1.673 x 10 -24 6.02 x 10

Mantissa

radix (base)

Sign, magnitude

Issues:

IEEE F.P.

1.M x 2

e - 127

Arithmetic (+, -, *, / ) Representation, Normal form Range and Precision Rounding Exceptions (e.g., divide by zero, overflow, underflow) Errors Properties ( negation, inversion, if A B then A - B 0 )
ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4 John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Floating-Point Numbers

Representation of floating point numbers in IEEE 754 standard: 1 8 23 single precision E sign S M mantissa: exponent: sign + magnitude, normalized excess 127 binary integer binary significand w/ hidden integer bit: 1.M 1 < E < 254 (E=0,255 are used for special values) -1.5 = 1 01111111 10 . . . 0

actual exponent is e = E - 127 S E-127 N = (-1) 2 (1.M) 0 = 0 00000000 0 . . . 0

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Floating point numbers

Problems:
Limited range of representable numbers

Overflow and underflow

Finite number of representable numbers within range Interval between numbers increases as numbers grow in magnitude

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Floating point numbers

From Numerical Methods for Engineers, Chapra and Canale, The McGraw-Hill Companies, Inc.
ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4 John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Overflow and underflow errors

Range of numbers for IEEE single-precision is from 2-126 to (2-2-23)2127 or 1.18x10-38 to 3.40x1038 Overflow becomes infinity and underflow becomes zero. Double precision extends range to (2-252)21023
John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

Infinity and NaNs

result of operation overflows, i.e., is larger than the largest number that can be represented overflow is not the same as divide by zero (raises a different exception) +/- infinity S 1...1 0...0

It may make sense to do further computations with infinity e.g., X/0 > Y may be a valid comparison

Not a number, but not infinity (e.q. sqrt(-4)) invalid operation exception (unless operation is = or =) NaN S 1 . . . 1 non-zero HW decides what goes here

NaNs propagate: f(NaN) = NaN

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Quantization

Finite number of representable numbers due to round-off or chopping becomes 3.1416 or 3.1415 instead of 3.1415926535 Due to base-conversion, rational numbers may become unrepresentable as well.
0.110 becomes 0.000110011001100110011001102

Round-off is better than chopping. Upper bound error of x/2 instead of x.

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

Increasing Interval x

Allows floating point numbers to preserve significant digits Quantization errors will be proportional to magnitude of number
x for chopping x x for rounding x 2

is the machine epsilon - measure of the precision of the mantissa

= b1t with no implicit 1

= bt with implicit 1
ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Round-off errors

Large numbers of interdependent computations Adding a large and small number Subtractive calculation
Subtracting two nearly equal numbers

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Real world examples

Why does round-off error matter? Real world examples (from

http://mathworld.wolfram.com/RoundoffError.html)

Vancouver Stock Exchange Ariane rocket Patriot missile

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Vancouver Stock Exchange

In 1982, the index was initiated with a starting value of 1000.000 with three digits of precision and truncation After 22 months, the index was at 524.881 The index should have been at 1009.811. Successive additions and subtractions introduced truncation error that caused the index to be off so much.

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

Ariane rocket

Ariane 5 rocket was launched in June 1996. The rocket was on course for 36 seconds and then veered off and crashed The internal reference system was trying to convert a 64-bit floating point number to a 16-bit integer. This caused an overflow which was sent to the onboard computer The on-board computer interpreted the overflow as real flight data and bad things happened.
John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

Patriot missile

The Patriot missile had an on-board timer that incremented every tenth of a second Software accumulated a floating point time value by adding 0.1 seconds Problem is that 0.1 in floating point is not exactly 0.1. With a 23 bit representation it is really only 0.0999999046326. Thus, after 100 hours (3,600,000 ticks), the software timer was off by .3433 seconds. Scud missile travels at 1676 m/s. In .3433 seconds, the Scud was 573 meters away from where the Patriot thought it was.
John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

Next Lecture

Taylor Series Truncation Error Read Chapter 4

ECE 257 Numerical Methods and Scientific Computing Fall 2004 Lecture 4

John A. Chandy Dept. of Electrical and Computer Engineering University of Connecticut