ERROR ANALYSIS 669
as YACC, Yet Another Compiler Compiler. Passing up
the obvious YANG (Yet AnotherNetwork Guru?),they
settled on Yahoo!-including the exclamation mark-
and began to seek venture capital.
Their first success was with the prestigious Sequoia
Capital, which had to its credit such winnersas Apple,
Atari, Oracle, and Cisco.With $4 million from that
source, Yahoo! was able to recruit additional staff and
the company began to grow.
Yahoo!is sometimes said to stand for Yet Another
Hierarchical Officious Oracle. The “H” is significant in
that only Yahoo!and afew other search engines usethe
intelligence of human labor to categorize the sites they
find.
The basic decision facing the Yahoo! founders was how
toproducerevenuefromsuchan obviouslyuseful
product. If they were to chargefor access, many users
would choose notto pay. And if they wereto decide to
accept paid ads, customers might be turned off. But it
was decided there was no alternative to paid advertis-
ing. And, after a small amount of initial grousing, users
stopped complaining andthenewrevenuesource
promptedReuters to grant Yahoo! $40 million of
fresh venture capital.
In 1996, Yahoo!sold a 30% interest in thecom-
pany to Softbank, the dominantsoftware firm of Japan,
with Yang retaining a 15% interest. Softbank, which
had previously purchased the Ziff-Davispublishing
company for $2.1 billion, prompted the launching of
the publication of Yahoo! Internet
Life, which
promptly attracted many thousands of subscribers.
In 1998Yahoo! earned its first profit and improved
greatly on this in 1999. By 1999alsoYahoo! had
become by f a r the most commonly visited search
engine site on theWorld Wide Web. Yanghas, indeed,
become more thanYet Another Network Guru.
Bibliography
1988. Rifkin, G., and Harrar, G. The Ultimate Entrepreneur:
The Stovy of Ken Olsen and Digital Equipment Corporation.
Chicago, IL: Contemporary Books.
1992. Wallace, J., and Erickson, J. Hard Drive: Bill Gates and
the Making of the Microsoft Empire. New York: John Wiley.
1993. Rogowski, S., and Reilly, E. D. “Entrepreneurs,”in The
Encyclopedia of Computer Science, 3rd Ed. (eds. A. Ralston
and E. D. Reilly), 517-526. New York: Van Nostrand
Reinhold. The third edition version included profiles of
Seymour Cray and Robert Noyce, who have been accorded
full biographies in this edition, and of Dan Bricklin, Nolan
Bushnell, Marylene Delbourg-Delphis, Gary Kildall, William
Millard, and Adam Osborne, which are not included in this
Fourth Edition article.
1996. Grove, A. Only the Paranoid Survive: How to Exploit
the Crisis Points that Challenge Every Company and Career.
New York: Doubleday.
1997. Cringely, R. X. Accidental Empires: How the Boys of
Silicon Valley Make Their Millions, Battle Foreign
Competition, and Still Can’t Get a Date. New York: Harper.
1997. Jager, R. D., and Ortiz, R. In the Company of Giants.
New York: McGraw-Hill. Profiles of Steve Case, Michael Dell,
Bill Gates, Andrew Grove, William Hewlett, Steve Jobs, Sandy
Kurtzig, and Ken Olsen.
1997. Murray, C. J., The Superman: The Story of Seymour
Cray and the Technical Wizardry Behind the Supercomputer.
New York: John Wiley.
1997. Reid, R. H. Architects of the Web. New York: John Wiley.
Profiles of Marc Andreessen, Kim Polese, Jerry Yang, and
others.
1997. Stross, R. E. Steve Jobs and the NeXT Big Thing. New
York: Athenium.
1997. Wallace, J. Overdrive : Bill Gates and the Race to Control
Cyberspace. New York: John Wiley.
1997. Wilson, M. The Difference Between God and Larry Ellison:
Inside Oracle Corporation.New York: William Morrow.
1998. Swisher, K.AOL.COM: How Steve Case Beat Bill Gates,
Nailed the Netheads, and Made Millions in the War for the
Web. New York: Times Books.
1998. Hamm, S. “The Education of Marc Andreessen,” Business
Week, 13 April, 85-92.
1998. Kirsner, S. “The Legend of Bob Metcalfe,” Wived, 6.11,
November, 182-186,232-234, 246-247.
1999. Byman, J. Andrew Grove and the Intel Corporation.
Greensboro, NC: Morgan Reynolds.
1999. Clark, J. Netscape Time: The Making of the Billion-Dollar
Start-up that Tookon Microsoft. New York:St Martin’s Press.
1999. Dell, M. Direct from Dell: Strategies That Revolutionized
An Industry. New Yrok: HarperBusiness.
1999. Hof, R. D., Harnm, S., and Sager, I. ”SUN Power: How
Scott McNealy is Shaping the Future of Computing,” Business
Week, 18 January, 64-72.
1999. Quittner, J. “Tim Berners-Lee,” Time, 153, 12 (29 March),
192-194.
1999. Ramo, J. C., and Quittner, J. “An Eye on the Future:
The Laughing Billionaire,” Time Magazine, 154, 26 (27 Dec.),
50-67 (Man-of-the Year Jeff Bezos).
2000. Lewis, M. The New New Thing. New York: W. W. Norton.
Stephen J. Rogowski and Edwin D. Reilly
ERGONOMICS
See HUMAN
FACTORS
IN COMPUTING,
In general, the basic arithmetic operations on digital
computers are not exact but aqe subject to rounding or
truncation errors. This article is concerned with the
cumulative effect of these errors. It will be assumed
thatthereaderhasread the article on MATRIX
COMPUTATIONS, since the results will be illustrated by
examples from that area.
Definitions
There are two main methodsof error analysis, known
as forward analysis and backward analysis, respec-
tively.Theymay be illustrated by considering the
solution of an n x n system of linear equations by
670 ERROR ANALYSIS

Gaussian elimination. In this algorithm, the original vector andmatrix norms. A norm of a vector x,
system is reduced successively to equivalent systems denoted by IJx/I,is a nonnegative quantity satisfying
A(')x = b(r),Y = 1 , 2 , . . . , n - 1. In the final system the relations
the matrix of coefficients, A("-'), is upper-triangular, ilxll 2 0 and llxil = 0 iff x = 0,
and the solution is found by back substitution.
IIQXII = IQI llxll7
Inaforward analysis, oneadoptsthe following
strategy: because of rounding errors, the computed IIX + YII 5 IIXII + IIYIl.
derived system A(')x = b(') differs from that which Wewilluseonly twonorms,denoted by XI/^ and
would beobtained by exact arithmetic. It seems 11 x 11 and defined by
reasonable to assume that, if the algorithm is stable,
A(') - A(") and b ( r ) - b(') will be small, and, with llxllz = (E lxi12)1'2 I llxllm = maxlxil.
sufficient ingenuity, bounds would be found for these
Similarly, a norm of a matrix A, denoted by IlAll, is a
"errors." This is perhaps the most natural approach.
nonnegative quantity satisfying the relations
Alternatively, one could adopt the following strategy: lIAJI2 0 and IlAll = 0 iff A = 0,
if the algorithm is stable, presumably the computed
solution % is the exact solution of some system IlQAll = IQI llAll>
+ +
(A E)% = b e, where E and e are relatively small.
IIA + BII I IlAll + IlBltt
Of course, there will be an infinite number of sets
of which x is the exact solution. A successful error IIABII 6 IlAll IIBII.
analysiswill obtain satisfactory bounds for the Wewill useonly two norms, denoted by JJAl/,and
elements of E and e . Such an approach is known as llAilm and defined by
backward error analysis, since it seeks to replace all
errorsmade in the course of the solution by an liA1lz = (max eigenvalue of A A H ) ' I 2 , where AH
equivalent perturbation of the original problem. It has represents the conjugate transpose of A
oneimmediate advantage. Itputstheerrorsmade
11A11, = ma@jlQiiJ).
during the computation on the same footing as those
arising from the data. Hence, when the initial data is It may be verified that
itself inexact, no additional problem is posed.
IIAJ42 5 l l 4 2 IIXIIZ

Early Error Analysis of Elimination llAxllco L llAllw llxllm.

Processes
Inthe 194Os, the imminent arrival of electronic
computers stimulated an interest in error analysis,
andone of the first algorithms to be studied was
Gaussian elimination. Early analyses were all of the
forward type, and typical of the results obtained was
that of Hotelling, who showed that errors in solvingan
n x n system might build up by a factor of qn-'. The
relevance of this result was widely accepted at the
time. Writingin1946, Bargmann, Montgomery, and
von Neumann said of Gaussian elimination: "An error
at any stage affectsall succeeding results and may
become greatly magnified;this explains why instability
should beexpected. " The mood of pessimism was very
infectious, andthe
tendency
become
to en-
meshed in the formal complexityof the algebra of the
analysis seemstohaveprecludedasound assess-
ment of the nature of the problem. Before giving any
error analyses, we discuss fundamental limitations on
the attainable accuracy.
Norms and Floating-point Arithmetic
Wewill need some way of assessing the "size" of a
vector or a matrix. Such a measure is provided by
Most of the early error analyses were for fixed-point
computation, but, since virtually all scientific compu-
tation isnow done infloating point, we restrict
discussion to this case. We use the notation fl(x x y )
to denote the product of two standard floating-point
(fl) numbers as given by the computer under exam-
ination, with ananalogous notation for the other
arithmetic operations. We have the following results
for each of the basic operations, using a mantissa of t
digits in the base p:
fl(x x y ) = x y ( 1 + E ) , I E ~ 5 mp-',
fl(x t Y ) = ( x / y ) ( l + E ) , l ~ Il dp-',
f l ( x ~ Y ) = x ( l + & l ) ~ Y ( 1 + & 21) ~, 1 1 , 1 E 2 / L ~ P - t
where m, d, and s are constants on the orderof unity,
dependingonthe details of therounding or chop-
ping procedure. Described in the language of back-
ward error analysis, we might say, for example, that
the computed sum of two numbers x and y is the
exact sum of two numbers x(1 E I ) and y ( 1 + ~ 2 ) + ,
eachhavinga lowrelative error. On well-designed
computers,
fl(x h y ) = ( x zk y ) ( l + E), IEl 6 sP-'.
 ERROR ANALYSIS 671

For convenience, from now on we assume that all E in attributing this to somemalignant instability in this
the above satisfy the bound I E ~ 5 k E t , where k is of simple arithmetic process; it is the natural loss to be
the order of unity. expected.
By repeated application we have, with an obvious Inherent Sensitivity of the Solution of a
notation, Linear System
fl(a1 + a2 + ' . + a,)
'
Forany computational problem, the inherent sensi-
=uI(~+E~)+u~(~+E~)...+u,(~+E,), tivity of the solution to changes in thedata is of
fundamental importance; yet oddly enough the early
(1 - kD-'),-' 5 1 + El 5 (1 + kK'),-' analyses of Gaussian elimination paid little attention to
(1 - k 3 - y l - r 5 1 + E, _< (1 + ko-')"+1-' it. We consider in a very elementary way the effect of
perturbations SA in the matrix A. We have
v=2,3, ... ,n.
X = (A + SA)-'b = (A-' - A-lS.4A-l + . . . )b
The bounds on the errors arereasonably realistic, and
examples can be constructedin which they are almost = X - A-'SAx + (A-16A)2~ - '.',
attained. Naturally, when n is large, the statistical
giving
distribution can be expected, in general, to result in '
some cancellation of errors and, thus,in actual errors I l X - xll/llxll 5 llA-l~All/(l - IIA-1~41)~
substantially less than the bounds.
provided IlA-'SA(I < 1. The relative error in x will not
One of the most important elementsin elimination be low unless IIA-'SAII is small. Writing
methods is the computation of expressions of the form
IISAII = V I I A I I ~
p = fl(a - x1 x y1 - ' . - x, x y n ) .
'
we see that
The computed p and the error bounds are dependent
on the orderin which operations are performed.If the
operations are performed in the order written above, The inherent sensitivity is thereforedependent on
we obtain //A/IIlA-'II, and this is usually known as the condition
number of A (for the given norm), with respect to
p = a(1 + E ) - xlyl(1 + Fl) - "' - x,y,(l + Fn), matrix inversion or to the solution of linear systems.
where We might now askourselves what sortof limitation we
(1 - 5 1 + E 5 (1 + should expect on theaccuracy of Gaussian elimination
even if it had no menacing instability. The solution of
(1 - k , 3 - y - i 5 1 + Fi 5 (1 + kp-'y+2-'. Ax = b requires n3/3multiplications and additions, an
If one computes 4
average of n per element.From the elementary
discussion given so far, we might risk the following
p = fl(-x1 x y1 - x2 x y2 - . ' . - x, x y, + a), prophecy: even if Gaussian elimination is a stable proc-
then ess, then we can scarcely expect to obtain a bound for
the resulting error whichisless than that resulting
~=-xlyl(l+El)-...-x,y,(l+En)+~(l+F) , from a perturbation SA in A satisfying, say
+
(1 - k9-f)nL3-i(1 E ~ 5) (1 +
IF1 5 kp-'. In fact, this bound for the effect is usually reasonably
In describing the last result in terms of backward error realistic, provided that pivoting is used. Indeed, the
analysis, we might say, for example, that itisexact advantages conferred by the statistical distribution of
for data xi(l + Ei), yi, and a(1 + F ) , putting all the rounding errors is such that the error is usually less
perturbations in the xi and a. Alternatively, we could than themaximum error thatcould be causedby such
+
say it is exact for data, xi, yi (1 Ei),and a( 1 F ). + a perturbation.

Note that although the errors made can be equated

with the effect of small relative perturbations in the
data, therelative error in the computedp may be arbi-
trarily high, depending on the degree of cancellation
that takes place. Indeed, if the true p is zero, one may
have an infinite relative error. One would not think of
Backward Error Analysis of Gaussian
Elimination
Gaussian elimination provides a verygood illustra-
tion of the power and simplicity of backward error
analysis. The elimination process may be described as
672 ERROR ANALYSIS

the production of a unit lower triangular matrix L and +

where the factors 1 E,i and 1 Fr are of the type +
an upper triangular matrix U such that LU = A. The discussedin connection with the computation of p
solution of the system Ax = b is then carried out in the above. Hence, the computed yi satisfyexactly the
two steps: relation
L y = b, Ux=y 1,1(1+Gr1)+lr2~2(1+Gr2)+"'

In the backward error analysis, one shows that the + l r , r - l y r - l ( l + Gr,r-1) + y r ( 1 + G r r ) = br,
computed L and U satisfy the relation LU = A + E and
obtains bounds for the elements of E. One then shows where
that the computed solutions y and x of the triangular (l+Gri)=(l+Eyi)/(l+Fy), i = l , ...,v - 1 .
systems satisfies the equations
( L + SL)y = b, (U + SU)x = y
1 + Grr = 1 / ( 1 + F r )
Notice that by dividing through by 1 + Fr we are
and obtains bounds for the elementsof SL and SU. The able to restrict ourselves to perturbations in L. The
computed x therefore solves exactly the system
computed y therefore satisfiesexactly the relation
(L+SL)(U+SU)x=b +
( L SL)y = b, where SLij = LijGij.
(A+E+SLU+LSU+SLSU)x=b We certainly have
Hence it is the exact solution of ( A + F )x = b, where (1 - kp-')" 5 ( 1 + G i j ) 5 ( 1 + k,8-')"
J1F11 = 1IE + SLU + LSU + SLSU I/ most of the factors, of course, satisfying much better
bounds. Bounds of the above type are cumbersome to
I IIE 11 + IlLlI I l S U It + It lJ II ll6LIl
use, and weobserve that, if k n P ' < 0 . 1 , as will
+ IISLII l l S U 11, usually be the case, then, using the binomial theorem,
and from the bounds for E, SL, and S U , one obtains a (1 + kp-')" 5 1 + (1.06)knp-',
bound for F.
( 1 - kp-')' 2 1 - (1.06)knp-'.
The simplicity of the technique may be illustrated by
presenting the analysis of the solution of the system Hence, we have
Ly = b. We first make the following observations: ISLijl 5 (l.06)knB-r(LijI,
1 . The relevant system to be analyzed is that with the giving, for example
computedmatrix L, not the L that wouldhave
resulted from exact computation.
Theanalysisis almost trivial, though earlier error
2. Since during the course of the analysis we do not
analyses of the solution of triangular systems were
attempta direct comparisonbetweencomputed
extremely complicated.
and exact values, there is no need to denote com-
puted quantities by bars. It is to beunderstood that If the computation of yr had been expressed in the
all symbols refer to computed quantities. form
3. It is only at the final stage when we have expressed Yr=fl(br-lrlyl-'"-1r,r-1Yr-1),
thecomputed solution as the exact solution of
(A + F )x = b and have obtained a bound for lIF 11 then wecouldstill obtain a relation of theform
that we attempt to compare the computed x with +
(L SL)y = b, but in this case the bounds on the
the true x, and at this stage we can use the result of elements of SL would be appreciably larger.
the previous section.
On most computers it is possible to accumulate either
of the expressions for yr in double precision, rounding
At a typical stage in the triangular solution, y1, y 2 , . . . ,
to single precision only on completion. If this is done,
y v - l have been computed and y r is determined from
then we again obtain a relation of the form
the relation
Y r = f l ( - l r l y l - l r 2 ~ 2- ' - 1r.r-1yr-1 + br), lrlYl(1 + G r 1 ) + I r 2 ~ 2 ( 1+ G r 2 ) + ' '

+ l r , r - l Y r - l ( l + G r , r - l ) + Y r ( 1 + G r r ) = br,
but now the quantities I Gri/( i < Y ) have bounds of
order P-2f and can therefore virtually be neglected,
while 1 Grr1 has the bound k6'. We therefore have a

result that might well be described as best possible,
having regard to theprecision of computation. Indeed,
the residual vector b - Ly corresponding to the com-
puted y willalmost certainly be smaller thanthat
corresponding to the correctly rounded solution!
Theanalysis of the solution of Ux = y is almost
identical to that of Ly = b, while the analysis of the
factorization process isonlymarginally morecom-
plicated. If the L and U areproduced as inclassi-
cal Gaussian elimination, thenonecanshowthat
+
LU = A E , where, denoting the maximum modulus
of any element arising during the decomposition by g,
we certainly have
If the factors L and U are determined directly, using
the relations
~ll, . ulI. .-- a . .ll- I . t1 u l j - ' . ' - 1+1uj-1,j
and
- al1, . - 1.t t t l l j - . . . - l i . i - l ~ i - l , j
u[.I.-
and the expressions on the right are accumulated in
double precision, an even more satisfactory bound
may be determined for E. Indeed, ignoring quantities
of the order of magnitude of p-2t, we certainly have
I eij 1 5 gkp-', where g is now the elementof maximum
modulus in the computed U.Again, we have what may
be regarded as a "best possible" result.
The reader may be surprised that no reference has
been made to pivoting or to the size of the lpq. The
importance of pivotingis concealed. If any of the
multipliersis large, g willusually bemuch larger
than max jaijl. When pivotingisused 11p4( 5 1, and
there will not usually be much growth in the size
of the elements of the reduced matrices or of U rela-
tive to the initial set of aij. When A is positive definite
or diagonally dominant, no growthcan take place,
and we have a guaranteed a priori bound for IIE // in
terms of A.
In 1947, von Neumann and Goldstine considered the
special case of the inversion of a positivedefinite
matrix with pivoting, and obtained a result for fixed-
point computation thatis only marginallyweaker than
canbeobtained by arguments of the above type,
though the analysis was far more complicated. Their
analysis is often described as a forward error analy-
sis, but itisin fact of the backward type, although
at no stage are results expressed in a form such as
to emphasize this. Thefinal result of an analysis of
the above type for the solution of a positive definite
ERROR ANALYSIS 673
system is to guarantee that it is the exact solution of
+
( A E )x = b and to give a bound for E of the type
I F Ii 5 f(n)kP-'IiAil,
where f ( n )is a modestfunction of n, depending alittle
on thedetails of the arithmetic. When backward error
analysisisapplied tomatrix inversion, onecannot
+
show that X is the exact solution of ( A E )X = I ,
with a similar bound for E, because itis not true.
However, the rth column, x,, of X is the exact solu-
+
tion of some ( A E,)x, = e,, where e , is the rth
column of I : the E , are all different, but havethe same
satisfactory uniform bound. This result is implicit in
that of von Neumannand Goldstine, but it is well
concealed!
Orthogonal Transformations
Experience with error analyses of matrix processes
gradually exposed the fact that control of growth in
j = 1 , . ..,i - 1 derived matrices is the key to stability. If orthogo-
nal transformations Q are used, then, since IlQAI12 =
llAQl12 = llAl12, no general growth C U M take place.
Although the algebra is a little complicated, a fairly
l.= t ., . . . , n, general analysis canbe given of wholeclasses of
algorithms based on orthogonal transformations, both
for the solution of equations and the eigenvalue prob-
lem. One can show, for example, that for a sequence
of Y orthogonal similarity transformations, the final
computed transform A(")satisfies exactly a relation of
the form
+
A(') = Q T ( A E ) Q ,
where Q is exactly orthogonal and
IIEII 5 rf(~)llAllkP-f,
where f ( n ) is some quite innocuous function of n.
Hence,the eigenvalues of A(') are exactly those of
+
A E, and we are back with perturbation theory.
A Posteriori Error Bounds
The bounds discussed so far are of the a priori type.
The main function of suchan analysis is to show
whether or not an algorithm is stable and, if not, to
pinpoint the reasons for its instability.
When a solution has been determined, one can usually
obtain much sharper backwarderrorbounds. For
example,from a computed eigenvalue X andan
eigenvector u, such that /Ju112= 1, one can compute
the residual defined by r = Au - Xu. Thismaybe
written in the form ( A - ruH)u= Xu, showing that X
and u are exact for the matrix A - ruH. When A is
Hermitian, this implies that A has an eigenvaluein
+
the interval X - Ilrl12,X /lrllz.Similarly, when solving
linear equations, one can compute r = b - Ax. If r is
674 ERROR
CORRECTING AND DETECTING
CODE
computed accurately, it can then beused to obtain an
improved solution by solving A6 = r. This process is
called iterative refinement.
Iterative Methods
It was at one time thought that iterative methods for
solving linear equations orthe eigenvalue problem
would give far greater accuracy than direct methods,
since one works with theinitial A throughout. In fact,
this advantage is largely illusory. In Jacobi’s method
for linear equations, one derives an improved xir+’)
from the relation
but the right-hand side cannot be computed exactly.
From the above analysis it is clear that one is really
workingwith a matrixwithelements +
aij( 1 eij),
where the eii are differentin each iteration. When
iterative methods are used in practice, iteration is
usually terminated before attaining the accuracygiven
immediately by a direct method, even without itera-
tive refinement. Since, as we mentioned earlier, the
results obtained with good direct methods are almost
“best possible,” this is to be expected.
Interval Arithmetic and Significant
Digit Arithmetic
Attemptshave beenmadeto obtain errorbounds
for computed quantities onthe computer itself.
In interval arithmetic (q.v.), an ordered pair [al,a,]
of floating-point numbers is stored at each stage in the
computation, and it is guaranteed that the true num-
ber a lies in the interval al <_ a 5 a,. Used in a direct
manner, the results achieved are very pessimistic; in
fact, the computer merely performs numerically the
analog of what was done algebraically in the early
forwarderror analysis of the Hotelling type. The
intervals become very large. The apparently reason-
able assumption that in stable algorithms thecom-
puted quantities will be close to those arising in exact
computation is frequently quite false. This is particu-
larly true of algorithms for the eigenvalue problem.
In significantdigit (q.v.) arithmetic, onedoesnot
work with normalized floating-point numbers, on the
grounds that when cancellation takes place, the zeros
introduced are non-significant. Thepossibilities of
significant digit arithmetic have been well exploited
by Metropolis and Ashenhurst.
The realization that neither interval arithmetic nor sig-
nificant digit arithmetic provides an automatic answer
to error analysis led to an overreaction against them.
The provisionof the relevant hardware facilities should
‘makethem economic,andwhencombinedwith a is a measure of this redundancy.
more general appreciation of theoretical error analysis,
they have an important role to play.
Bibliography
1963. Wilkinson, J. H. Rounding Errors i n Algebraic Processes.
London: Her Majesty’s StationeryOffice and Upper Saddle
River, NJ: Prentice Hall.
1965. Wilkinson, J. H. The Algebraic Eigenvalue Problem.
Oxford: Clarendon Press.
1967. Forsythe, G. E. and Moler, C. B. Computer Solution of
Linear Algebraic Systems. Upper Saddle River, NJ: Prentice
Hall.
1971. Wilkinson,J. H. “Modern Error Analysis,” SIAM Review,
13, 548-568.
1979. Moore, R. E. Methods and Appiications of Interval
Analysis, Philadelphia: SIAM Publications.
1989. Kahaner, D.,Moler, C., and Nash, S. Numerical Methods
and Software. Upper Saddle River, NJ: Prentice Hall.
James H.Wilkinson
ERRORCORRECTING A N D
DETECTINGCODE
Error-detecting and error-correcting codes are essen-
tial parts of most forms of digital communication and
storage. Proper operation of the Internet (q.v.),
modems (q.v.), compact discs (see OFTICALSTORAGE),
and computers would be impossible without them.
Many recent advances in the speed and reliability of
digital communication are directly related to improved
design of codes.
Codes introduce redundant bits into a data stream so
that transmission errors in the original data bits can be
detected and sometimes corrected. Perhaps the best-
knowncode is the simple parity check code (see
PARITY) that uses a single redundant bit and candetect
an odd number of errors in a group of bits. In inter-
active communication, such as the Internet, it is often
sufficient to detect errors, and then request retrans-
mission of the data. In other situations, such as data
storage on a compact disc, rereading erroneous data
from the discwould frequently result in thesame
error, so the errors must be corrected.
A diagram of a simplified coding scheme is shown in
Fig. 1. In this type of coding (called block coding) a
group of k data bits are read into the encoder which
then produces a corresponding n-bit code word with
n > k. There are 2k possible k-bitpatterns at the input,
and 2 corresponding n-bit code words. This collection
of possible code words is called a “code.” Since it can
produce only 2 k out of the 2” possible n-bit patterns,
the encoder introduces redundancy into thedata
stream. The code rate, defined as the ratio of k to n,