Textbook Numerical Linear Algebra An Introduction 1St Edition Holger Wendland Ebook All Chapter PDF

Numerical Linear Algebra An
Introduction 1st Edition Holger

Wendland
Visit to download the full and correct content document:
https://textbookfull.com/product/numerical-linear-algebra-an-introduction-1st-edition-h
olger-wendland/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...
Numerical Linear Algebra A Concise Introduction with

MATLAB and Julia Folkmar Bornemann
https://textbookfull.com/product/numerical-linear-algebra-a-
concise-introduction-with-matlab-and-julia-folkmar-bornemann/
SF1624 Algebra and Geometry Introduction to Linear

Algebra for Science Engineering Daniel Norman
https://textbookfull.com/product/sf1624-algebra-and-geometry-
introduction-to-linear-algebra-for-science-engineering-daniel-
norman/
Introduction to Linear Algebra for Science and

Engineering Daniel Norman
https://textbookfull.com/product/introduction-to-linear-algebra-
for-science-and-engineering-daniel-norman/
Linear Algebra Seymour Lipschutz
https://textbookfull.com/product/linear-algebra-seymour-
lipschutz/
Introduction to Linear Algebra for Science and
Engineering, 3rd Edition Daniel Norman
https://textbookfull.com/product/introduction-to-linear-algebra-
for-science-and-engineering-3rd-edition-daniel-norman/
Introduction to Applied Linear Algebra Vectors Matrices

and Least Squares 1st Edition Stephen Boyd
https://textbookfull.com/product/introduction-to-applied-linear-
algebra-vectors-matrices-and-least-squares-1st-edition-stephen-
boyd/
Basic Linear Algebra Jürgen Müller
https://textbookfull.com/product/basic-linear-algebra-jurgen-
muller/
Elementary Linear Algebra Ron Larson
https://textbookfull.com/product/elementary-linear-algebra-ron-
larson/
Linear Algebra M. Thamban Nair
https://textbookfull.com/product/linear-algebra-m-thamban-nair/
Cambridge University Press
978-1-107-14713-3 — Numerical Linear Algebra
Holger Wendland
Frontmatter
More Information
Numerical Linear Algebra
This self-contained introduction to Numerical Linear Algebra provides a

comprehensive, yet concise, overview of the subject. It includes standard material
such as direct methods for solving linear systems and least-squares problems, error,
stability and conditioning, basic iterative methods and the calculation of eigenvalues.
Later chapters cover more advanced material, such as Krylov subspace methods,
multigrid methods, domain decomposition methods, multipole expansions,
hierarchical matrices and compressed sensing.
The book provides rigorous mathematical proofs throughout, and gives algorithms
in general-purpose language-independent form. Requiring only a solid knowledge in
linear algebra and basic analysis, this book will be useful for applied mathematicians,
engineers, computer scientists and all those interested in efficiently solving linear
problems.
H O L G E R W E N D L A N D holds the Chair of Applied and Numerical Analysis at the

University of Bayreuth. He works in the area of Numerical Analysis and is the author
of two other books, Scattered Data Approximation (Cambridge, 2005) and
Numerische Mathematik (Springer 2004, with Robert Schaback).
© in this web service Cambridge University Press www.cambridge.org

Holger Wendland
Frontmatter
More Information
Cambridge Texts in Applied Mathematics

All titles listed below can be obtained from good booksellers or from Cambridge
University Press. For a complete series listing, visit www.cambridge.org/mathematics.
Nonlinear Dispersive Waves

MARK J. ABLOWITZ
Flow, Deformation and Fracture

G. I. BARENBLATT
Hydrodynamic Instabilities
FRANÇOIS CHARRU
The Mathematics of Signal Processing
STEVEN B. DAMELIN & WILLARD MILLER, JR
Introduction to Magnetohydrodynamics (2nd Edition)
P. A. DAVIDSON
An Introduction to Stochastic Dynamics
JINQIAO DUAN
Singularities: Formation, Structure and Propagation
J. EGGERS & M. A. FONTELOS
A Physical Introduction to Suspension Dynamics

ÉLISABETH GUAZZELLI & JEFFREY F. MORRIS
Discrete Systems and Integrability

J. HIETARINTA, N. JOSHI & F. W. NIJHOFF
Iterative Methods in Combinatorial Optimization

LAP CHI LAU, R. RAVI & MOHIT SINGH
An Introduction to Polynomial and Semi-Algebraic Optimization

JEAN BERNARD LASSERRE
An Introduction to Computational Stochastic PDEs
GABRIEL J. LORD, CATHERINE E. POWELL & TONY SHARDLOW

Holger Wendland
Frontmatter
More Information
Numerical Linear Algebra

An Introduction
HOLGER WENDLAND
Universität Bayreuth, Germany

Holger Wendland
Frontmatter
More Information
University Printing House, Cambridge CB2 8BS, United Kingdom

One Liberty Plaza, 20th Floor, New York, NY 10006, USA
477 Williamstown Road, Port Melbourne, VIC 3207, Australia
4843/24, 2nd Floor, Ansari Road, Daryaganj, Delhi – 110002, India
79 Anson Road, #06-04/06, Singapore 079906
Cambridge University Press is part of the University of Cambridge.

It furthers the University’s mission by disseminating knowledge in the pursuit of
education, learning, and research at the highest international levels of excellence.
www.cambridge.org
Information on this title: www.cambridge.org/9781107147133
DOI: 10.1017/9781316544938
c Holger Wendland 2018

This publication is in copyright. Subject to statutory exception
and to the provisions of relevant collective licensing agreements,
no reproduction of any part may take place without the written
permission of Cambridge University Press.
First published 2018
Printed in the United Kingdom by Clays, St Ives plc
A catalogue record for this publication is available from the British Library.
ISBN 978-1-107-14713-3 Hardback
ISBN 978-1-316-60117-4 Paperback
Cambridge University Press has no responsibility for the persistence or accuracy of
URLs for external or third-party internet Web sites referred to in this publication,
and does not guarantee that any content on such websites is, or will remain,
accurate or appropriate.

Holger Wendland
Frontmatter
More Information
Contents
Preface page ix
PART ONE PRELIMINARIES 1

1 Introduction 3
1.1 Examples Leading to Linear Systems 5
1.2 Notation 10
1.3 Landau Symbols and Computational Cost 13
1.4 Facts from Linear Algebra 17
1.5 Singular Value Decomposition 24
1.6 Pseudo-inverse 26
Exercises 29
2 Error, Stability and Conditioning 30
2.1 Floating Point Arithmetic 30
2.2 Norms for Vectors and Matrices 32
2.3 Conditioning 46
2.4 Stability 54
Exercises 55
PART TWO BASIC METHODS 57

3 Direct Methods for Solving Linear Systems 59
3.1 Back Substitution 59
3.2 Gaussian Elimination 61
3.3 LU Factorisation 65
3.4 Pivoting 71
3.5 Cholesky Factorisation 76

Holger Wendland
Frontmatter
More Information
vi Contents
3.6 QR Factorisation 78
3.7 Schur Factorisation 84
3.8 Solving Least-Squares Problems 87
Exercises 100
4 Iterative Methods for Solving Linear Systems 101
4.1 Introduction 101
4.2 Banach’s Fixed Point Theorem 102
4.3 The Jacobi and Gauss–Seidel Iterations 106
4.4 Relaxation 116
4.5 Symmetric Methods 125
Exercises 130
5 Calculation of Eigenvalues 132
5.1 Basic Localisation Techniques 133
5.2 The Power Method 141
5.3 Inverse Iteration by von Wielandt and Rayleigh 143
5.4 The Jacobi Method 153
5.5 Householder Reduction to Hessenberg Form 159
5.6 The QR Algorithm 162
5.7 Computing the Singular Value Decomposition 171
Exercises 180
PART THREE ADVANCED METHODS 181

6 Methods for Large Sparse Systems 183
6.1 The Conjugate Gradient Method 183
6.2 GMRES and MINRES 203
6.3 Biorthogonalisation Methods 226
6.4 Multigrid 244
Exercises 258
7 Methods for Large Dense Systems 260
7.1 Multipole Methods 261
7.2 Hierarchical Matrices 282
7.3 Domain Decomposition Methods 307
Exercises 327
8 Preconditioning 329
8.1 Scaling and Preconditioners Based on Splitting 331
8.2 Incomplete Splittings 338
8.3 Polynomial and Approximate Inverse Preconditioners 346

Holger Wendland
Frontmatter
More Information
Contents vii
8.4 Preconditioning Krylov Subspace Methods 357

Exercises 368
9 Compressed Sensing 370
9.1 Sparse Solutions 370
9.2 Basis Pursuit and Null Space Property 372
9.3 Restricted Isometry Property 378
9.4 Numerical Algorithms 384
Exercises 393
Bibliography 395
Index 403

Holger Wendland
Frontmatter
More Information
Preface
Numerical Linear Algebra (NLA) is a subarea of Applied Mathematics. It is

mainly concerned with the development, implementation and analysis of nu-
merical algorithms for solving linear problems. In general, such linear prob-
lems arise when discretising a continuous problem by restricting it to a finite-
dimensional subspace of the original solution space. Hence, the development
and analysis of numerical algorithms is almost always problem-dependent. The
more is known about the underlying problem, the better a suitable algorithm
can be developed.
Nonetheless, many of the so-derived methods are more general in the sense
that they can be applied to larger classes of problems than initially intended.
One of the challenges in Mathematics is deciding how to describe the neces-
sary assumptions, under which a certain method works, in the most general
way. In the context of NLA, this means finding for each method the most gen-
eral description of matrices to which the method can be applied. It also means
extracting the most general methods from the vast number of available algo-
rithms. Particularly for users with new problems this is crucial, as it allows
them to apply and test well-established algorithms first, before starting to de-
velop new methods or to extend existing ones.
In this book, I have attempted to use this matrix-driven approach rather than
the problem-driven one. Naturally, the selection of the material is biased by
my own point of view. Also, a book on NLA without any examples would be
rather dire, so there are typical examples and applications included to illustrate
the methods, but I have tried to restrict myself to simple examples, which do
not require much previous knowledge on specific problems and discretisation
techniques.
During the past years, I have given courses on Numerical Linear Algebra at
advanced BSc and early MSc level at the University of Sussex (UK), the Uni-
versity of Oxford (UK) and the University of Bayreuth (Germany). I have also
ix

Holger Wendland
Frontmatter
More Information
x Preface
given courses on Numerical Analysis which covered parts of the NLA material
in Oxford, Göttingen (Germany) and Bayreuth.
This book on Numerical Linear Algebra is based on these courses and the ma-
terial of these courses. It covers the standard material, as well as more recent
and more specific techniques, which are usually not found in standard text-
books on NLA. Examples include the multigrid method, the domain decompo-
sition method, multipole expansions, hierarchical matrices and applications to
compressed or compressive sensing. The material on each of these topics fills
entire books so that I can obviously present only a selection. However, this
selection should allow the readers to grasp the underlying ideas of each topic
and enable them to understand current research in these areas.
Each chapter of this book contains a small number of theoretical exercises.
However, to really understand NLA one has to implement the algorithms by
oneself and test them on some of the matrices from the examples. Hence, the
most important exercises intrinsic to each chapter are to implement and test the
proposed algorithms.
All algorithms are stated in a clean pseudo-code; no programming language
is preferred. This, I hope, allows readers to use the programming language of
their choice and hence yields the greatest flexibility.
Finally, NLA is obviously closely related to Linear Algebra. However, this is
not a book on Linear Algebra, and I expect readers to have a solid knowledge
of Linear Algebra. Though I will review some of the material, particularly to
introduce the notation, terms like linear space, linear mapping, determinant etc.
should be well-known.

P AR T ONE
PRELIMINARIES
13:04:03, subject to the Cambridge

Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/9781316544938
Core terms of use, available at https://www.cambridge.org/core/terms. https://doi.org/10.1017/9781316544938
1
Introduction
In this book, we are concerned with the basic problem of solving a linear sys-
tem
Ax = b,
where A ∈ Rn×n is a given invertible matrix, b ∈ Rn is a given vector and x ∈ Rn

is the solution we seek. The solution is, of course, given by
x = A−1 b,
but does this really help if we are interested in actually computing the solu-
tion vector x ∈ Rn ? What are the problems we are facing? First of all, such
linear systems have a certain background. They are the results of other math-
ematical steps. Usually, they are at the end of a long processing chain which
starts with setting up a partial differential equation to model a real-world prob-
lem, continues with discretising this differential equation using an appropriate
approximation space and method, and results in such a linear system. This is
important because it often tells us something about the structure of the matrix.
The matrix might be symmetric or sparse. It is also important since it tells us
something about the size n of the matrix. With simulations becoming more
and more complex, this number nowadays becomes easily larger than a mil-
lion, even values of several hundreds of millions are not unusual. Hence, the
first obstacle that we encounter is the size of the matrix. Obviously, for larger
dimensions n it is not possible to solve a linear system by hand. This means
we need an algorithmic description of the solution process and a computer to
run our program.
Unfortunately, using a computer leads to our second obstacle. We cannot rep-
resent real numbers accurately on a computer because of the limited number
system used by a computer. Even worse, each calculation that we do might
lead to a number which is not representable in the computer’s number system.

02
4 Introduction
Hence, we have to address questions like: Is a matrix that is invertible in the

real numbers also invertible in the number system used by a computer? What
are the errors that we make when representing the matrix in the computer and
when using our algorithm to compute the solution. Further questions that easily
come up are as follows.
1. How expensive is the algorithm? How much time (and space) does it require
to solve the problem? What is the best way of measuring the cost of an
algorithm?
2. How stable is the algorithm? If we slightly change the input, i.e. the matrix
A and/or the right-hand side b, how does this affect the solution?
3. Can we exploit the structure of the matrix A, if it has a special structure?
4. What happens if we do not have a square system, i.e. a matrix A ∈ Rm×n
and a vector b ∈ Rm . If m > n then we have an over-determined system
and usually cannot find a (unique) solution but might still be interested in
something which comes close to a solution. If m < n we have an under-
determined system and we need to choose from several possible solutions.
Besides solving a linear system, we will also be interested in a related topic,
the computation of eigenvalues and eigenvectors of a matrix. This means we
are interested in finding numbers λ ∈ C and vectors x ∈ Cn \ {0} such that
Ax = λx.
Finding such eigenvectors and eigenvalues is again motivated by applications.
For example, in structural mechanics a vibrating system is represented by finite
elements and the eigenvectors of the corresponding discretisation matrix reflect
the shape modes and the roots of the eigenvalues reflect the frequencies with
which the system is vibrating. But eigenvalues will also be helpful in better
understanding some of the questions above. For example, they have a crucial
influence on the stability of an algorithm.
In this book, we are mainly interested in systems of real numbers, simply be-
cause they arise naturally in most applications. However, as the problem of
finding eigenvalues indicates, it is sometimes necessary to consider complex
valued systems, as well. Fortunately, most of our algorithms and findings will
carry over from the real to the complex case in a straightforward way.
We will look at direct and iterative methods to solve linear systems. Direct
methods compute the solution in a finite number of steps, iterative methods
construct a sequence of approximations to the solution.
We will look at how efficient and stable these methods are. The former means
that we are interested in how much time and computer memory they require.
Particular emphasis will be placed on the number of floating point operations

02
required with respect to the dimension of the linear system. The latter means
for example investigating whether these methods converge at all, under what
conditions they converge and how they respond to small changes in the input
data.
1.1 Examples Leading to Linear Systems

As mentioned above, linear systems arise naturally during the discretisation
process of mathematical models of real-world problems. Here, we want to col-
lect three examples leading to linear systems. These examples are our model
problems, which we will refer to frequently in the rest of this book. They com-
prise the problem of interpolating an unknown function only known at discrete
data sites, the solution of a one-dimensional boundary value problem with fi-
nite differences and the solution of a (one-dimensional) integral equation with
a Galerkin method. We have chosen these three examples because they are
simple and easily explained, yet they are significant enough and each of them
represents a specific class of problems. In particular, the second problem leads
to a linear system with a matrix A which has a very simple structure. This
matrix will serve us as a role model for testing and investigating most of our
methods since it is simple to analyse yet complicated enough to demonstrate
the advantages and drawbacks of the method under consideration.
1.1.1 Interpolation
Suppose we are given data sites X = {x1 , . . . , xn } ⊆ Rd and observations
f1 , . . . , fn ∈ R. Suppose further that the observations follow an unknown gen-
eration process, i.e. there is a function f such that f (xi ) = fi , 1 ≤ i ≤ n.
One possibility to approximately reconstruct the unknown function f is to
choose basis functions φ1 , . . . , φn ∈ C(Rd ) and to approximate f by a function
s of the form
n
s(x) = α j φ j (x), x ∈ Rd ,
j=1
where the coefficients are determined by the interpolation conditions

n
fi = s(xi ) = α j φ j (xi ), 1 ≤ i ≤ n.
j=1

02
6 Introduction
This leads to a linear system, which can be written in matrix form as

⎛ ⎞⎛ ⎞ ⎛ ⎞
⎜⎜⎜φ1 (x1 ) φ2 (x1 ) . . . φn (x1 )⎟⎟⎟ ⎜⎜⎜α1 ⎟⎟⎟ ⎜⎜⎜ f1 ⎟⎟⎟
⎜⎜⎜⎜φ (x ) φ (x ) . . . φ (x )⎟⎟⎟⎟ ⎜⎜⎜⎜α ⎟⎟⎟⎟ ⎜⎜⎜⎜ f ⎟⎟⎟⎟
⎜⎜⎜ 1 2 2 2 n 2 ⎟ ⎜ 2⎟
⎟ ⎜ ⎟ ⎜⎜ 2 ⎟⎟
⎜⎜⎜ .
. .. .. ⎟⎟⎟⎟ ⎜⎜⎜⎜ .. ⎟⎟⎟⎟ = ⎜⎜⎜⎜ .. ⎟⎟⎟⎟ . (1.1)
⎜⎜⎜ . . ... . ⎟⎟⎟⎟ ⎜⎜⎜⎜ . ⎟⎟⎟⎟ ⎜⎜⎜⎜ . ⎟⎟⎟⎟
⎜⎝ ⎠⎝ ⎠ ⎝ ⎠
φ1 (xn ) φ2 (xn ) . . . φn (xn ) αn fn
From standard Numerical Analysis courses we know this topic usually in the
setting that the dimension is d = 1, that the points are ordered a ≤ x1 < x2 <
· · · < xn ≤ b and that the basis is given as a basis for the space of polynomials
of degree at most n−1. This basis could be the basis of monomials φi (x) = xi−1 ,
1 ≤ i ≤ n, in which case the matrix in (1.1) becomes the transpose of a so-
called Vandermonde matrix, i.e. a matrix of the form
⎛ ⎞
⎜⎜⎜1 x1 x12 . . . x1n−1 ⎟⎟⎟
⎜⎜⎜⎜1 x x2 . . . xn−1 ⎟⎟⎟⎟
⎜⎜⎜ 2 2 ⎟ ⎟
.. ⎟⎟⎟⎟ .
2
⎜⎜⎜ . . ..
⎜⎜⎜ .. .. . ... . ⎟⎟⎟⎟
⎜⎝ n−1 ⎠
1 xn xn . . . xn
2
This matrix is a full matrix, meaning that each entry is different from zero, so
that the determination of the interpolant requires the solution of a linear system
with a full matrix.
However, we also know, from basic Numerical Analysis, that we could alter-
natively choose the so-called Lagrange functions as a basis:
n
x − xi
φ j (x) = L j (x) = , 1 ≤ j ≤ n.
i=1
x j − xi
i j
They obviously have the property L j (x j ) = 1 and L j (xi ) = 0 for j i. Thus,

with this basis, the matrix in (1.1) simply becomes the identity matrix and the
interpolant can be derived without solving a linear system at all.
1 1 1
−2 −1 0 1 −2 −1 0 1 −2 −1 0 1
Figure 1.1 Typical radial basis functions: Gaussian, inverse multiquadric and a
compactly supported one (from left to right).
In higher dimensions, i.e. d ≥ 2, polynomial interpolation can become quite

problematic and a more elegant way employs a basis of the form φi = Φ(· − xi ),

02
where Φ : Rd → R is a fixed function. In most applications, this function is

chosen to be radial, i.e. it is of the form Φ(x) = φ(x2 ), where φ : [0, ∞) → R
is a univariate function and x2 = x12 + · · · + xd2 denotes the Euclidean norm.
Examples of possible univariate functions are
Gaussian: φ(r) = exp(−r2 ),

Multiquadric: φ(r) = (r2 + 1)1/2 ,
Inverse Multiquadric: φ(r) = (r2 + 1)−1/2 ,
Compactly Supported: φ(r) = (1 − r)4+ (4r + 1),
where (x)+ is defined to be x if x ≥ 0 and to be 0 if x < 0. The functions are

visualised in Figure 1.1.
In all these cases, except for the multiquadric basis function, it is known that
the resulting interpolation matrix is positive definite (with the restriction of
d ≤ 3 for the compactly supported function). Such functions are therefore
called positive definite. More generally, a good choice of a basis is given by
φ j (x) = K(x, x j ) with a kernel K : Rd × Rd → R, which is positive definite
in the sense that for all possible, pairwise distinct points x1 , . . . , xn ∈ Rd , the
matrix (K(xi , x j )) is symmetric and positive definite.
In the case of the multiquadric basis function, it is known that the interpolation
matrix is invertible and has only real, non-vanishing eigenvalues and that all
but one of these eigenvalues are negative.
Note that in the case of the inverse multiquadric and the Gaussian the matri-
ces are dense while in the case of the compactly supported basis function the
matrix can have a lot of zeros depending on the distribution of the data sites.
Details on this topic can be found in Wendland [133].
1.1.2 Boundary Value Problem

Another application is to compute a stationary solution to the heat equation. In
one dimension, we could imagine an infinitely thin rod of length one, which is
heated in the interior of (0, 1) with a heat source f and is kept at zero degrees
at the boundary points 0, 1. Mathematically, this means that we want to find a
function u : [0, 1] → R with
−u (x) = f (x), x ∈ (0, 1),
with boundary conditions u(0) = u(1) = 0. If the function f is too complicated

or even given only at discrete points then it is not possible to compute the
solution u analytically. In this case a numerical scheme has to be used and the

02
8 Introduction
simplest idea is to approximate the derivative by differences:

u(x + h) − u(x) u(x) − u(x − h)
u (x) ≈ , or u (x) ≈ .
h h
The first rule could be referred to as a forward rule while the second is a back-
ward rule. Using first a forward and then a backward rule for the second deriva-
tive leads to
u (x + h) − u (x) 1 u(x + h) − u(x) u(x) − u(x − h)
u (x) ≈ ≈ −
h h h h
u(x + h) − 2u(x) + u(x − h)
= .
h2
For finding a numerical approximation using such finite differences we may
divide the domain [0, 1] into n + 1 pieces of equal length h = 1/(n + 1) with
nodes
i
xi = ih = , 0 ≤ i ≤ n + 1,
n+1
and set ui := u(xi ). We now define the finite difference approximation uh to u,
as follows: find uh such that uh0 = uhn+1 = 0 and
⎛ h ⎞
⎜⎜ u − 2uhi + uhi−1 ⎟⎟⎟
− ⎜⎜⎝ i+1 ⎟⎠ = fi , 1 ≤ i ≤ n.
h2
Alternatively this linear system of n equations may be written in the form

⎛ ⎞⎛ ⎞ ⎛ ⎞
⎜⎜⎜ 2 −1 0 ⎟⎟⎟ ⎜⎜⎜ uh1 ⎟⎟⎟ ⎜⎜⎜ f1 ⎟⎟⎟
⎜⎜⎜ ⎟⎟⎟ ⎜⎜⎜ h ⎟⎟ ⎜⎜⎜ ⎟ ⎟
⎜⎜⎜−1 2 −1 ⎟⎟⎟ ⎜⎜⎜ u2 ⎟⎟⎟ ⎜⎜⎜ f2 ⎟⎟⎟⎟⎟
⎜⎜ ⎟⎟⎟ ⎜⎜⎜ uh ⎟⎟⎟ ⎜⎜⎜ f ⎟⎟⎟
1 ⎜⎜⎜⎜ −1 2 −1 ⎟⎟⎟ ⎜⎜⎜ 3 ⎟⎟⎟ ⎜⎜⎜ 3 ⎟⎟⎟
⎜
⎜
⎜ .. .. .. ⎟⎟⎟ ⎜⎜⎜ . ⎟⎟⎟ = ⎜⎜⎜ . ⎟⎟⎟ . (1.2)
h2 ⎜⎜⎜ . . . ⎟⎟⎟ ⎜⎜⎜ .. ⎟⎟⎟ ⎜⎜⎜ .. ⎟⎟⎟
⎜⎜⎜ ⎟⎟ ⎜⎜ ⎟⎟⎟ ⎜⎜ ⎟⎟⎟
⎜⎜⎜
⎜⎝ −1 2 −1⎟⎟⎟⎟⎟ ⎜⎜⎜⎜⎜uhn−1 ⎟⎟⎟⎟ ⎜⎜⎜⎜⎜ fn−1 ⎟⎟⎟⎟
⎠⎝ h ⎠ ⎝ ⎠
0 −1 2 un fn
This system of equations is sparse, meaning that the number of non-zero en-
tries is much smaller than n2 . This sparsity can be used to store the matrix and
to implement matrix–vector and matrix–matrix multiplications efficiently.
To obtain an accurate approximation to u, we may have to choose h very small,
thereby increasing the size of the linear system.
For a general boundary value problem in d-dimensions the size of the linear
system can grow rapidly. For example, three-dimensional problems grow over
eight times larger with each uniform refinement of the domain.

02
1.1.3 Integral Equations

In the last section we have introduced a way of solving a differential equation.
A differential equation can also be recast as an integral equation but integral
equations often also come up naturally during the modelling process. Hence,
let us consider a typical integral equation as another example.
We now seek a function u : [0, 1] → R satisfying
1
log(|x − y|)u(y)dy = f (x), x ∈ [0, 1],
0
where f : [0, 1] → R is given. Note that the integral on the left-hand side
contains the kernel K(x, y) := log(|x − y|), which is singular on the diagonal
x = y.
To solve this integral equation numerically we will use a Galerkin approxi-
mation. The idea here is to choose an approximate solution un from a fixed,
finite-dimensional subspace V = span{φ1 , . . . , φn } and to test the approximate
solution via
1 1 1
log(|x − y|)un (y)dy φi (x)dx = f (x)φi (x)dx, 1 ≤ i ≤ n. (1.3)
0 0 0

Since we choose un ∈ V it must have a representation un = nj=1 c j φ j with
certain coefficients c j . Inserting this representation into (1.3) and changing the
order of summation and integration yields

n 1 1 1
cj log(|x − y|)φ j (y)φi (x)dy dx = f (x)φi (x)dx, 1 ≤ i ≤ n,
j=1 0 0 0
which we easily identify as a linear system Ac = f with the matrix A having

entries
1 1
ai j = log(|x − y|)φ j (y)φi (x)dy dx, 1 ≤ i, j ≤ n.
0 0
A typical choice for the space V is the space of piece-wise constant functions.
To be more precise, we can choose
⎧
⎪
⎪
⎨1 if n ≤ x < n ,
i−1 i
φi (x) = ⎪
⎪
⎩0 else,
but other basis functions and approximation spaces are possible. But we note
that particularly in this case the matrix A is once again a full matrix as its
entries are given by
i/n j/n
ai j = log(|x − y|)dy dx.
(i−1)/n ( j−1)/n

02
10 Introduction
An obvious generalisation of this problem to arbitrary domains Ω ⊆ Rd leads

to matrix entries of the form
ai j = K(x, y)φi (x)φ j (y)dy dx

Ω Ω
with a given kernel K : Ω × Ω → R.
1.2 Notation
Now, it is time to set up the notation which we will use throughout this book.
However, we will specify only the most basic notation and definitions here and
introduce further concepts whenever required.
1.2.1 Mathematics
We will denote the real, complex, natural and integer numbers as usual with
R, C, N and Z, respectively. The natural numbers will not include zero. We
will use the notation x ∈ Rn to denote vectors. The components of x will be
denoted by x j ∈ R, i.e. x = (x1 , . . . , xn )T . Thus vectors will always be column
vectors. We will denote the unit standard basis of Rn by e1 , . . . , en , where the
ith unit vector ei has only zero entries except for a one at position i. In general,
we will suppress the dimension n when it comes to this basis and we might use
the same notation to denote the ith unit vector for Rn and, say, Rm . It should
be clear from the context which one is meant. On Rn we will denote the inner
product between two vectors x and y by either xT y or x, y 2 , i.e.

n
xT y = x, y 2 = x jy j.
j=1
For a matrix A with m rows, n columns and real entries we will write A ∈ Rm×n
and A = (ai j ), where the index i refers to the rows and the index j refers to the
columns:
⎛ ⎞
⎜⎜⎜ a11 a12 · · · a1n ⎟⎟⎟
⎜⎜⎜ ⎟
⎜⎜⎜ a21 a22 · · · a2n ⎟⎟⎟⎟⎟
A := (ai j ) := ⎜⎜⎜ . ⎜ .. ⎟⎟⎟⎟ .
⎜⎜⎜ .. . ⎟⎟⎟⎟
⎜⎝ ⎠
am1 am2 · · · amn
For a non-square matrix A ∈ Rm×n , we can write ai j = eTi Ae j , where the first
unit vector is from Rm while the second unit vector is from Rn .

02
1.2 Notation 11
We will use the Kronecker δ-symbol δi j , which is defined as

⎧
⎪
⎪
⎨1 if i = j,
δi j = ⎪
⎪
⎩0 if i j.
For the identity matrix in Rn we will use the symbol I ∈ Rn×n . We obviously
have I = (δi j )1≤i, j≤n . Again, as in the case of the unit vectors, we will usually
refrain from explicitly indicating the dimension n of the underlying space Rn .
We will also denote the columns of a matrix A ∈ Rm×n by a j := Ae j ∈ Rm ,
1 ≤ j ≤ n. Hence, we have
A = (a1 , a2 , . . . , an ).

For a matrix A ∈ Rm×n and a vector x ∈ Rn , we can write x = j x j e j and
hence
n
Ax = x ja j.
j=1
We will encounter specific forms of matrices and want to use the following,
well-known names.
Definition 1.1 A matrix A = (ai j ) ∈ Rm×n is
• a square matrix, if m = n,
• a diagonal matrix, if ai j = 0 for i j,
• an upper triangular matrix, if ai j = 0 for i > j,
• a lower triangular matrix, if ai j = 0 for i < j,
• a band-matrix, if there are k, ∈ N0 such that ai j = 0 if j < i − k or j > i + ,
• sparse, if more than half of the entries are zero,
• dense or full, if it is not sparse.
In the case of a diagonal matrix A with diagonal entries aii = λi , we will also
use the notation
A = diag(λ1 , . . . , λn ).
In particular, we have for the identity matrix I = diag(1, . . . , 1) ∈ Rn×n .
Most of these names are self-explanatory. In the case of a band matrix, we have
all entries zero outside a diagonally bordered band. Only those entries ai j with
indices i − k ≤ j ≤ i + may be different from zero. This means we have at
most k sub-diagonals and super-diagonals with non-zero entries. The most
prominent example is given by k = = 1, which has one super-diagonal and
one sub-diagonal of non-zero entries and is hence called a tridiagonal matrix.

02
12 Introduction
Schematically, upper triangular, lower triangular and tridiagonal matrices look

as follows:
⎛ ⎞ ⎛ ⎞ ⎛ ⎞
⎜⎜⎜∗ ∗ ∗ · · · ∗⎟⎟⎟ ⎜⎜⎜∗ 0⎟⎟⎟ ⎜⎜⎜∗ ∗ 0 · · · 0⎟⎟
⎟
⎜⎜⎜ ⎜⎜⎜ .. ⎟⎟⎟⎟
⎜⎜⎜ .. ⎟⎟⎟⎟ ⎜⎜⎜
⎜ ∗ ∗
⎟⎟⎟
⎟ ⎜
⎜
∗ ∗ . ⎟⎟⎟⎟ ⎜⎜⎜ ⎟⎟⎟ ⎜⎜⎜∗ ∗ ∗ . ⎟⎟⎟⎟
⎜⎜⎜⎜ ⎟
⎟ ⎜⎜⎜ . . ⎟
⎟
⎟ ⎜
⎜ ⎟⎟⎟
⎜⎜⎜ .. .. ⎟⎟⎟ , ⎜⎜⎜∗ . . . . ⎟⎟⎟ , ⎜⎜⎜ ⎜ .. .. .. ⎟⎟⎟ ,
⎜⎜⎜⎜ . . ∗⎟⎟⎟ ⎜⎜⎜⎜ . . . .
⎟ ⎟⎟⎟⎟ ⎜⎜⎜⎜ 0 0 ⎟⎟⎟⎟
⎜⎜⎜ ∗ ∗ ⎟⎟⎟ ⎜⎜⎜ .. ⎟⎟⎟ ⎜⎜⎜ .. ⎟⎟
⎜⎜⎝ ⎟⎟⎠ ⎜⎜⎝ ∗ ∗ ⎟⎟⎠ ⎜⎜⎜ . ∗ ∗ ∗⎟⎟⎟⎟
0 ∗ ∗ ··· ∗ ∗ ∗ ⎝ ⎠
0 ··· 0 ∗ ∗
where a ∗ marks a possible non-zero entry.
For a matrix A = (ai j ) ∈ Rm×n we denote the transpose of A by AT . It is given by
exchanging columns and rows from A, i.e. AT = (a ji ) ∈ Rn×m . A square matrix
A ∈ Rn×n is said to be symmetric if AT = A. If the matrix A is invertible, we
have (A−1 )T = (AT )−1 which we will simply denote with A−T . If A is symmetric
and invertible then also the inverse A−1 is symmetric. Both the transpose and
the inverse satisfy the rules
(AB)T = BT AT , (AB)−1 = B−1 A−1 ,
as long as these operations are well-defined.
1.2.2 Algorithms
We will not use a specific computing language to describe the algorithms in
this book. However, we will assume that the reader is familiar with basic pro-
gramming techniques. In particular, we expect the reader to know what a for
and a while loop are. We will use if, then and else in the usual way and, when
assigning a new value to a variable x, this variable might appear also on the
right-hand side of the assignment, i.e. such an assignment can, for example, be
of the form x := x + y, which means that x and y are first evaluated and the sum
of their values is then assigned to x. For each algorithm we will declare the es-
sential input data and the output data. There will, however, be no explicit return
statement. Each algorithm should also have a deterministic stopping criterion.
To demonstrate this, a first example of an algorithm is given in Algorithm 1,
which computes the inner product s := xT y of the two vectors x, y ∈ Rn .
At the beginning, we will formulate algorithms very close to actual programs
using only basic operations. An implementation within a modern computing
language should be straightforward. Later on, when it comes to more sophisti-
cated algorithms, we will use higher-level mathematical notation to compress
the representation of the algorithm. For example, an inner product will then
only appear as s := xT y.

02
Algorithm 1: Inner product

Input : x, y ∈ Rn .
Output: s = xT y.
1 s := 0
2 for j = 1 to n do
3 s := s + x j y j
In this book, we will not discuss low-level data structures, i.e. ways of storing a
vector or a matrix within a computer. We will assume that the reader is familiar
with the concepts of arrays, which are usually used to store (full) matrices, and
index lists, which can be used to store sparse matrices.
1.3 Landau Symbols and Computational Cost

Before developing algorithms to solve linear equations, we will introduce con-
cepts to analyse the cost of such algorithms and their stability. Though, of
course, it is possible to compare the actual run-times of two algorithms on a
computer, the actual run-time is not a particularly good measure. It is more
important to understand how the computational cost of an algorithm changes
with the number of unknowns.
The multiplication of a matrix A ∈ Rm×n with a vector x ∈ Rn results in a vector
b = Ax ∈ Rm with components

n
bi = (Ax)i = ai j x j , 1 ≤ i ≤ m.
j=1
The multiplication of a matrix A ∈ Rm×n with a matrix B ∈ Rn×p gives a matrix

C = AB ∈ Rm×p , with entries

n
ci j = aik bk j , 1 ≤ i ≤ m, 1 ≤ j ≤ p.
k=1
So, how much does the computation of it cost? Usually, the cost is measured
in flops, which stands for floating point operations. A floating point operation
consists of one addition plus one multiplication. Sometimes, it is helpful to
distinguish between additions and multiplications and count them separately.
This was particularly true when a multiplication on a computer was substan-
tially more expensive than an addition. However, as this is no longer the case

02
14 Introduction
on modern computers, where multiplications are realised as efficiently as addi-

tions, we will stick to the above definition of flops. It is nonetheless important
to note that while subtractions are as efficient as additions and multiplications,
this is not true for divisions, which are significantly slower.
Most of the time, we will not be interested in the actual number of floating
point operations but rather in the asymptotic behaviour with respect to the di-
mension.
For example, if we look at a matrix–vector multiplication b = Ax, then we
must for every index i compute the sum

n
ai j x j ,
j=1
which means we have n multiplications and n − 1 additions, i.e. n floating point

operations. Hence, if we double the size of the matrix, we would require twice
as many flops for each component. This, however, would also be true if the
actual computing cost would be cn with a positive constant c > 0. The total
cost of the matrix–vector multiplication becomes mn since we have m entries
to compute.
If we are not interested in the constant c > 0 then we will use the following
notation.
Definition 1.2 (Landau notation) For two functions f, g : Nd → R, we will

write
f (n) = O(g(n))
if there is a constant c > 0 such that
| f (n)| ≤ c|g(n)|, n ∈ Nd .
It is important to see that the constant has to be independent of the argument

n ∈ Nd . Moreover, though in most cases we will ignore the constant c in our
considerations, a huge c > 0 can mean that we will never or only for very large
n see the actual asymptotic behaviour.
With this definition, we can say that matrix–vector multiplication of a matrix
A ∈ Rm×n with a vector x ∈ Rn costs
time(Ax) = O(mn).
In the case of a square matrix m = n, the cost is therefore O(n2 ), which means
that doubling the input size of the matrix and the vector, i.e. replacing n by 2n,
will require four times the time of the original matrix–vector multiplication.

02
We can also use this notation to analyse the space required to store the infor-
mation on a computer. We will have to store each matrix entry ai j and each
entry of x as well as the result Ax. This requires
O(mn + n + m) = O(mn)
space. When developing algorithms it is important to consider both resources,

time and space, and it might sometimes be necessary to sacrifice something of
one resource to gain in the other.
It is now easy to see that for the matrix–matrix multiplication C = AB, we
would require O(mnp) operations. Hence, for square systems with m = n = p
the time is O(n3 ) and doubling the input size results in computations that are
eight times longer.
Let us summarise our findings so far, with some obvious additions.
Lemma 1.3 Let A ∈ Rm×n , B ∈ Rn×p , x ∈ Rn and α ∈ R.
• It costs O(n) space to store the vector x ∈ Rn and O(mn) space to store the
matrix A.
• It costs O(n) time to compute the product αx.
• It costs O(mn) time to compute the product Ax.
• It costs O(mnp) time to compute the product AB.
The cost is called linear if it is O(n), quadratic if it is O(n2 ) and cubic if it is

O(n3 ).
More sophisticated matrix–matrix products have been developed with the goal
of reducing the computational cost by reducing the number of multiplications
at the cost of a mild increase in the number of additions. The most famous
one is a recursive algorithm by Strassen (see [120]) which can compute the
product of two n × n matrices using at most 4.7 · nlog2 7 = O(n2.807355 ) flops.
Since then, other such algorithms have been introduced, most notably one by
Coppersmith and Winograd in [37] which reduces the cost to O(n2.375477 ). The
latter algorithm is, however, more complicated and the constant hidden in the
O-notation substantially larger so that the algorithm is not used in practice.
Though superior in this context, even the Strassen algorithm is not seriously
used in practical applications, which is due to the fact that it is not as stable as
the conventional scheme, see Higham [84].
Finally, let us see how additional information can be used to reduce the cost.
If, for example, we have a tridiagonal matrix, i.e. a matrix which has only non-
zero entries on the diagonal and the sub- and super-diagonal, i.e. which is of

02
16 Introduction
the form
⎛ ⎞
⎜⎜⎜a11 a12 0 ··· 0 ⎟⎟⎟
⎜⎜⎜ .. ⎟⎟⎟
⎜⎜⎜a . ⎟⎟⎟
⎜⎜⎜⎜ 21 a22 a23 ⎟⎟⎟
⎟⎟
A = ⎜⎜⎜⎜⎜ 0 ..
.
..
.
..
. 0 ⎟⎟⎟⎟⎟ ,
⎜⎜⎜ ⎟⎟⎟
⎜⎜⎜ .. ⎟
⎜⎜⎜ . an−1,n−2 an−1,n−1 an−1,n ⎟⎟⎟⎟
⎝ ⎠
0 ··· 0 an,n−1 ann
then a matrix–vector multiplication reduces to

n
(Ax)i = ai j x j = ai,i−1 xi−1 + aii xi + ai,i+1 xi+1 .
j=1
Hence, the time for the full matrix–vector product is now only O(n) instead
of O(n2 ). Also the matrix can be stored by exploiting its special form in O(n)
space instead of O(n2 ).
We will sometimes encounter algorithms which have a recursive structure. This
means, for example, that solving a problem with problem size n is reduced to
solving problems with problem size n/b, where b > 1. The Strassen algorithm
mentioned above is one such example. To analyse the cost of such an algorithm
it is often helpful to assume that n = bk with some k ∈ N. The following result
will be helpful.
Lemma 1.4 The recursion T (1) = c and T (n) = aT (n/b)+cnα with c, a, α > 0
and b > 1 satisfies
⎧ bα α
⎪
⎪
⎪ c bα −a n if a < bα ,
⎪
⎪
⎨ α
T (n) ≤ ⎪
⎪ cn (logb n + 1) if a = bα ,
⎪
⎪
⎪
⎩c a α nlogb a if a > bα .a−b
Proof As mentioned above, we will prove this only for n = bk . Though the
results remain true for general n, the proof is more tedious in the general case.
Using induction, it is easy to see that

k
T (bk ) = c ai (bα )k−i .
i=0
Hence, if a < bα , we have

k+1
k
a i α
1 − baα
α b
α
T (b ) = cb
k
= kα
cn < cn .
i=0
bα 1 − bαa
bα − a

02
1.4 Facts from Linear Algebra 17
For a = bα we have
T (bk ) = cnα (k + 1) = cnα (logb n + 1)
and for a > bα we finally use
α k+1
k
bα
i 1− b
a a a
T (bk ) = cak = ca k
≤ cak =c nlogb a ,
i=0
a 1− bα
a
a − bα a − bα
where we have used

log a log a
a =e
k k log a
= exp log b = (bk ) log b = nlogb a .
k

log b
Finally, let us mention that the definition of the Landau symbol O can even be
further generalised in the following sense.
Definition 1.5 For two functions f, g : Rn → R and x0 ∈ Rn , we will write
f (x) = O(g(x)), x → x0 ,
if there is a constant c > 0 and a surrounding U = U(x0 ) ⊆ Rn of x0 such that
| f (x)| ≤ c|g(x)|, x ∈ U.
1.4 Facts from Linear Algebra

In this section, we want to collect further material on matrices and vectors,
which should be known from classical, basic linear algebra courses. The char-
acter of this section is more to remind the reader of the material and to intro-
duce the notation. However, we will also prove some results which are less
familiar.
In Rn we have the canonical inner product defined by x, y 2 := xT y for all
x, y ∈ Rn . It particularly satisfies x, x 2 = x12 + · · · + xn2 > 0 for all x 0.
As mentioned above, we will use both notations x, y 2 and xT y equally. The
√
canonical inner product defines a canonical norm or length x2 := x, x 2 ,
the Euclidean norm. This norm has the usual properties of a norm, which can
more generally be defined for an arbitrary linear space. Of course, we assume
the reader to be familiar with the concept of linear spaces, linear sub-spaces,
linear independent vectors and the dimension of a linear space.
Definition 1.6 Let V be a real (or complex) linear space. A mapping · :
V → [0, ∞) is called a norm on V if it satisfies
1. homogeneity: λx = |λ|x for all x ∈ V and λ ∈ R (or λ ∈ C),

02
18 Introduction
2. definiteness: x = 0 if and only if x = 0,

3. triangle inequality: x + y ≤ x + y for all x, y ∈ V.
The space V with norm · is called a normed space.
In the case of V = Rn and · = ·2 , the first two properties follow immediately
from the definition, while the triangle inequality follows from the Cauchy–
Schwarz inequality
| x, y 2 | ≤ x2 y2
which holds for all x, y ∈ Rn and where equality occurs if and only if y is a
scalar multiple of x. This all remains true, in the more general situation when
the norm is defined by an inner product.
Definition 1.7 Let V be a real linear space. A mapping ·, · : V × V → R is
called an inner product if it is
1. symmetric: x, y = y, x for all x, y ∈ V,
2. linear: αx + βy, z = α x, z + β y, z for all x, y, z ∈ V and α, β ∈ R,
3. definite: x, x > 0 for all x ∈ V \ {0}.
A space V with an inner product ·, · : V × V → R is called a pre-Hilbert
space.
The second property together with the first property indeed guarantees that
the mapping ·, · is bilinear, i.e. it is also linear in the second argument. Each
pre-Hilbert space becomes a normed space upon defining the canonical norm
√
x := x, x .
If V is a complex linear space then an inner product on V is again a mapping
·, · : V × V → C but the first two conditions above have to be modified
appropriately. For example, the first condition becomes x, y = y, x for all
x, y ∈ V, where α is the complex conjugate of α ∈ C. The second property
must now hold for all α, β ∈ C, which means that the inner product is linear
in its first and anti-linear in its second component. The canonical norm is then
defined as before.
Throughout this book, we will mainly be concerned with the space V = Rn and
hence introduce most concepts only for this space. But it is worth noting that
some of them immediately carry over to more general spaces.
As usual, for given x1 , . . . , xn ∈ V, we use the notation span{x1 , . . . , xn } to
denote the linear sub-space of V spanned by these elements, i.e.
⎧ ⎫
⎪
⎪
⎪ n ⎪
⎪
⎪
⎨ ⎬
span{x1 , . . . , xn } = ⎪
⎪ α x : α , . . . , α ∈ R ⎪
⎪ .
⎪
⎩ j=1
j j 1 n
⎪
⎭

02
Another random document with
no related content on Scribd:
—— Council, No. ——,
—— county of ——, N. C.
This is to certify that —— and —— were at the regular meeting of

this council, held on the ——, 185—, duly elected delegates to
represent this council in the next annual meeting of the state council,
to be held in ——, on the 3d Monday in November next. And by
virtue of the authority in me reposed, I do hereby declare the said
—— and —— to be invested with all the rights, powers, and privileges
of the delegates as aforesaid. This being the —— day of ——, 185—,
and the —— year of our national independence.
—— —— President of
—— Council, No. ——
—— —— Secretary.
FORM OF NOTICE
From the Subordinate Council to the State Council, whenever any

Member of a Subordinate Council is expelled.
—— Council, No. ——,

—— county of ——, N. C.
To the President of the State Council of North Carolina:

Sir:—This is to inform you that at a meeting of this council, held on
the —— day of ——, 185—, —— —— was duly expelled from
membership in said council, and thus deprived of all the privileges,
rights, and benefits of this organization.
In accordance with the provisions of the constitution of the state
council, you are hereby duly notified of the same, that you may
officially notify all the subordinate councils of the state to be upon
their guard against the said ——, as one unworthy to associate with
patriotic and good men, and (if expelled for violating his obligation)
as a perjurer to God and his country. The said —— is about —— years
of age, and is by livelihood a ——.
Duly certified, this the —— day of —— 185—, and in the —— year of
our national independence.
—— —— President of
—— Council, No. ——.
—— Secretary.
First Degree Council.
To be admitted to membership in this order, the applicant shall be

—
1st. Proposed and found acceptable.
2nd. Introduced and examined under the guarantee of secrecy.
3rd. Placed under the obligation which the order imposes.
4th. Required to enrol his name and place of residence.
5th. Instructed in the forms and usages and ceremonies of the
order.
6th. Solemnly charged as to the objects to be obtained, and his
duties.
[A recommendation of a candidate to this order shall be received
only from a brother of approved integrity. It shall be accompanied by
minute particulars as to name, age, calling, and residence, and by an
explicit voucher for his qualifications, and a personal pledge for his
fidelity. These particulars shall be recorded by the secretary in a book
kept for that purpose. The recommendation may be referred, and the
ballot taken at such time and in such a manner as the state council
may prescribe; but no communication shall be made to the candidate
until the ballot has been declared in his favor. Candidates shall be
received in the ante-room by the marshal and secretary.]
OUTSIDE.
Marshal.—Do you believe in a Supreme Being, the Creator and

Preserver of the universe?
Ans.—I do.
Marshal.—Before proceeding further, we require a solemn
obligation of secrecy and truth. If you will take such an obligation,
you will lay your right hand upon the Holy Bible and cross.
(When it is known that the applicant is a Protestant, the cross may
be omitted, or affirmation may be allowed.)
OBLIGATION.
You do solemnly swear (or affirm) that you will never reveal
anything said or done in this room, the names of any persons
present, nor the existence of this society, whether found worthy to
proceed or not, and that all your declarations shall be true, so help
you God?
Ans.—“I do.”
Marshal.—Where were you born?
Marshal.—Where is your permanent residence?
(If born out of the jurisdiction of the United States, the answer
shall be written, the candidate dismissed with an admonition of
secrecy, and the brother vouching for him suspended from all the
privileges of the order, unless upon satisfactory proof that he has
been misinformed.)
Marshal.—Are you twenty-one years of age?
Ans.—“I am.”
Marshal.—Were you born of Protestant parents, or were you
reared under Protestant influence?
Ans.—“Yes.”
Marshal.—If married, is your wife a Roman Catholic?
(“No” or “Yes”—the answer to be valued as the Constitution of the
State Council shall provide.)
Marshal.—Are you willing to use your influence and vote only for
native-born American citizens for all offices of honor, trust, or profit
in the gift of the people, to the exclusion of all foreigners and aliens,
and Roman Catholics in particular, and without regard to party
predilections?
Ans.—“I am.”
INSIDE.
(The marshal shall then repair to the council in session, and

present the written list of names, vouchers, and answers to the
president, who shall cause them to be read aloud, and a vote of the
council to be taken on each name, in such manner as prescribed by
its by-laws. If doubts arise in the ante-room, they shall be referred to
the council. If a candidate be dismissed, he shall be admonished to
secrecy. The candidates declared elected shall be conducted to seats
within the council, apart from the brethren. When all are present the
president by one blow of the gavel, shall call to order and say:)
President.—Brother marshal, introduce the candidates to the vice-
president.
Marshal.—Worthy Vice-President, I present to you these
candidates, who have duly answered all questions.
Vice-President, rising in his place.—Gentlemen, it is my office to
welcome you as friends. When you shall have assumed the patriotic
vow by which we are all bound, we will embrace you as brothers. I
am authorized to declare that our obligations enjoin nothing which is
inconsistent with the duty which every good man owes to his Creator,
his country, his family, or himself. We do not compel you, against
your convictions, to act with us in our good work; but should you at
any time wish to withdraw, it will be our duty to grant you a
dismissal in good faith. If satisfied with this assurance, you will rise
upon your feet (pausing till they do so), place the left hand upon the
breast, and raise the right hand towards heaven.
(The brethren to remain seated till called up.)
OBLIGATION.
In the presence of Almighty God and these witnesses, you do

solemnly promise and swear, that you will never betray any of the
secrets of this society, nor communicate them even to proper
candidates, except within a lawful council of the order; that you
never will permit any of the secrets of this society to be written, or in
any other manner made legible, except for the purpose of official
instruction; that you will not vote, nor give your influence for any
man for any office in the gift of the people, unless he be an American
born citizen, in favor of Americans ruling America, nor if he be a
Roman Catholic; that you will in all political matters, so far as this
order is concerned, comply with the will of the majority, though it
may conflict with your personal preference, so long as it does not
conflict with the Constitution of the United States of America, or that
of the state in which you reside; that you will not, under any
circumstances whatever, knowingly recommend an unworthy person
for initiation, nor suffer it to be done, if in your power to prevent it;
that you will not, under any circumstances, expose the name of any
member of this order, nor reveal the existence of such an association;
that you will answer an imperative notice issued by the proper
authority; obey the command of the state council, president, or his
deputy, while assembled by such notice, and respond to the claim of
a sign or cry of the order, unless it be physically impossible; and that
you will acknowledge the State Council of —— as the legislative head,
the ruling authority, and the supreme tribunal of the order in the
state of ——, acting under the jurisdiction of the National Council of
the United States of North America.
Binding yourself in the penalty of excommunication from the
order, the forfeiture of all intercourse with its members, and being
denounced in all the societies of the same, as a wilful traitor to your
God and your country.
(The president shall call up every person present, by three blows of
the gavel, when the candidates shall all repeat after the vice-
president in concert:)
All this I voluntarily and sincerely promise, with a full
understanding of the solemn sanctions and penalties.
Vice-President.—You have now taken solemn oaths, and made as
sacred promises as man can make, that you will keep all our secrets
inviolate; and we wish you distinctly to understand that he that takes
these oaths and makes these promises, and then violates them,
leaves the foul, the deep and blighting stain of perjury resting on his
soul.
President.—(Having seated all by one blow of the gavel.)—Brother
Instructor, these new brothers having complied with the demand of
the order, are entitled to the secrets and privileges of the same. You
will, therefore, invest them with everything appertaining to the first
degree.
Instructor.—Brothers: the practices and proceedings in our order
are as follows:
We have pass-words necessary to be used to obtain admission to
our councils; forms for our conduct while there; means of
recognizing each other when abroad; means of mutual protection;
and methods for giving notices to members.
At the outer door you will[4] (make any ordinary alarm to attract
the attention of the outside sentinel).
When the wicket is opened you will pronounce the (words—what’s
the pass), in a whisper. The outside sentinel will reply (Give it), when
you will give the term pass-word and be admitted to the ante-room.
You will then proceed to the inner door and give (one rap). When the
wicket is opened, give your name, the number of, and location of
your council, the explanation of the term pass, and the degree pass-
word.
If these be found correct, you will be admitted; if not, your name
will be reported to the vice president, and must be properly vouched
for before you can gain admission to the council. You will then
proceed to the centre of the room and address the (President) with
the countersign, which is performed thus (placing the right hand
diagonally across the mouth). When this salutation is recognized,
you will quietly take your seat.
This sign is peculiar to this degree, and is never to be used outside
the council room, nor during the conferring of this degree. When
retiring, you will address the (Vice-President) in the same manner,
and also give the degree pass-word to the inside sentinel.
The “term pass-word” is (We are).
(The pass-word and explanation is to be established by each State
Council for its respective subordinates.)
The “explanation” of the “term pass,” to be used at the inner door,
is (our country’s hope).
The “degree pass-word” is (Native).
The “traveling pass-word” is (The memory of our pilgrim fathers).
(This word is changed annually by the President of the National
Council of the United States, and is to be made and used only when
the brother is traveling beyond the jurisdiction of his own state,
district, or territory. It and all other pass-words must be
communicated in a whisper, and no brother is entitled to
communicate them to another, without authority from the presiding
officer.)
“The sign of recognition” is (grasping the right lappel of the coat
with the right hand, the fore finger being extended inwards).
The “answer” is given by (a similar action with the left hand).
The “grip” is given by (an ordinary shake of the hand).
The person challenging shall (then draw the fore finger along the
palm of the hand). The answer will be given by (a similar action
forming a link by hooking together the ends of the fore finger);
when the following conversation ensues—the challenging party first
saying (is that yours?). The answer, (it is). Then the response (how
did you get it?), followed by the rejoinder (it is my birthright).
Public notice for a meeting is given by means of a (piece of white
paper the shape of a heart).
(In cities[5] the *** of the *** where the meeting is to be held, will
be written legibly upon the notice; and upon the election day said ***
will denote the *** where your presence is needed. This notice will
never be passed, but will be *** or thrown upon the sidewalk with
a *** in the centre.)
If information is wanting of the object of the gathering, or of the
place, &c., the inquirer will ask of an undoubted brother (where’s
when?) The brother will give the information if possessed of it; if not
it will be yours and his duty to continue the inquiry, and thus
disseminate the call throughout the brotherhood.
If the color of the paper (be red), it will denote actual trouble,
which requires that you come prepared to meet it.
The “cry of distress”—to be used only in time of danger, or where
the American interest requires an immediate assemblage of the
brethren—is (oh, oh, oh). The response is (hio, hio, h-i-o).
The “sign of caution”—to be given when a brother is speaking
unguardedly before a stranger—is (drawing the fore finger and
thumb together across the eyes, the rest of the hand being closed),
which signifies “keep dark.”
Brothers, you are now initiated into and made acquainted with the
work and organization of a council of this degree of the order; and
the marshal will present you to the worthy president for admonition.
President.—It has no doubt, been long apparent to you, brothers,
that foreign influence and Roman Catholicism have been making
steady and alarming progress in our country. You cannot have failed
to observe the significant transition of the foreigner and Romanist
from a character quiet, retiring, and even abject, to one bold,
threatening, turbulent, and despotic in its appearance and
assumptions. You must have become alarmed at the systematic and
rapidly augmenting power of these dangerous and unnatural
elements of our national condition. So it is, brothers, with others
beside yourselves in every state of the Union. A sense of danger has
struck the great heart of the nation. In every city, town, and hamlet,
the danger has been seen and the alarm sounded. And hence true
men have devised this order as a means of disseminating patriotic
principles, of keeping alive the fire of national virtue, of fostering the
national intelligence, and of advancing America and the American
interest on the one side, and on the other of checking the strides of
the foreigner or alien, or thwarting the machinations and subverting
the deadly plans of the papist and Jesuit.
Note.—The President shall impress upon the initiates the
importance of secrecy, the manner of proceeding in recommending
candidates for initiation, and the responsibility of the duties which
they have assumed.
Second Degree Council.
Marshal.—Worthy President: These brothers have been duly

elected to the second degree of this order. I present them to you for
obligation.
President.—Brothers: You will place your left hand upon your right
breast, and extend your right hand towards the flag of our country,
preparatory to obligation. (Each council room should have a neat
American flag festooned over the platform of the President.)
OBLIGATION.
You, and each of you, of your own free will and accord, in the
presence of Almighty God and these witnesses, your left hand resting
upon your right breast, and your right hand extended to the flag of
your country, do solemnly and sincerely swear, that you will not
under any circumstances disclose in any manner, nor suffer it to be
done by others, if in your power to prevent it, the name, signs, pass-
words, or other secrets of this degree, except in open council for the
purpose of instruction; that you will in all things conform to all the
rules and regulations of this order, and to the constitution and by-
laws of this or any other council to which you may be attached, so
long as they do not conflict with the Constitution of the United
States, nor that of the State in which you reside; that you will under
all circumstances, if in your power so to do, attend to all regular signs
or summons that may be thrown or sent to you by a brother of this or
any other degree of this order; that you will support in all political
matters, for all political offices, members of this order in preference
to other persons; that if it may be done legally, you will, when elected
or appointed to any official station conferring on you the power to do
so remove all foreigners, aliens, or Roman Catholics from office or
place, and that you will in no case appoint such to any office or place
in your gift. You do also promise and swear that this and all other
obligations which you have previously taken in this order shall ever
be kept through life sacred and inviolate. All this you promise and
declare, as Americans, to sustain and abide by, without any
hesitation or mental reservation whatever. So help you God and keep
you steadfast.
(Each will answer “I do.”)
President.—Brother Marshal, you will now present the brothers to
the instructor for instructions in the second degree of the order.
Marshal.—Brother Instructor, by direction of our worthy
president, I present these brothers before you that you may instruct
them in the secrets and mysteries of the second degree of the order.
Instructor.—Brothers, in this degree we have an entering sign and
a countersign. At the outer door proceed (as in the first degree). At
the inner door you will make (two raps), and proceed as in the first
degree, giving the second degree pass-word, which is American,
instead of that of the first degree. If found to be correct, you will then
be admitted, and proceed (to the centre of the room), giving the
countersign, which is made thus (extending the right arm to the
national flag over the president, the palm of the hand being
upwards).
The sign of recognition in this degree is the same as in the first
degree, with the addition of (the middle finger), and the response to
be made in a (similar manner).
Marshal, you will now present the brothers to the worthy president
for admonition.
Marshal.—Worthy President, I now present these candidates to
you for admonition.
President.—Brothers, you are now duly initiated into the second
degree of this order. Renewing the congratulations which we
extended to you upon your admission to the first degree, we
admonish you by every tie that may nerve patriots, to aid us in our
efforts to restore the political institutions of our country to their
original purity. Begin with the youth of our land. Instil into their
minds the lessons of our country’s history—the glorious battles and
the brilliant deeds of patriotism of our fathers, through which we
received the inestimable blessings of civil and religious liberty. Point
them to the example of the sages and the statesmen who founded our
government. Implant in their bosoms an ardent love for the Union.
Above all else, keep alive in their bosoms the memory, the maxims,
and the deathless example of our illustrious Washington.
Brothers, recalling to your minds the solemn obligations which
you have severally taken in this and the first degree, I now
pronounce you entitled to all the privileges of membership in this the
second degree of our order.
Third Degree Council.

Marshal.—Worthy President, these brothers having been duly
elected to the third degree of this order, I present them before you
for obligation.
President.—Brothers, you will place yourselves in a circle around
me, each one crossing your arms upon your breasts, and grasping
firmly each other’s hands, holding the right hand of the brother on
the right and the left hand of the brother on the left, so as to form a
circle, symbolical of the links of an unbroken chain, and of a ring
which has no end.
Note.—This degree is to be conferred with the national flag
elevated in the centre of the circle, by the side of the president or
instructor, and not on less than five at any one time, in order to give
it solemnity, and also for the formation of the circle—except in the
first instance of conferring it on the officers of the state and
subordinate councils, that they may be empowered to progress with
the work.
The obligation and charge in this degree may be given by the
president or instructor, as the president may prefer.
OBLIGATION.
You, and each of you, of your own free will and accord, in the
presence of Almighty God and these witnesses, with your hands
joined in token of that fraternal affection which should ever bind
together the States of this Union—forming a ring, in token of your
determination that, so far as your efforts can avail, this Union shall
have no end—do solemnly and sincerely swear [or affirm] that you
will not under any circumstances disclose in any manner, nor suffer
it to be done by others if in your power to prevent it, the name, signs,
pass-words, or other secrets of this degree, except to those to whom
you may prove on trial to be brothers of the same degree, or in open
council, for the purpose of instruction; that you do hereby solemnly
declare your devotion to the Union of these States; that in the
discharge of your duties as American citizens, you will uphold,
maintain, and defend it; that you will discourage and discountenance
any and every attempt, coming from any and every quarter, which
you believe to be designed or calculated to destroy or subvert it, or to
weaken its bonds; and that you will use your influence, so far as in
your power, in endeavoring to procure an amicable and equitable
adjustment of all political discontents or differences which may
threaten its injury or overthrow. You further promise and swear [or
affirm] that you will not vote for any one to fill any office of honor,
profit or trust of a political character, whom you know or believe to
be in favor of a dissolution of the Union of these States, or who is
endeavoring to produce that result; that you will vote for and support
for all political offices, third or union degree members of this order
in preference to all others; that if it may be done consistently with
the constitution and laws of the land, you will, when elected or
appointed to any official station which may confer on you the power
to do so, remove from office or place all persons whom you know or
believe to be in favor of a dissolution of the Union, or who are
endeavoring to produce that result; and that you will in no case
appoint such person to any political office or place whatever. All this
you promise and swear [or affirm] upon your honor as American
citizens and friends of the American Union, to sustain and abide by
without any hesitation or mental reservation whatever. You also
promise and swear [or affirm] that this and all other obligations
which you have previously taken in this order, shall ever be kept
sacred and inviolate. To all this you pledge your lives, your fortunes,
and your sacred honors. So help you God and keep you steadfast.
(Each one shall answer, “I do.”)
President.—Brother Marshal, you will now present the brothers to
the instructor for final instruction in this third degree of the order.
Marshal.—Instructor, by direction of our worthy president, I
present these brothers before you that you may instruct them in the
secrets and mysteries of this the third degree of our order.
Instructor.—Brothers, in this degree as in the second, we have an
entering pass-word, a degree pass-word, and a token of salutation. At
the outer door (make any ordinary alarm. The outside sentinel will
say U; you say ni; the sentinel will rejoin on). This will admit you to
the inner door. At the inner door you will make (three) distinct
(raps), Then announce your name, with the number (or name) and
location of the council to which you belong, giving the explanation to
the pass-word, which is (safe). If found correct, you will then be
admitted, when you will proceed to the centre of the room, and
placing the (hands on the breast with the fingers interlocked), give
the token of salutation, which is (by bowing to the president). You
will then quietly take your seat.
The sign of recognition is made by the same action as in the second
degree, with the addition of (the third finger), and the response is
made by (a similar action with the left hand).
(The grip is given by taking hold of the hand in the usual way, and
then by slipping the finger around on the top of the thumb; then
extending the little finger and pressing the inside of the wrist. The
person challenging shall say, do you know what that is? The answer
is yes. The challenging party shall say, further, what is it? The
answer is, Union.)
[The instructor will here give the grip of this degree, with
explanations, and also the true pass-word of this degree, which is
(Union).]
CHARGE.
To be given by the president.

Brothers, it is with great pleasure that I congratulate you upon
your advancement to the third degree of our order. The
responsibilities you have now assumed, are more serious and
weighty than those which preceded, and are committed to such only
as have been tried and found worthy. Our obligations are intended as
solemn avowals of our duty to the land that gave us birth; to the
memories of our fathers; and to the happiness and welfare of our
children. Consecrating to your country a spirit unselfish and a
fidelity like that which distinguished the patriots of the Revolution,
you have pledged your aid in cementing the bonds of a Union which
we trust will endure for ever. Your deportment since your initiation
has attested your devotion to the principles we desire to establish,
and has inspired a confidence in your patriotism, of which we can
give no higher proof than your reception here.
The dangers which threaten American liberty arise from foes
without and from enemies within. The first degree pointed out the
source and nature of our most imminent peril, and indicated the first
measure of safety. The second degree defined the next means by
which, in coming time, such assaults may be rendered harmless. The
third degree, which you have just received, not only reiterates the
lessons of the other two, but it is intended to avoid and provide for a
more remote, but no less terrible danger, from domestic enemies to
our free institutions.
Our object is briefly this:—to perfect an organization modeled after
that of the Constitution of the United States, and coextensive with
the confederacy. Its object and principles, in all matters of national
concern, to be uniform and identical whilst in all local matters the
component parts shall remain independent and sovereign within
their respective limits.
The great result to be attained—the only one which can secure a
perfect guarantee as to our future—is UNION; permanent, enduring,
fraternal UNION! Allow me, then, to impress upon your minds and
memories the touching sentiments of the Father of his Country, in
his Farewell Address:—
“The unity of government which constitutes you one people,” says
Washington, “is justly dear to you, for it is the main pillar in the
edifice of your real independence, the support of your tranquillity at
home, of your peace abroad, of your safety, your prosperity—even
that liberty you so justly prize.
“* * * It is of infinite moment that you should properly estimate the
immense value of your National Union, to your collective and
individual happiness. You should cherish a cordial, habitual, and
immovable attachment to it; accustoming yourselves to think and
speak of it, as the palladium of your political safety and prosperity;
watching for its preservation with jealous anxiety; discountenancing
whatever may suggest even a suspicion that it can in any event be
abandoned; and indignantly frowning upon the dawning of every
attempt to alienate any portion of our country from the rest, or to
enfeeble the sacred ties which now bind together the various parts.”
Let these words of paternal advice and warning, from the greatest
man that ever lived, sink deep into your hearts. Cherish them, and
teach your children to reverence them, as you cherish and reverence
the memory of Washington himself. The Union of these states is the
great conservator of that liberty so dear to the American heart.
Without it, our greatness as a nation would disappear, and our
boasted self-government prove a signal failure. The very name of
liberty, and the hopes of struggling freedom throughout the world,
must perish in the wreck of this Union. Devote yourselves, then, to
its maintenance, as our fathers did to the cause of independence;
consecrating to its support, as you have sworn to do, your lives, your
fortunes, and your sacred honors.
Brothers: Recalling to your minds the solemn obligations which
you have severally taken in this and the preceding degrees, I now
pronounce you entitled to all the privileges of membership in this
organization, and take pleasure in informing you that you are now
members of the order of (the American Union.)
American, Whig, Republican and Democratic
Nominations of 1856.
The American convention met the next day after the session of the
National Council of the Order, on the 22d February, 1856. It was
composed of 227 delegates; all the States being represented except
Maine, Vermont, Georgia and South Carolina. Hon. Millard Fillmore
was nominated for President, and Andrew J. Donelson for Vice-
President.
The Whig Convention met at Baltimore, September 17, 1856, and
endorsed the nominations made by the American party, and in its
platform declared that “without adopting or referring to the peculiar
doctrines of the party which has already selected Mr. Fillmore as a
candidate” * * * Resolved, that in the present exigency of political
affairs, we are not called upon to discuss the subordinate questions
of the administration in the exercising of the constitutional powers of
the government. It is enough to know that civil war is raging, and
that the Union is in peril; and proclaim the conviction that the
restoration of Mr. Fillmore to the Presidency will furnish the best if
not the only means of restoring peace.
The first National Convention of the new Republican party met at
Philadelphia, June 18, 1856, and nominated John C. Fremont for
President, and William L. Dayton for Vice-President. Since the
previous Presidential election, a new party consisting of the
disaffected former adherents of the other parties—Native and
Independent Democrats, Abolitionists, and Whigs opposed to slavery
—had sprung into existence, and was called by its adherents and
friends, the Republican party.
This convention of delegates assembled in pursuance of a call
addressed to the people of the United States, without regard to past
political differences or divisions, who were opposed to the repeal of
the Missouri Compromise. To the policy of President Pierce’s
administration: To the extension of slavery into free territory: In
favor of the admission of Kansas as a free State: Of restoring the
action of the federal government to the principles of Washington and
Jefferson.
It adopted a platform, consisting of a set of resolutions, the
principal one of which was: “That we deny the authority of Congress,
of a territorial legislature, of any individual, or association of
individuals, to give legal existence to slavery in any territory of the
United States, while the present Constitution shall be maintained.”
And closed with a resolution: “That we invite the approbation and
co-operation of the men of all parties, however different from us in
other respects, in support of the principles herein declared; and
believing that the spirit of our institutions, as well as the Constitution
of our country, guaranties liberty of conscience and equality of rights
among citizens, we oppose all legislation impairing their security.”
The Democratic Convention, met at Cincinnati, in May 1856, and
nominated James Buchanan for President, and John C. Breckenridge
for Vice-President. It adopted a platform which contained the
material portions of all its previous platforms, and also defined its
position on the new issues of the day, and declared (1) that the
revenue to be raised should not exceed the actual necessary expenses
of the government, and for the gradual extinction of the public debt;
(2) that the Constitution does not confer upon the general
government the power to commence and carry on a general system
of internal improvements; (3) for a strict construction of the powers
granted by the Constitution to the federal government; (4) that
Congress has no power to charter a national bank; (5) that Congress
has no power to interfere with slavery in the States and Territories;
the people of which have the exclusive right and power to settle that
question for themselves. (6) Opposition to native Americanism.
At the election which followed, in November, 1856, the Democratic
candidates were elected, though by a popular minority vote, having
received 1,838,160 popular votes, and 174 electoral votes, against
2,215,768 popular votes, and 122 electoral votes for John C.
Fremont, the Republican candidate, and Mr. Fillmore, the Whig and
American candidate.
The aggregate vote cast for Mr. Fillmore, who was the nominee on
both the Whig and American tickets, was 874,534, and his electoral
vote was eight; that of the State of Maryland. This was the last
national election at which the Whigs appeared as a party, under that
name; they having joined with the American and with the
Republican parties, and finally united with the latter after the
downfall and extinction of the former. In the State elections of that
year, (1856) the American party carried Rhode Island and Maryland;
and in the 35th Congress, which met in December, 1857, the party
had 15 to 20 Representatives and five Senators. When the 36th
Congress met, in 1859, it had become almost a border State or
Southern party, having two Senators; one from Kentucky and one
from Maryland; and 23 Representatives, five from Kentucky, seven
from Tennessee, three from Maryland, one from Virginia, four from
North Carolina, two from Georgia, and one from Louisiana. The
American party had none of the elements of persistence. It made
another desperate effort, however, in the next Presidential campaign,
but having failed to carry the South, disappeared finally from
politics.
The new Republican party polled a very large vote—1,341,234 out
of a total vote of 4,053,928—and its candidates received 114 votes out
of 296, in the electoral college; having secured majorities in all the
free States, except Illinois, Indiana, Pennsylvania, New Jersey and
California.
The successful candidate, Mr. James Buchanan, was duly
inaugurated as President of the United States, and entered upon the
discharge of his duties as such, March 4, 1857.
After the election of November, 1856, the Republican Association
of Washington issued an address to the people, in which the results
of the election were examined, and the future policy of the party
stated. It is an interesting paper, as laying the foundation of the
campaign of 1860, which followed, and is here given in full:
“Republican Association of Washington.
Address to the Republicans of the United States.
“Washington, Nov. 27, 1856.
“The Presidential contest is over, and at last we have some materials to enable us
to form a judgment of the results.
“Seldom have two parties emerged from a conflict with less of joy in the victors,
more of hope in the vanquished. The pro-slavery party has elected its Presidential
candidate, only, however, by the votes of a minority, and that of such a character as
to stamp the victory as the offspring of sectionalism and temporary causes. The
Republicans, wherever able to present clearly to the public the real issue of the
canvass—slavery restriction or slavery extension—have carried the people with
them by unprecedented majorities; almost breaking up in some States the
organization of their adversaries. A sudden gathering together of the people,
alarmed at the inroads of the slave power, rather than a well organized party, with
but a few months to attend to the complicated details of party warfare; obstructed
by a secret Order, which had pre-occupied the field, and obtained a strong hold of
the national and religious prejudices of the masses; opposed to an old party,
commencing the canvass with the united support of a powerful section, hardened
by long party drill, accustomed to victory, wielding the whole power of the federal
administration—a party which only four years ago carried all but four of the States,
and a majority of the popular vote—still, under all these adverse circumstances,
they have triumphed in eleven, if not twelve of the free States, pre-eminent for
enterprise and general intelligence, and containing one-half of the whole
population of the country; given to their Presidential candidate nearly three times
as many electoral votes as were cast by the Whig party in 1852; and this day control
the governments of fourteen of the most powerful States of the Union.
“Well may our adversaries tremble in the hour of their victory. ‘The Democratic
and Black Republican parties,’ they say, ‘are nearly balanced in regard to power.
The former was victorious in the recent struggle, but success was hardly won, with
the aid of important accidental advantages. The latter has abated nothing of its
zeal, and has suffered no pause in its preparations for another battle.’
“With such numerical force, such zeal, intelligence, and harmony in counsel;
with so many great States, and more than a million voters rallied to their standard
by the efforts of a few months, why may not the Republicans confidently expect a
victory in the next contest?
“The necessity for their organization still exists in all its force. Mr. Buchanan has
always proved true to the demands of his party. He fully accepted the Cincinnati
platform, and pledged himself to its policy—a policy of filibustering abroad,
propagandism at home. Prominent and controlling among his supporters are men
committed, by word and deed, to that policy; and what is there in his character, his
antecedents, the nature of his northern support, to authorize the expectation that
he will disregard their will? Nothing will be so likely to restrain him and counteract
their extreme measures, as a vigorous and growing Republican organization, as
nothing would be more necessary to save the cause of freedom and the Union,
should he, as we have every reason to believe, continue the pro-slavery policy of
the present incumbent. Let us beware of folding our arms, and waiting to see what
he will do. We know the ambition, the necessities, the schemes of the slave power.
Its policy of extension and aggrandizement and universal empire, is the law of its
being, not an accident—is settled, not fluctuating. Covert or open, moderate or
extreme, according to circumstances, it never changes in spirit or aim. With Mr.
Buchanan, the elect of a party controlled by this policy, administering the
government, the safety of the country and of free institutions must rest in the
organization of the Republican party.
“What, then, is the duty before us? Organization, vigilance, action; action on the
rostrum, through the press, at the ballot-box; in state, county, city, and town
elections; everywhere, at all times; in every election, making Republicanism, or
loyalty to the policy and principles it advocates, the sole political test. No primary
or municipal election should be suffered to go by default. The party that would
succeed nationally must triumph in states—triumph in the state elections, must be
prepared by municipal success.
“Next to the remaining power in the states already under their control, let the
Republicans devote themselves to the work of disseminating their principles, and
initiating the true course of political action in the states which have decided the
election against them. This time we have failed, for reasons nearly all of which may
be removed by proper effort. Many thousand honest, but not well-informed voters,
who supported Mr. Buchanan under the delusive impression that he would favor
the cause of free Kansas will soon learn their mistake, and be anxious to correct it.
The timid policy of the Republicans in New Jersey, Pennsylvania, and Indiana, in
postponing their independent action, and temporizing with a party got up for
purposes not harmonizing with their own, and the conduct of Mr. Fillmore’s
friends in either voting for Mr. Buchanan, or dividing the opposition by a separate
ticket, can hardly be repeated again. The true course of the Republicans is to
organize promptly, boldly, and honestly upon their own principles, so clearly set
forth in the Philadelphia platform, and, avoiding coalitions with other parties,
appeal directly to the masses of all parties to ignore all organizations and issues
which would divert the public mind from the one danger that now threatens the
honor and interests of the country, and the subtlety of the Union—slavery
propagandism allied with disunionism.

Textbook Numerical Linear Algebra An Introduction 1St Edition Holger Wendland Ebook All Chapter PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Textbook Numerical Linear Algebra An Introduction 1St Edition Holger Wendland Ebook All Chapter PDF

Uploaded by

Copyright:

Available Formats

Numerical Linear Algebra An

Introduction 1st Edition Holger

Numerical Linear Algebra A Concise Introduction with

SF1624 Algebra and Geometry Introduction to Linear

Introduction to Linear Algebra for Science and

Linear Algebra Seymour Lipschutz

Introduction to Applied Linear Algebra Vectors Matrices

Basic Linear Algebra Jürgen Müller

Elementary Linear Algebra Ron Larson

Linear Algebra M. Thamban Nair

Numerical Linear Algebra

This self-contained introduction to Numerical Linear Algebra provides a

H O L G E R W E N D L A N D holds the Chair of Applied and Numerical Analysis at the

© in this web service Cambridge University Press www.cambridge.org

Cambridge Texts in Applied Mathematics

Nonlinear Dispersive Waves

Flow, Deformation and Fracture

A Physical Introduction to Suspension Dynamics

Discrete Systems and Integrability

Iterative Methods in Combinatorial Optimization

An Introduction to Polynomial and Semi-Algebraic Optimization

© in this web service Cambridge University Press www.cambridge.org

Numerical Linear Algebra

© in this web service Cambridge University Press www.cambridge.org

University Printing House, Cambridge CB2 8BS, United Kingdom

Cambridge University Press is part of the University of Cambridge.

© in this web service Cambridge University Press www.cambridge.org

PART ONE PRELIMINARIES 1

PART TWO BASIC METHODS 57

© in this web service Cambridge University Press www.cambridge.org

PART THREE ADVANCED METHODS 181

© in this web service Cambridge University Press www.cambridge.org

8.4 Preconditioning Krylov Subspace Methods 357

© in this web service Cambridge University Press www.cambridge.org

Numerical Linear Algebra (NLA) is a subarea of Applied Mathematics. It is

© in this web service Cambridge University Press www.cambridge.org

© in this web service Cambridge University Press www.cambridge.org

13:04:03, subject to the Cambridge

where A ∈ Rn×n is a given invertible matrix, b ∈ Rn is a given vector and x ∈ Rn

08:35:57, subject to the Cambridge

Hence, we have to address questions like: Is a matrix that is invertible in the

08:35:57, subject to the Cambridge

1.1 Examples Leading to Linear Systems

where the coeﬃcients are determined by the interpolation conditions

08:35:57, subject to the Cambridge

This leads to a linear system, which can be written in matrix form as

They obviously have the property L j (x j ) = 1 and L j (xi ) = 0 for j i. Thus,

In higher dimensions, i.e. d ≥ 2, polynomial interpolation can become quite

08:35:57, subject to the Cambridge

where Φ : Rd → R is a ﬁxed function. In most applications, this function is

Gaussian: φ(r) = exp(−r2 ),

where (x)+ is deﬁned to be x if x ≥ 0 and to be 0 if x < 0. The functions are

1.1.2 Boundary Value Problem

−u (x) = f (x), x ∈ (0, 1),

with boundary conditions u(0) = u(1) = 0. If the function f is too complicated

08:35:57, subject to the Cambridge

simplest idea is to approximate the derivative by diﬀerences:

Alternatively this linear system of n equations may be written in the form

08:35:57, subject to the Cambridge

1.1.3 Integral Equations

which we easily identify as a linear system Ac = f with the matrix A having

08:35:57, subject to the Cambridge

An obvious generalisation of this problem to arbitrary domains Ω ⊆ Rd leads

2. deﬁniteness: x = 0 if and only if x = 0,