Advanced Engineering Mathematics

(MEng 6301)

SHIMELIS BEKELE

Addis Ababa Science and Techinology University

Mathematics Department

January 19, 2021

UNIT I: LINEAR ALGEBRA
1.1 Matrices: linear transformations, null and range spaces.
1.2 Linear algebraic equations: existence and uniqueness of
solution,
1.3 Elementary row/column operations, Gauss elimination and
Gauss Jordon methods, Echelon form, pivoting, LU
decomposition and Cholesky method,
1.4 Gauss-Seidel and Jacobi iterative methods,
1.5 condition number, minimum norm and least square error
solutions.
1.6 Eigenvalues and eigenvectors of matrices:
1.7 Similarity transformation and Jordon canonical form,
1.8 Eigenvalues/eigenvectors of symmetric matrices: orthogonal
diagonalization.
1.9 Iterative methods to find eigenvalues/eigenvectors of
symmetric matrices
2 / 77
Linearly Dependent and Independent
Definition
Let v1 , v2 , . . . , vn be elements of an arbitrary vector space V , and
α1 , α2 , . . . , αn be scalars. An expression of the form
α1 v1 + α2 v2 + . . . + αn vn is called linear combinations of the
vectors v1 , v2 , . . . , vn .

Definition:
I A set of vectors {v1 , ..., vn } in <m is said to be linearly
independent if the vector equation

α1 v1 + α2 v1 + ... + αc vn = 0

has only the trivial solution (α1 = ... = αn = 0).

I The set {v1 , ..., vn } is said to be linearly dependent if there
exist weights {α1 , ..., αn }, not all zero, such that

α1 v1 + α2 v1 + ... + αn vn = 0 (1) 3 / 77
 I Equation (1) is called a linear dependence relation among
v1 , ..., vn when the weights are not all zero.
I A set of vectors is linearly dependent if and only if it is not
linearly independent.

Example:Let

     
1 4 2
     
v1 = 2 , v2 = 5 , v3 = 1 ,
   
3 6 0

Determine if the set {v1 , v2 , v3 } is linearly independent or not.

4 / 77
Definitions
Let V be any vector space over a field F, and let the set
S = {v1 , v2 , . . . , vn } be a set of vectors in V.Then:
a) S is said to spans (or generates) V if each element of V is a
linear combinations of elements of S.
b) S is called a basis for V if S is a linearly independent set and it
spans V.

Examples
a) {(1, 0), (0, 1)} is the basis for vectors spaces in <2
b) {e1 , e2 , e3 } is the basis for vectors spaces in <3
c) set S = {(1, 0, 0), (0, 1, 0), (0, 0, 5)} form a basis of the vector
space <3

5 / 77
Linear Transformations

Let X , Y be vector spaces. A function T : X −→ Y is said to be

a linear transformation if

T (ax1 + bx2 ) = aT (x1 ) + bT (x2 )

for all x1 , x2 ∈ X and all a, b ∈ <.

Examples of linear transformations:
1) The identity transformation, that is T : V −→ V , given by
T (v ) = v .
2) A stretching by a ∈ <, that is, T : V −→ V , given by
T (v ) = av
3) A projection, that is T : <3 −→ <3 given by
T (x1 e1 + x2 e2 + x3 e3 ) = x1 e1 + x2 e2 .

6 / 77
Matrix Representations
Definition: Given a linear map T : <n −→ <m , we will say that
an m × n matrix A is a matrix representing the linear
transformation T if the image of a vector x in <n is given by the
matrix vector product T (x) = Ax

Example:
Mapping a vector space from <3 to <2 can be expressed as a 3x2
matrix. Let

L(x1 , x2 , x3 ) = (x1 + 2x2 , 3X2 + 4x3 )

can be written as
 
    x1
y1 1 2 0  
= x1
y2 0 3 4
x3

7 / 77
Example:
Find the matrix representing the linear map M : <2 −→ <2 that
reflects a vector x through the line Span (generate) {e1 + e2 }.
Solution: First, note that
 
1
e1 + e2 =
1

so, Span {e1 + e2 } is the solution set of the linear equation

x1 − x2 = 0, i.e., it is the line x1 = x2 .
Reflection through this line swaps the vectors e1 and e2 , and in
general acts on a vector by swapping its components. So
M(e1 ) = e2 and M(e2 ) = e1 .

Consider an arbitrary vector x = x1 e1 + x2 e2 ,

By linearity

M(x) = M(x1 e1 + x2 e2 ) = x1 M(e1 ) + x2 M(e2 ) = x1 e1 + x2 e2 .

8 / 77
Comparing the image
      
x2 a b x1 ax1 + bx1
M(x) = with =
x1 c d x2 cx1 + dx2

we see that
      
x2 0x1 + 1x1 0 1 x1
= =
x1 1x1 + 0x2 1 0 x2

So we can represent the reflection map x 7−→ M(x) by the

matrix-vector product map
 
1 0  
M(x) = x = e1 e2 x
0 1
9 / 77
Null and range spaces

Range Spaces
The range space of a transformation T : X −→ Y is the set of all
vectors that can be reached by the transformation.

R(T ) = {y = T (x) : x ∈ X }

Null Spaces
The null space of the transformation is the set of all vectors in X
that are transformed to the null vector in Y .

N(T ) = {x ∈ X |T (x) = 0}

10 / 77
Consistency of System of Linear Equations

11 / 77
Consistency of System of Linear Equations

12 / 77
Existence and uniqueness of solution

I Given a system Ax = b, what is the connection between the

existence of a solution x and the linear transformation
T : <n −→ <m given by T (x) = Ax?
I If a solution exists for some b, that means there is some
x ∈ <n such that T (x) = b.
I implies there exists at least one x, a pre-image of b, which
solves the system Ax = b.
I A negative answer implies the system Ax = b is inconsistent.

13 / 77
The pre-image of a vector b ∈ <m under a map T : <n −→ <m by

T −1 (b) = {x ∈ <n |T (x) = b}

I Let T (x) = Ax be a linear transformation. Then a solution to

Ax = b always exists (for any b) if and only if for every
b ∈ <m , the cardinality of the pre-image of b is at least 1:
T −1 (b) ≥ 1.
I The uniqueness question concerns whether there is only one
solution x such that Ax = b, or in infinitely many. This
corresponds to asking whether the pre-image of b contains
more than one element.

14 / 77
I if b = 0, then the system is homogenous. Any nontrivial
solution would imply that the solution is not unique.
I The kernel of the map x 7−→ Ax: is precisely the pre-image of
the zero vector. For any linear map T : <n −→ <m , let

kerT := T −1 (0) = {x ∈ <n |T (x) = 0} .

I The kernel is called trivial if it contains only the zero vector,

i.e.,kerT = {0}.
I Then the homogeneous system T (x) = Ax = 0 has nontrivial
solutions if and only if if the kernel is nontrivial.

15 / 77
Proposition (Unique solutions for n linear equations in n
variables)
In particular, if the row reduced Echelon form of

then the corresponding system of linear equations has solution

x1 = v1 , ..., xn = vn .

16 / 77
Theorem:
A linear transformation T : <n −→ <m is injective iff
ker (T ) = {0}.

Proof:
If T is injective and x ∈ ker (T ), then T (x) = 0 = T (0), so that
x = 0, hence ker (T ) = {0}.
Conversely, if ker (T ) = {0} and T (x) = T (y ), then,
0 = T (x) − T (y ) = T (x − y ) = x − y = 0 or x = y , and so T is
injective.

If T (x) = Ax, the system Ax = b has a unique solution if and only

if T is injective, that is, if and only if the kernel is trivial.

The kernel of T is trivial if and only if the columns of the

representative matrix A are linearly independent.
17 / 77
Elementary row/column operations

Elementary Row Operations

I (Replacement) Add one row to a multiple of another row.
I (Interchange) Interchange two rows.
I (Scaling) Multiply all entries in a row by a nonzero constant.

Row Equivalent Matrices

Two matrices where one matrix can be transformed into the other
matrix by a sequence of elementary row operations.

Fact about Row Equivalence

If the augmented matrices of two linear systems are row
equivalent, then the two systems have the same solution set.

18 / 77
Gauss elimination, Echelon form,

Definition:
A matrix is said to be in row echelon form if it satisfies the
following two conditions:
a) A ll zero rows are gathered near the bottom.
b) The first nonzero entry of a row, called the leading entry of
that row, is ahead of the first nonzero entry of the next row.
A matrix in row echelon form is said to be in reduced row echelon
form if it satisfies two more conditions:
c) The leading entry of every nonzero row is 1.
d) Each leading entry 1 is the only nonzero entry in its column.
A matrix in (reduced) row echelon form is called a (reduced)
row echelon matrix.

19 / 77
Example: (Row Eliminations to a Triangular Form)

20 / 77
Thus, the solution is (29, 16, 3)

21 / 77
Gauss Jordon methods

I This is the modified of Guss elimination.

I When an unknown is eliminated, it eliminates from all
equations.
I The method does not required backward substitution.

Example
Solve the system using Gauss Jordn method

22 / 77
Solution:

23 / 77
Pivoting,
Definition:
A pivot position of a matrix A is a location of entries of A that
corresponds to a leading entry in a row echelon form of A. A pivot
column (pivot row) is a column (row) of A that contains a pivot
position.

(Row Reduction Algorithm).

(1) Begin with the leftmost nonzero column, which is a pivot
column; the top entry is a pivot position.
(2) If the entry of the pivot position is zero, select a nonzero
entry in the pivot column, interchange the pivot row and the
row containing this nonzero entry.
(3) If the pivot position is nonzero, use elementary row operations
to reduce all entries below the pivot position to zero.
(4) Cover the pivot row and the rows above it; repeat (1)-(3) to
the remaining submatrix.
23 / 77
LU decomposition and Cholesky method

LU decomposition
If A can be lower reduced to a row-echelon (hence upper
triangular) matrix U, then we have

A = LU

for some lower triangular, invertible matrix L.

Definition:(LU factorization)
A factorization A = LU is called an LU factorization or LU
decomposition of A.

24 / 77
Example:
Let’s find an LU factorization of

We first lower reduce A to row-echelon form:

25 / 77
Then A = LU with

Once the LU factors for a non-singular matrix An×n have been

obtained, rewriting Ax = b as L(Ux) = b and setting y = Ux, we
see that Ax = b is equivalent to the two triangular systems Ly = b
and Ux = y .
First, the lower-triangular system Ly = b is solved for y by forward
substitution. That is, if

26 / 77
Example:
Use the LU factorization of A to solve Ax = b, where

LU factorization gives:

27 / 77
The strategy is to set Ux = y and solve Ax = L(Ux) = b by
solving the two triangular systems Ly = b and Ux = y .
First solve the lower-triangular system Ly = b by using forward
substitution:

Now use back substitution to solve the upper-triangular system

Ux = y :

28 / 77
Positive Definit
An n × n symmetric matrix, A, is positive semidefinite iff
x T Ax ≥ 0 for all x ∈ <n .
An n × n symmetric matrix, A, is positive definite iff x T Ax > 0 for
all 0 6= x ∈ <n .
If A is symmetric and n × n, then x T Ax is the function

this is called a quadratic form

Equivalently, A is symmetric positive definite iff all its eigenvalues
are strictly positive.

29 / 77
Example
Find the value/s of a such that the matrix A is positive definite
 
9 6
A=
6 a

Solution:
A is positive definite for a > 4.
That is, x T Ax > 0 for all nonzero x
A is positive semidefinite but not positive definite for a = 4
x T Ax ≥ 0 for all x, x T Ax ≥ 0 for x = 2, −3
A is not positive semidefinite for a < 4
x T Ax < 0 for all x, x T Ax ≥ 0 for x = 2, −3

30 / 77
Cholesky Factorization
Let A be a symmetric positive definite matrix. Then, there is some
lower-triangular matrix, B, so that A = RR T . Furthermore, R can
be chosen so that its diagonal elements are strictly positive, in
which case, R is unique.
R is called the Cholesky factor of A
Cholesky factorization algorithm

compute first row of R:

31 / 77
compute 2, 2 block R2:n,2:n from

this is a Cholesky factorization of order n − 1

Example:

32 / 77
Example:

33 / 77
Jacobi iterative methods

34 / 77
35 / 77
36 / 77
37 / 77
38 / 77
Example:
Use the Gauss-Seidel method to obtain the solution of the same
system

The true solution is x1 = 3, x2 = 22.5, and x3 = 7.

Solution:
First, solve each of the equations for its unknown on the diagonal.

39 / 77
By assuming that x2 and x3 are zero,

For the second iteration, the same process is repeated to compute

40 / 77
41 / 77
Norm of a matrix
Definition:
A norm of a matrix A of order n × n, written as kAk, is a single
number. The norm is a function of the elements of A, and the
following relations hold:
1) kAk = 0 and kAk= 0 if and only if A = 0.
2) kcAk = ckAk or any scalar c.
3) kA + Bk ≤ kAk + kBk or matrices A and B.
4) kABk ≤ kAkkBkor matrices A and B.

The following are frequently used matrix norms:

1) kAk∞ = maxi nj=1 |aij |
P

2) kAk1 = maxj ni=1 |aij |

P
√
3) kAk2 = λ∗ , λ∗ = maximum eigenvalue of AT A
where for a symmetric matrix A 42 / 77
we have kAk∞ = kAk1 and kAk2 = max λi . The norm kAk2 is
called the spectral norm of A.

Example:
Calculate the ∞−, 1−, and 2-norms of the matrix A,
 
5 −4 −7
A = −4 2 −4
−7 −4 5

Solution:
kAk∞ = 5 + 4 + 7 = 16,
kAk1 = 5 + 4 + 7 = 16,
The eigenvalues of A are λ1 = 6, λ2 = −6 and λ3 = 12. The
2-norm is equal to |λ3 , and hence kAk2 = 12
43 / 77
Condition Number of a Matrix
A measure of how close a matrix is to singular

or for a nonsingular matrix A,

I cond(I)=1
I cond(singular matrix) = ∞
43 / 77
Condition number properties

44 / 77
Example:
Calculated the condition number for the matrix:
 
1 1/2 1/3
A = 1 2/3 1/2
1 3/4 3/5

Solution:
Summing each of the rows gives 1.833, 2.1667, and 2.35. Thus,
the third row has the largest sum and the row-sum norm is
kAk∞ = 1 + 3/4 + 3/5 = 2.35. The inverse of the matrix A:
 
9 −18 10
A−1 = −36 96 60
30 −90 60

Then, kA−1 k∞ = | − 36| + |96| + |60| = 192

Thus, the condition number can be calculated as
Cond A= 2.35(192) = 451.2 45 / 77
Least square error solutions
. suppose m × n matrix A is tall, so Ax = b is over-determined
. for most choices of b, there is no x that satisfies Ax = b
. residual is r = Ax − b
. least squares problem: choose x to minimize kAx − bk2
. kAx − bk2 is the objective function
. x ∗ is a solution of least squares problem if

kAx ∗ − bk2 ≤ kAx − bk2

for any n-vector x

. idea: x ∗ makes residual as small as possible, if not 0
. also called regression (in data fitting context)
. x ∗ called least squares approximate solution of Ax = b
. x ∗ is sometimes called ‘solution of Ax = b in the least squares
sense’
46 / 77
Example:

47 / 77
Eigenvalues/eigenvectors of symmetric matrices:
orthogonal diagonalization
Consider the problem of solving the system of two first-order linear
differential equations,
du1
= 7u1 − 4u2
dt
du2
= 5u1 − 2u2
dt
In matrix notation, this system is:
 0   
u1 4 −4 u1 0
0 = or, equivalently, u = Au (2)
u2 5 −2 u2

 0    
0u1 4 −4 u
where,U = 0 , A = and u = 1
u2 5 −2 u2

48 / 77
 0
Because solutions of a single equation u = λu have the form
u = αe λt , we are motivated to seek solution of (2) that also have
the form

u1 = α1 e λt and u2 = α2 e λt (3)

Differentiating these two expressions and substituting the results in

(2) yields

α1 λe λt =7α1 e λt − 4α2 e λt
α2 λe λt =5α1 e λt − 2α2 e λt

=⇒ α1 λ =7α1 − 4α2
α2 λ =5α1 − 2α2

    
7 −4 α1 α
=⇒ =λ 1
5 −2 α2 α2

49 / 77
Eigenvalue Formulation

50 / 77
51 / 77
Examples:

52 / 77
I What we really need are scalars λ and nonzero vectors x that
satisfy Ax = λx. Writing Ax = λx as (A − λI )x = 0 shows
that the vectors of interest are the nonzero vectors in
N(A − λI ). But N(A − λI ) contains nonzero vectors if and
only if A − λI is singular.
I For an n × n matrix A, scalars λ and vectors xn×1 6= 0
satisfying Ax = λx are called eigenvalues and eigenvectors of
A, respectively, and any such pair, (λ, x), is called an
eigenpair for A. The set of distinct eigenvalues, denoted by
σ(A), is called the spectrum of A.
a) λ ∈ σ(A) ⇐⇒ A − λI is singular ⇐⇒ det(A − λI ) = 0.
b) {x 6= 0|x ∈ N(A − λI )} is the set of all eigenvectors
associated with λ. From now on, N(A − λI ) is called an
eigenspace for A.

53 / 77
Proposition:
Suppose the polynomial p(λ) = λn + ... + b has integer coefficients
and leading coefficient 1. Then any rational number that is a root
of p(λ) must be an integer that divides b.

Proposition: (Eigenvalues of Triangular Matrix):

The eigenvalues of an upper-triangular or lower-triangular matrix
are its diagonal entries.
If the characteristic polynomial has a repeated root, then the
associated eigenvalue has ”multiplicity”

54 / 77
Example:
 
1 −1 0
Find the eigenvalues of A = 1 3 0
0 0 0
 
1 − λ −1 0 
 
Solution: det(A − λI ) =  1 3 − λ 0  = λ(λ − 2)2 = 0
 0 0 λ
Thus, the characteristic polynomial has a single root λ = 0 and a
double root λ = 2, so A has an eigenvalue 0 of multiplicity 1 and
an eigenvalue 2 of multiplicity 2. As a list, the eigenvalues are λ=
0, 2, 2 .

Example:
 
1 1 0
Find the eigenvalues of A = 0 1 1
0 0 1
Solution: Since A is upper-triangular, the eigenvalues are the
diagonal entries, so A has an eigenvalue 1 of multiplicity 3. As a
list, the eigenvalues are λ = 1, 1, 1 .
55 / 77
Example:
 
1 1
Find the eigenvalues of A =
−2 3
 
1 − λ 1 
Solution: det(A − λI ) =  = λ2 − 4λ + 5 = 0
−2 3 − λ
Thus, the eigenvalues are 2 ± i

Eigenspaces
Proposition (Eigenspaces): If T : V −→ V is linear, then for any
fixed value of λ, the set Eλ of vectors in V satisfying T (v ) = λv is
a subspace of V . This space Eλ is called the eigenspace
associated to the eigenvalue λ, or more simply the λ-eigenspace.

Example:
 
1 0
Find the 1-eigenspaces, and their dimensions, for A = and
0 1
 
1 1
B=
0 1
56 / 77
Solution:
I 
For the
 1-eigenspace
   of A, we want to find all vectors with
1 0 a a
= and has dimension 2.
0 1 b b
I 
For the
 1-eigenspace
   of B, we want to find  allvectors
 with
1 1 a a a+b a
= or equivalently, =
0 1 b b b b
The vectors satisfying the equation are those with b = 0, so
a
the 1-eigenspace of B is the set of vectors of the form
0
and has dimension 1.

57 / 77
Basis for the λ-eigenspace
In the finite-dimensional case,
I computing a basis for the λ-eigenspace is equivalent to solving
the system (λI − A)v = 0,
I can do by row-reducing the matrix (λI − A)

Example:
Find all eigenvalues,
 and a basis for each eigenspace, for the matrix
2 2
A=
3 1

Solution:
P(λ) = det(λI − A) = λ2 − 3λ − 4. Thus, the eigenvalues are
λ = −1, 4

58 / 77
I For
 λ = −1, we want
 to find the nullspace of
−1 − 2 −2 −3 −2
=
−3 −1 − 1 −3 −2
 
−3 −2
By row-reducing we find the row-echelon form is
0 0
so
  the (-1)-eigenspace is 1-dimensional and is spanned by
−3
2
I For
 λ = 4, we want to find the nullspace of
4 − 2 −2 2 −2
=
−3 4 − 1 −3 3
 
1 −1
By row-reducing we find the row-echelon form is
0 0
 
1
so the (4)-eigenspace is 1-dimensional and is spanned by
1

59 / 77
Orthonormal vectors
set of n-vectors a1 , ..., an are:
a) (mutually) orthogonal if ai ⊥aj for i 6= j
b) they are normalized if |ai | = 1 for i = 1, ..., n
c) they are orthonormal if both hold

Similarity of matrices
I Two square matrices A and B are similar if there is an
invertible matrix P such that P −1 AP = B.
I If A and B are square matrices, then we say that A and B are
orthogonally similar if there is an orthogonal matrix P such
that P T AP = B.

60 / 77
Similarity transformation

61 / 77
62 / 77
Matrix Diagonalization

Definition:
A matrix A is said to be diagonalizable if there exists a non-
singular matrix P such that P −1 AP is a diagonal matrix.

Remark
Let A be an n × n diagonalizable matrix with eigenvalues
λ1 , λ2 , ...λn . By definition, A is similar to a diagonal matrix,
D = diag (λ1 , λ2 , ...λn ) as similar matrices have the same set of
eigenvalues and the eigenvalues of a diagonal matrix are its
diagonal entries.

63 / 77
Multiplicity and Diagonalizability
I Multiplicity is number of times root appears when polynomial
is written as product of linear factors
I Simple eigenvalue has multiplicity 1
I Defective matrix has eigenvalue of multiplicity k > 1 with
fewer than k linearly independent corresponding eigenvectors
I Nondefective matrix A has n linearly independent
eigenvectors, so it is diagonalizable

P −1 AP = D

where P is nonsingular matrix of eigenvectors

64 / 77
Orthogonally Diagonalizable
An n × n matrix A is said to be orthogonally diagonalizable if there
are an orthogonal matrix P (with P −1 = P T ) and a diagonal
matrix D such that

A = PDP T = PDP −1 (4)

Such a diagonalization requires n linearly independent and

orthonormal eigenvectors.
If A is orthogonally diagonalizable as in 4, then

(A)T = (PDP T )T = P TT D T P T = PDP T = A (5)

Thus A is symmetric.

Theorem:
a) A symmetric matrix is a matrix A such that AT = A.
b) If A is symmetric, then any two eigenvectors from different
eigenspaces are orthogonal. 65 / 77
 Orthogonally Diagonalizing Process
we obtain the following procedure for orthogonally diagonalizing a
symmetric matrix A.
Step 1. Find a basis for each eigenspace of A.
Step 2. Apply the Gram-Schmidt process to each of these bases to
obtain an orthonormal basis for each eigenspace.
Step 3. Form the matrix P whose columns are the basis vectors
constructed in Step2, this matrix orthogonally diagonalizes A.

66 / 77
67 / 77
Theorem:
An n × n matrix A is orthogonally diagonalizable if and only if A is
symmetric matrix.

Example:
Orthogonally diagonalize the matrix

Solution: The characteristic equation is

Bases for the eigenspaces:

68 / 77
But v1 and v2 are not orthogonal to each other. Use Gram
Schmidt to make the two vectors orthogonal.The component (the
projection v2 onto v1 ) of v2 orthogonal to v1 is

   1 
1
v2 .v1 −1/2    4 
projvv12 = v1 = 0 = 0
v1 .v1 2
1 − 14
 1    1
−2 1 −
v2 .v1 −1/2    4 
z2 = v2 − v1 = 1 −
  0 = 1
v1 .v1 2 1
0 1 4

Then {v1 , z2 } is an orthogonal set in the eigenspace for λ = 7.

(Note that z2 is linear combination of the eigenvectors v1 and v2 ,
so z2 is in the eigenspace).
Since the eigenspace is two-dimensional (with basis {v1 , v2 }, the
orthogonal set {v1 , z2 } is an orthogonal basis for the eigenspace
69 / 77
Normalize v1 and z2 to obtain the orthonormal basis for eigenspace
for λ = 7

An orthonormal basis for the eigenspace for λ = −2 is

u3 is orthogonal to the other eigenvectors u1 and u2 .

70 / 77
Hence {u1 , u2 , u3 } is an orthonormal set.

Then P orthogonally diagonalizes A, and A = PDP −1 .

Example
Find an orthogonal matrix P that diagonalizes
 
4 2 2
A = 2 4 2
2 2 4

71 / 77
Solution:
The characteristic equation of A is
 
λ − 4 −2 −2
det(λI − A) =det  −2 λ − 4 −2 
2 2 λ−4
=(λ − 2)2 (λ − 8) = 0

The basis
  of the eigenspace
 corresponding to λ = 2 is
−1 −1
u1 =  1  and u2 =  0 
0 1
Applying the Gram-Schmidt process to {u1 , u2 } yields the following
orthonormal
 eigenvectors:
 1 
1 − √6
− 2√
 √1  − √1 
v1 =  2  and v2 =  6
0 √2
6

72 / 77
  
1
The basis of the eigenspace corresponding to λ = 8 is u3 = 1

1
 1 
√
3
Applying the Gram-Schmidt process to {u3 } yields: v3 =  √13 
 
√1
3
Thus,
 1
− √16 √1

−√
  12 3
− √16 √1 

P = (v1 v2 v3 ) =  √2 3
0 √2 √1
6 3

orthogonally diagonalizes A.

73 / 77
The set of eigenvalues of a matrix A is sometimes called the
spectrum of A, and the following description of the eigenvalues is
called a spectral theorem.

Theorem:
An n × n symmetric matrix A has the following properties:
(a) A has n real eigenvalues, counting multiplicities.
(b) The dimension of the eigenspace for each eigenvalue λ equals
the multiplicity of λ as a root of the characteristic equation.
(c) The eigenspaces are mutually orthogonal, in the sense that
eigenvectors corresponding to different eigenvalues are
orthogonal.
(d) A is orthogonally diagonalizable.

74 / 77
Iterative methods to find eigenvalues/eigenvectors of
symmetric matrices

Assume that the matrix A has a dominant eigenvalue with

corresponding dominant eigenvectors. Then we choose an initial
approximation of one of the dominant eigenvectors of A. This
initial approximation must be a nonzero vector in <n
Finally we form the sequence given by

75 / 77
Example:
a) Find the dominant eigenvalue and corresponding eigenvectors
b) Complete six iterations of the power method to approximate a
dominant eigenvector of the matrix
 
2 −12
A=
1 −5

Solution:
a) the characteristic polynomial of A is Therefore the eigenvalues
of A are λ1 = −1 and λ2 = −2 of which the dominant one is
λ2 = −2 .
The dominant eigenvectors of A (those corresponding to λ2 = −2)
are of the form

76 / 77
  
1
b) We begin with an initial nonzero approximation of x0 =
1
We obtain the following approximations.

77 / 77