Mat2324 1 PDF

MAT5101
Applied Mathematics
Linear Algebra
Applied Mathematics 1/142

Contents I
1 Linear Algebra
Matrices and Linear Systems
Linear Systems
Gauss Elimination
Vector Spaces
Determinants
Cramer’s Rule
Real Inner Product Spaces

Linear Transformations
Eigenvalues, Eigenvectors and Eigenspaces

Diagonalization

Linear Algebra Matrices and Linear Systems
Linear Algebra
Matrices and Linear Systems

• Roughly speaking, a matrix is a rectangular array.
• We shall discuss existence and uniqueness of solutions for systems of
linear equations.
• The method of Gauss elimination will be introduced to solve systems.

We will denote by capital bold letters such as A, B, · · · , or by writing the

general entry in the brackets such as A := (aij ), and so on. By an m × n
matrix (read as m by n), we mean a matrix with m rows and n columns.
It should be noted that rows come always first! m × n is called the size
of the matrix. Explicitly, an m × n matrix is of the form
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A :=  . ..  . (1)
 
.. ..
 .. . . . 
am1 am2 · · · amn

Each entry in (1) has two subscripts. The first one stands for the row
number and the second one stands for the column number. That is, a21
is the entry in the second row and first column.
If m = n, then the matrix A will be called as a square matrix. In this
case, a11 , a22 , · · · , ann are called as the main diagonal entries of the
matrix A.

If either m = 1 or n = 1, then a matrix is called as a vector. We will

denote vectors by lowercase bold letters such as a, b, · · · or by their
general terms in brackets such as a = (ai ), and so on. If m = 1, then the
vector is called a row vector, i.e.,

a := a1 a2 · · · an
while if n = 1, then the vector is called a column vector, i.e.,

 
a1
 a2 
a :=  .  .
 
 .. 
am

Definition 1 (Equality of Matrices)

Two matrices A = (aij ) and B = (bij ) are said to be equal (and written
as A = B) if and only if they have same size and the corresponding
entries are equal, i.e., a11 = b11 , a12 = b12 , and so on. Matrices which
are not equal are called as different. Thus, matrices of different sizes are
always different.

Definition 2 (Addition of Matrices)

The sum of two matrices A = (aij ) and B = (bij ) of the same sizes is
written as A + B and is the matrix A + B = (aij + bij ), i.e., the matrix
obtained by adding the corresponding entries of A and B. Matrices of
different sizes cannot be added.
Example 1
Readily, we compute that

0 1 2 6 5 4 0+6 1+5 2+4
+ =
9 8 7 3 4 5 9+3 8+4 7+5

6 6 6
= .
12 12 12

Definition 3 (Scalar Multiplication)

The product of any m × n matrix A = (aij ) by any scalar α is written as
αA and is the m × n matrix αA = (αaij ), i.e., the matrix whose entries
obtained by multiplying each entry of A by α.
Example 2
Obviously, we compute that

4 5 6 3·4 3·5 3·6
3 =
7 8 9 3·7 3·8 3·9

12 15 18
= .
21 24 27

Property 1 (Rules for Matrix Addition and Scalar Multiplication)

From the familiar laws for the addition of numbers, we obtain the
following similar laws for the addition of matrices of the same size m × n.
1. A + B = B + A.
2. (A + B) + C = A + (B + C ), which is written as A + B + C .
3. A + 0 = A.
4. A + (−A) = 0.
Here, 0 denotes the zero matrix (of the size m × n), i.e., the m × n
matrix whose all entries are zero. Similarly, for scalar multiplication, we
have the following rules.
5. α(A + B) = αA + αB.
6. (α + β)A = αA + βA.
7. α(βA) = (αβ)A, which written as αβA.
8. 1A = A.

Definition 4 (Matrix Multiplication)
The product C = AB (in this order) of an m × n matrix A = (aij ) times

and p × q matrix B = (bjk ) is defined if and only if n = p and is then the
m × q matrix C = (cik ) whose entries are obtained by
n
X
cik := aij bjk , i = 1, 2, · · · , m, k = 1, 2, · · · , q. (2)
j=1

The condition n = p means that the second factor B must have as many
rows as the first factor A has columns. As a diagram of sizes, we can
give the following.
A B = C .
[m×n][n×q] [m×q]
In (2), cik is obtained by multiplying each entry in the i-th row of A by

the corresponding entry in the k-th column of B, and then adding these
n products. For instance, c21 = a21 b11 + · · · + a2n bn1 . One calls this
briefly as “multiplication of rows into columns”.

 
b1k · · · b1q
 
 .. . . . 
. . ..
 
 
× 
 bjk · · · bjq


 
.. .
 
..
. ..
 
× 
 . 

bnk · · · bnq
×
  
  
  
 
 ai1 · · · aij · · · ain cik · · · ciq

 
  
 . .  .. .

 . . . ... . . . ...  ..
. ..

 . 
 . 

am1 · · · amj · · · amn cm1 · · · cmk · · · cmq
Figure 1: An illustration of matrix multiplication.

Example 3
Using Definition 4, we compute that

3 2 2 (−1) 3
4 (−2) 5 3 2

3×2+2×5 3 × (−1) + 2 × 3 3×3+2×2
=
4 × 2 + (−2) × 5 4 × (−1) + (−2) × 3 4 × 3 + (−2) × 2

16 3 13
= .
(−2) (−10) 8
Note that multiplication in the reverse order is not defined.

It should be noted that matrix multiplication is in general not

commutative, i.e., AB 6= BA.
Example 4

1 1 (−1) 1 0 0
=
100 100 1 (−1) 0 0
and

(−1) 1 1 1 99 99
= .
1 (−1) 100 100 (−99) (−99)

Our examples show that order of factors in matrix multiplication must

always be observed very carefully. Otherwise, matrix multiplication
satisfies the following rules similar to those for numbers.
Property 2 (Rules for Matrix Multiplication)

Let α be a scalar while A, B and C be matrices such that the expressions
on the left are defined.
1. (αA)B = α(AB) = A(αB), which is written as αAB or AαB or
ABα.
2. A(BC ) = (AB)C , which is written as ABC .
3. (A + B)C = AC + BC .
4. A(B + C ) = AB + AC .
Second property is called as associate law, third and fourth properties
are called as distributive laws.

Definition 5 (Transposition of a Matrix)

The transpose of an m × n matrix A = (aij ) is the n × m matrix AT
(read as A transpose), which has the first row of A as its first column,
second row of A as its second column, and so on. Thus, the transpose of
A in (1) is AT = (aji ), which is written as
 
a11 a21 · · · am1
 a12 a22 · · · am2 
AT :=  . ..  .
 
.. ..
 .. . . . 
a1n a2n · · · amn
As a particular case, transposition converts row vectors to column vectors

and vice versa.

Property 3 (Rules for Matrix Transposition)

Let α be a scalar and A, B be matrices such that the expressions on the
left are defined.
T
1. AT = A.
2. (A + B)T = AT + B T .
3. (αA)T = α AT .

4. (AB)T = B T AT .

Transposition gives a rise to two useful classes of matrices as follows.
Definition 6 (Symmetric and Skew-Symmetric Matrices)

Let A be a square matrix. A is said to be symmetric matrix if and only
if AT = A while it is said to be skew-symmetric matrix if and only if
AT = −A.
Note that skew-symmetric matrices have zero as their diagonal entries.

Example 5
   
(−2) 3 2 0 (−5) 1
 3 8 (−5)  is symmetric while  5 0 (−3) 
2 (−5) 1 (−1) 3 0
is skew-symmetric.

Any real square matrix A may be written as the sum of a symmetric

matrix and a skew-symmetric matrix. More precisely, we write
1 1
A + AT + A − AT .

A=
|2 {z } |2 {z }
symmetric skew-symmetric
Example 6
Consider that
7
(− 32 )
     
3 2 7 3 2 7 0 0
 5 4 3  =  72 4 9
2
+ 3
2 0 (− 23 )  ,
9 3
7 6 4 7 2 4 0 2 0
where the first and the second matrices on the right-hand side are
symmetric and skew-symmetric, respectively.

Definition 7 (Upper and Lower Triangular Matrices)

Let A = (aij ) be a square matrix. If aij = 0 for i > j, then A is called as
upper triangular matrix. If aij = 0 for i < j, then A is called as lower
triangular matrix.
   
a11 a12 · · · a1n a11

 a22 · · · a2n 

 a a
 21 22


   
.. .  . .. . .
. ..  ..
  



  . . 

ann an1 an2 · · · ann
Figure 2: Upper and lower triangular matrices, respectively. All the entries in
the red triangles are zero.

Definition 8 (Diagonal and Identity Matrices)

Let A = (aij ) be a square matrix. If aij = 0 for i 6= j, then A is called as
diagonal matrix.
In particular, a scalar matrix whose entries on the main diagonal are all 1
is called as the identity matrix (or the unit matrix), and is denoted by
In (or simply I).
   
a11 1
 

 a22 
 
 1 

 
..
 

.
  .. 



  . 
 
ann 1
Figure 3: Diagonal and the identity matrices, respectively. All the entries in the
red triangles are zero.
The diagonal matrix A whose diagonal entries are ak is denoted by

A := diag(ak ).
Definition 9 (Orthogonal Matrices)

Let A be a square matrix. A is said to be orthogonal matrix if and only
if AAT = I.
Example 7
 
2 (−2) 1
1 1 1 1
Show that √ and 1 2 2  are
2 1 (−1) 3
2 1 (−2)
orthogonal matrices.

Definition 10 (Linear Systems)

A linear system of m equations and n unknowns x1 , x2 , · · · , xn is the set
of equations of the form
a11 x1 + a12 x2 + · · · + a1n xn =b1

a21 x1 + a22 x2 + · · · + a2n xn =b2
.. (3)
.
am1 x1 + am2 x2 + · · · + amn xn =bm .
The system is called as linear because x1 , x2 , · · · , xn appear in the first

power only, just as in the equation of a straight line. a11 , a12 , · · · , amn are
given numbers called as coefficients of the system. b1 , b2 , · · · , bm on
the right are also given numbers. If b1 , b2 , · · · , bm = 0, then (3) is called
as homogeneous system, otherwise it is called as non-homogeneous
system.

From the definition of matrix multiplication, we see that the m equations

of (3) may be written as a single vector equation of the form
Ax = b,
where A, x and b are defined as

     
a11 a12 ··· a1n x1 b1
 a21 a22 ··· a2n   x2   b2 
A :=  .  , x :=   and b :=  .
     
.. .. .. .. ..
 .. . . .   .   . 
am1 am2 ··· amn xn bm
We assume that aij are not all zero, so that A is not a zero matrix. Note
that x has n components and b has m components.

The matrix  
a11 a12 ··· a1n b1
 a21 a22 ··· a2n b2 
A
e :=  
 .. .. .. .. .. 
 . . . . . 
am1 am2 ··· amn bm
is called the augmented matrix of the system (3). The vertical line can
be omitted (as we shall do later). It is merely a reminder that the last
column of A e does not belong to A.

Gaussian Elimination
Elementary Row Operations for Matrices

• Interchange of two rows.
• Addition of a constant multiple of one row to another row.
• Multiplication of a row by a non-zero constant.
Elementary Operations for Equations

• Interchange of two equations.
• Addition of a constant multiple of an equation to the others.
• Multiplication of an equation by a non-zero constant.

A rectangular matrix is in row echelon form if it has the following

properties:
1. All nonzero rows are above any rows of all zeros.
2. Each leading entry of a row is in a column to the right of the leading
entry of the row above it.
3. All entries in a column below a leading entry are zeros.
If a matrix in echelon form satisfies the following additional conditions,

then it is in reduced row echelon form.
4. The leading entry in each nonzero row is 1.
5. Each leading 1 is the only nonzero entry in its column.
Note: The leading entry in all the entire rows of a matrix is considered as
the first nonzero entry in that row.

Example: Determine which are of the following are in the reduced row
echelon form, row echelon form, or none.
   
1 2 0 4 1 0 3 4
0 0 0 0 0 2 1 2
0 0 1 2 0 0 1 1
 
  1 2 0 3
1 0 0 0
0 0 1 1
0 1 
0

0 0 1
0 1 0
0 0 0 0

Gaussian Elimination (with Row Echelon Form)
Although a system can be solved rather quickly by noticing its particular

form, Gauss elimination is a systematic approach and will work in
general, also for larger systems. Let us illustrate this method on the
following example.
x1 + 2x2 + x3 = 0 E (1)
2x1 + 2x2 + 3x3 = 3 E (2) (4)
−x1 − 3x2 = 2 E (3)

Step1: Eliminate x1 from E (2) and E (3). Subtract 2 times E (1) from E (2);
and subtract (−1) times E (1) from E (3). This results in the new
system:
x1 + 2x2 + x3 = 0 E (1)
− 2x2 + x3 = 3 E (2)
− x2 + x3 = 2 E (3)
1
Step2: Eliminate x2 , from E (3). Subtract times E (2) from E (3). This
2
yields
x1 + 2x2 + x3 = 0 E (1)
− 2x2 + x3 = 3 E (2) (5)
1 1
x3 = E (3)
2 2

Step3: Solve for x3 , x2 , x1 .
x3 = 1
−2x2 + 1 = 3
x2 = −1
x1 + 2(−1) + 1 = 0
x1 = 1
Steps 1 and 2 are the elimination steps, resulting in (5), which is called
an upper triangular system of linear equations. This system has exactly
the same solutions as (4), but (5) is in a form that is easier to solve.
Step 3 is called solution by back substitution. The entire process is called
Gaussian elimination, and it is generally the most efficient means of
solving a linear system of small to medium size, say, of an order up to
several hundreds.

The elimination steps are more conveniently carried out using the matrix
notation and operations. For this purpose, we form the augmented
matrix for the system (4):
 
1 2 1 0
A | b = 2 2 3 3  (6)
−1 −3 0 2
Notice that eliminating a variable from an equation is equivalent to

making the corresponding coefficient zero through elementary row
operations.
Recall
A | b
is called augmented matrix.

For example, to eliminate x1 from E (2) and E (3), we multiply row 1 by

(−2) and add it to row 2, and then add row 1 to row 3 to obtain
 
1 2 1 0
 0 −2 1 3 
0 −1 1 2
Then, to eliminate x2 from E (3), row 2 is multiplied by (−1/2) and is

added to row 3:  
1 2 1 0
 0 −2 1 3 
1 1
 
0 0
2 2

This matrix is obtained from the matrix (6) through elementary row
operations and we have reduced the original system (6) to the upper
triangular system     
1 2 1 x1 0
0 −2 1  x2  =  3  (7)
1 1
0 0 2
x 3 2
which is the same as (5). Then the backward substitution procedure can
be applied to solve (7).

To see the formal structure of Gaussian elimination, we again look at the

n = 3 case, but with general coefficients. The system to be solved is
a11 x1 + a12 x2 + a13 x3 = b1 E (1)

a21 x1 + a22 x2 + a23 x3 = b2 E (2) (8)
a31 x1 + a32 x2 + a33 x3 = b3 E (3)
Step1: Eliminate x1 from E (2) and E (3). To simplify the presentation,

assume a11 6= 0. Define
a21 a31
m21 = , m31 =
a11 a11
Subtract m21 times E (1) from E (2), and subtract m31 times E (1)
from E (3). This changes (8) to the equivalent system:

a11 x1 + a12 x2 + a13 x3 = b1 E (1)
(2) (2) (2)

a22 x2 + a23 x3 = b2 E (2) (9)
(2) (2) (2)

a32 x2 + a33 x3 = b3 E (3)
(2)
The coefficients aij are defined by
(2)
aij = aij − mi1 a1j , i, j = 2, 3
(2)
bi = bi − mi1 b1 , i = 2, 3

(2)
Step2: Eliminate x2 from E (3). Again assume temporarily that a22 6= 0.
Define
(2)
a
m32 = 32 (2)
a22
Subtract m32 times E (2) from E (3). This yields
a11 x1 + a12 x2 + a13 x3 = b1 E (1)
(2) (2) (2)

a22 x2 + a23 x3 = b2 E (2)
(3) (3)
a33 x3 = b3 E (3)
The new coefficients are defined by

(3) (2) (2) (3) (2) (2)
a33 = a33 − m32 a23 , b3 = b3 − m32 b3

Step3: Use back substitution to solve successively for x3 , x2 , x1 .

(3)
b3
x3 = (3)
a33
(2) (2)
b2 − a23 x3
x2 = (2)
a22
b1 − a12 x2 − a13 x3
x1 =
a11
Note that every step of the elimination procedure can be achieved
through elementary row operations on certain rows of the augmented
matrix.

Generally,
(1) (1) (1)
a11 x1 + ··· + a1n xn = b1 E (1)
.. (10)
.
(1) (1) (1)
an1 x1 + ··· + ann xn = bn E (n)
for k = 1, 2, . . . , n − 1, carry out the following elimination step.

Eliminate xk from E (k + 1) through E (n) The results of the preceding

steps 1, 2, . . . , k − 1 will have yielded a system of the form
(1) (1) (1) (1)
a11 x1 + a12 x2 + ··· + a1n xn = b1 E (1)
(2) (2) (2)

a22 x2 + ··· + a2n xn = b2 E (2)
.. ..
. .
(k) (k) (k)
akk xk + ··· + akn xn = bk E (k)
..
.
(k) (k) (k)
ank xk + ··· + ann xn = bn E (n)

(k)
Assume akk 6= 0, and define multipliers
(k)
aik
mik = (k)
, i = k + 1, . . . , n
akk
For equations i = k + 1, . . . , n, subtract mik times E (k) from E (i),

eliminating xk from E (i). The new coefficients and the right-hand side
numbers in E (k + 1) through E (n) are defined by
(k+1) (k)
aij = aijk − mik akj , i, j = k + 1, . . . , n
(k) (k)
bik+1 = bi − mik bk , i = k + 1, . . . , n

When step n − 1 is completed, the linear system will be upper triangular

form. We will denote it by
u11 x1 + ··· + u1n xn = g1
..
.
unn xn = gn
These coefficients are related to the earlier ones

(i) (i)
uij = aij , gi = bi

Solve using back substitution.

gn
xn =
unn
Pn
gi − j=i+1 uij xj
xi = , i = n − 1, . . . , 1
uii

Example: Solve the system

4x1 + 3x2 + 2x3 + x4 = 1
3x1 + 4x2 + 3x3 + 2x4 = 1
2x1 + 3x2 + 4x3 + 3x4 = −1
x1 + 2x2 + 3x3 + 4x4 = −1

Although a system can be solved rather quickly by noticing its particular

form, Gauss elimination is a systematic approach and will work in
general, also for larger systems. Let us illustrate this method on the
following example.
Example 8
Consider the system
x1 − x2 + x3 =0
−x1 + x2 − x3 =0
10x2 + 25x3 =90
20x1 + 10x2 =80.

The associated augmented matrix is

 
1 (−1) 1 0
 (−1) 1 (−1) 0 
A
e := 
 0
.
10 25 90 
20 10 0 80
Step 1. Elimination of the first unknown. We mark the first row as the
pivot row and its first entry as the pivot. To eliminate x1 in the other
equations, we add 1 times the pivot row to the second row and add
(−20) times the pivot row to the fourth row. This yields
 
1 (−1) 1 0
 0 0 0 0 
 .
 0 10 25 90 
0 30 (−20) 80

Step 2. Elimination of the second unknown. First, push the zero row at
the end of the matrix
 
1 (−1) 1 0
 0 10 25 90 
 .
 0 30 (−20) 80 
0 0 0 0
Then, mark the second row as the pivot row and its first non-zero entry
as the pivot. To eliminate x2 in the other equations, we add (−3) times
the pivot row to the third row. Hence, the result is
 
1 (−1) 1 0
 0 10 25 90 
 .
 0 0 (−95) (−190) 
0 0 0 0

Step 3. Back substitution. Now, working backwards from the last to the
first row of this triangular matrix, we can readily find x3 , x2 and x1 as
follows.
(−190)
x3 = = 2,
−95x3 = − 190 (−95)
10x2 + 25x3 =90 =⇒ 1
x2 = (90 − 25 × 2) = 4,
x1 − x2 + x3 =0 10
x1 =4 − 2 = 2.
Therefore, we have a unique solution.

Definition 11
We now call a linear system S1 row-equivalent to another linear system
S2 if S1 can be obtained from S2 only by applying finitely many row
operations.
Thus, we can state the following theorem, which also justifies the Gauss
elimination.
Theorem 1
Row-equivalent linear systems have the same set of solutions.

At the end of the Gauss elimination the form of the coefficient matrix,
the augmented matrix or the system itself is called row-reduced form.
In it, rows of zeroes, if present, are the last rows, and in each non-zero
row the leftmost non-zero entry is farther to the right than in the
previous row. For instance, (see Example 8) the following coefficient
matrix and its augmented matrix are in the row-reduced form.
   
1 (−1) 1 1 (−1) 1 0
 0 10 25   and  0 10 25 90
 
 .
 0 0 (−95)   0 0 (−95) (−190) 
0 0 0 0 0 0 0
Note that we do not require that the leftmost non-zero entries be 1 since
this has no theoretic or numeric advantage.

Example 9
Transform the following matrices into row-reduced forms.
   
(−1) 4 1 1 (−2) 1 3
 0 0 0 0  and  0 1 1 .
0 0 0 1 2 0 1

Solvability of Linear Systems
A system of linear equations has
• No solution, or
• Exactly one solution, or
• Infinitely many solutions.
A system of linear equations is said to be consistent if it has either one

solution or infinitely many solutions; a system is inconsistent if it has no
solution.

 
 a11 a12 · · · · · · · · · a1n b1 

 a22 · · · · · · · · · a2n b2 

 
 .. .. .. 

 . . . 

a · · · a br
 
 rr rn 
 

 br +1 

..
 
 

 . 

bm
Figure 4: Row reduced form of the augmented matrix at the end of the Gauss
elimination. Here, r ≤ m and a11 , a22 , · · · , arr 6= 0, and all the entries in the red
triangle and the blue rectangle are zero.
From this, we see that with respect to the solutions of the system with
augmented matrix in Figure 4 (and thus, with respect to the originally
given system) there are three possible cases.

Exactly One Solution
If r = n and br +1 , br +2 , · · · , bm = 0 (if present), then we have unique

solution. To get the solution, we solve the n-th equation ann xn = bn , and
then use back substitution.
 
 a11 a12 · · · a1n b1 
a · · · a b
 
 22 2n 2 
 
 .. .. .. 

 . . . 
ann bn 
 

 
 
 
 
 
Figure 5: All the entries in the red triangle and the blue rectangles are zero.

Infinitely Many Solutions

If r < n and br +1 , br +2 , · · · , bm = 0 (if present), then we have infinitely
many solutions. To get the solution, we pick xr +1 , xr +2 , · · · , xn as
arbitrary and solve xr from the r -th equation
arr xr + ar (r +1) xr +1 + · · · + arn xn = br , and then use back substitution.
 
 a11 a12 · · · · · · · · · a1n b1 
a22 · · · · · · · · · a2n b2 
 

 
 .. .. .. 

 . . . 

arr · · · arn br 
 

 
 
 
 
 
Figure 6: All the entries in the red triangle and the blue rectangles are zero.

No Solution
If r < m and at least one br +1 , br +2 , · · · , bm is non-zero, then we have

no solution.
 
a
 11 12 a · · · · · · · · · a1n b 1 

 a22 · · · · · · · · · a2n b 2


 

 . .. .
.. .
..


 
arr · · · arn br 
 

 

 br +1 
..
 
 

 . 

bm
Figure 7: All the entries in the red triangle and the blue rectangle are zero, but
there exists non-zero entries in the orange rectangle.

Example: Solve
x +y +z = 0
2x − y + 3z = 1
x + y − 2z = 1

Example: Solve
x +y +z = 0
2x − y + 3z = 1
3x + 3y + 3z = 1

Example: Solve
x +y +z = 0
2x − y + 3z = 1
3x + 3y + 3z = 0

Example: Find the values of a, b for which the system

2x − y + 2az + t = b
2x + (a − 1)y + 2az + t = 1
2x − y + (2a + 1)z + (a + 1)t = 0
−2x + y + (1 − 2a)z − 2t = −2b − 2
i) has unique solution

ii) has infinitely many solution
iii) has no solution.

Vector Spaces
Vector Spaces
• A quantity such as work, area or energy, which is described in terms of

magnitude only is called a scalar.
• A quantity, which needs both magnitude and direction to be described
is called a vector.
A vector is an element of vector space, which will be introduced below.

Vector Spaces
Definition 12 (Vector Space)

A vector space V over the field R is a set satisfying the following
properties.
1. Closure: u + v ∈ V for all u, v ∈ V .
2. Commutativity: u + v = v + u for all u, v ∈ V .
3. Associativity: u + (v + z) = (u + v ) + z for all u, v , z ∈ V .
4. Identity Element: There exists 0 ∈ V such that u + 0 = u for all
u ∈ V.
5. Inverse Element: There exists (−u) ∈ V such that u + (−u) = 0 for
all u ∈ V .
6. Closure: αu ∈ V for all u ∈ V and all α ∈ R.
7. Distributivity of scalar multiplication: α(u + v ) = αu + αv .
8. Distributivity of scalar multiplication: (α + β)u = αu + βu.
9. Compatibility of scalar multiplication: α(βu) = (αβ)u.
10. Identity Element: 1u = u.

Vector Spaces
Example 10
Some examples of vector spaces are listed below.
1. The singleton {0}.
2. Rn (the set of vectors).
3. Rm×n (the set of m × n matrices).
4. Pn [x] (the set of polynomials).
5. F[a, b] (the set of functions on [a, b]).
6. C1 [a, b] (the set of continuously differentiable functions on [a, b]).

Vector Spaces
Definition 13 (Linear Combination)

Let u 1 , u 2 , · · · , u n ∈ V and α1 , α2 , · · · , αn ∈ R. Then, the vector
α1 u 1 + α2 u 2 + · · · + αn u n
is called as a linear combination of the vectors u 1 , u 2 , · · · , u n .
Definition 14 (Linear Dependence)

Let u 1 , u 2 , · · · , u n ∈ V . If the equation
α1 u 1 + α2 u 2 + · · · + αn u n = 0
implies that α1 , α2 , · · · , αn = 0, then the vectors u 1 , u 2 , · · · , u n are

called as linearly independent. If there exist scalars α1 , α2 , · · · , αn ∈ R
not all zero such that the linear combination satisfies
α1 u 1 + α2 u 2 + · · · + αn u n = 0,
then the vectors u 1 , u 2 , · · · , u n are called as linearly dependent.

Vector Spaces
Example 11
1. Show that (1, 1), (−3, 2) are linearly independent.
2. Show that (1, 1), (−3, 2), (2, 4) are linearly dependent.
10 20 2×
(− −→
3,
2)
16
(2, →
4)
−
1
)×
0
5
(−
−→ )
,1
(1
×
16
-10 0
-10 -3 0 1 2 10 0 10 16 20
Figure 8: The figures show that 16(1, 1) + 2(−3, 2) + (−5)(2, 4) = (0, 0), i.e.,
(1, 1), (−3, 2), (2, 4) are linearly dependent.
3. Show that 1, x, x 2 , (x − 4)2 are linearly dependent.

4. Show that e−t , et , cosh(t) are linearly dependent.

Vector Spaces
Definition 15 (Rank of a Matrix)

The rank of a matrix A is the maximum number of linearly independent
row vectors of A, and it is denoted by rank(A).
Example 12
Find the rank of the matrix
 
3 0 2 2
A :=  (−6) 42 24 54  .
21 (−21) 0 (−15)
Clearly, we have
12(3, 0, 2, 2) + (−1)(−6, 42, 24, 54) + (−2)(21, −21, 0, −15) = (0, 0, 0, 0)
shows that rank(A) ≤ 2. On the other hand, we obtain
α(3, 0, 2, 2) + β(21, −21, 0, −15) = (0, 0, 0, 0) =⇒ α = 0 and β = 0.
Thus, rank(A) = 2.

Vector Spaces
Theorem 2 (Rank of Row-equivalent Matrices)

Row-equivalent matrices have the same rank.
In other words, this theorem allows us to use Gauss elimination to find

the rank. In that case, the number of non-zero rows of the row-reduced
matrix is the rank.
Example 13
   
(−2) 2 1 3 (−2) 2 1 3
 0 0 1 1   0 0 1 1 
  and   are row-equivalent.
 2 (−2) 0 1   0 0 0 3 
2 (−2) 4 8 0 0 0 0
The number of non-zero rows of the last matrix is 3, i.e., its rank is 3.
Thus, the first matrix has the rank 3 too.

Vector Spaces
Theorem 3 (Linear Dependence of a Set of Vectors)

A set of p vectors with n components each are linearly independent if the
matrix with these vectors as row vectors has rank p and linearly
dependent if the rank is less than p.
Example 14
The set{(1, 2, 1), (−2, −3, 1),
 (3, 5, 0)} is linearly dependent since the
1 2 1
matrix  (−2) (−3) 1  has the rank 2, which is less than 3.
3 5 0

Vector Spaces
Theorem 4 (Rank in Terms of Column Vectors)

The rank of a matrix A is equal to the maximum number of linearly
independent column vectors of A.
Thus, A and AT have the same rank.

Vector Spaces
Definition 16 (Spanning of a Set)

Let V be a vector space and S be a subset of V . Span(S) is the set of
all linear combinations of the vectors in S, i.e.,
Span(S) := {α1 u 1 +· · ·+αn u n : u 1 , · · · , u n ∈ S, α1 , · · · , αn ∈ R, n ∈ N}.
Example 15
Show that Span{(1, 1), (−3, 2), (2, 4)} = R2 .

Vector Spaces
Definition 17 (Basis of Vector Spaces)

Let V be a vector space and B be a linearly independent subset of V . B
is a basis of V provided that every element in V can be written as a
linear combination of vectors in B, i.e., for every u ∈ V , there exist
n ∈ N, u 1 , u 2 , · · · , u n ∈ B and α1 , α2 , · · · , αn ∈ R such that
u = α1 u 1 + α2 u 2 + · · · + αn u n . In this case, we write V = Span(B).
Definition 18 (Dimension of Vector Spaces)

Let V be a vector space and B be a basis of V . In this case, dimension
of V is the number of vectors in B and is denoted by dim(V ).
Example 16
Show that {(1, 1), (−3, 2)} is a basis for R2 . Hence, dim(R2 ) = 2.

Vector Spaces
Definition 19 (Dimension of Row-Space and Column-Space of Matrices)

Let A be a matrix. The row-space (the spanning set of the row vectors)
and the column space (the spanning set of the column vectors) of A have
the same dimension, which is equal to the rank of A.
Finally for a given matrix A, the solution set of the homogeneous system
Ax = 0 is a vector space, which is called as the null space of A, and its
dimension is called as the nullity of A.
Theorem 5 (Dimension Theorem)
Let A be a matrix of the size m × n. Then,
rank(A) + nullity(A) = n.

Vector Spaces
Consider the non-homogeneous system
a11 x1 + a12 x2 + · · · + a1n xn =b1

a21 x1 + a22 x2 + · · · + a2n xn =b2
.. (11)
.
am1 x1 + am2 x2 + · · · + amn xn =bm ,
where a11 , a12 , · · · , amn , b1 , b2 , · · · , bm are scalars and x1 , x2 , · · · , xn are

unknowns.

Vector Spaces
Theorem 6 (Fundamental Theorem for Linear Systems)
1. (11) is consistent (i.e., it has solutions) if and only if the coefficient

matrix A and its augmented matrix A e have the same rank.
2. (11) has unique solution if and only if the common rank r of A and A e
is equal to n.
3. (11) has infinitely many solutions if the common rank r is less than n.
In this case, a total of r unknowns can be obtained in terms of (n − r )
unknowns (or parameters).
When (11) admits solutions, one can apply Gauss elimination to find
those solutions.

Vector Spaces
Consider the homogeneous system
a11 x1 + a12 x2 + · · · + a1n xn =0

a21 x1 + a22 x2 + · · · + a2n xn =0
.. (12)
.
am1 x1 + am2 x2 + · · · + amn xn =0,
where a11 , a12 , · · · , amn are scalars and x1 , x2 , · · · , xn are unknowns. Note
that x1 , x2 , · · · , xn = 0 is always a solution, which is called as the trivial
solution.
According to Theorem 6, (12) admits non-trivial solutions if r < n, where
r := rank(A). If r < n, then these solutions, together with x = 0, form a
vector space of dimension (n − r ), which is called as the solution space
of (12).

Vector Spaces
In particular, if x 1 and x 2 are two solutions of (12), then the vector

x := c1 x 1 + c2 x 2 , where c1 and c2 are scalars, is also a solution of (12).
This property does not hold for non-homogeneous systems. It should also
be mentioned that the term “solution space” is only used for
homogeneous systems.

Vector Spaces
The solution space of (12) is also called as the null space of A because
of Ax = 0 for every x in the solution space of (12). Its dimension is
called as the nullity of A. Hence, by Theorem 5, we have
rank(A) + nullity(A) = n,
where n is the number of unknowns (i.e., the number of columns of A).

Vector Spaces
Example 17
Show that the solution space (null space) of
 
x
1 2 3 4   y 
= 0
2 4 7 8  z  0
w
is   

 x 

 y  : x = −2y − 4w and z = 0
  
 z  
 
w
 
or equivalently
     

 (−2) (−4) 

1   0 
  
y
  +w
  : y, w ∈ R .

 0  0  

0 1
 
Hence, the dimension of the solution space (nullity) is 2.

Vector Spaces
Theorem 7
A homogeneous linear system with fewer equations than the number of
unkowns has always non-trivial solutions.

Determinants
Determinants
Determinant is a function from square matrices to scalars.

Our efficient computational procedure will be cofactor expansion.
However, for square matrices of the size 2 or 3, we will give simpler rules.

Determinants
Definition 20 (Determinant of 2 × 2 Matrices)

a11 a12
Let A := , then its determinant is the number defined by
a21 a22

a11 a12
a21 a22 := a11 a22 − a12 a21 .

Determinat of A is also denoted by |A| or det(A).
Example 18
We compute that

3 2

(−4) = 3 · 1 − 2 · (−4) = 11.
1

Determinants
Definition 21 (Determinant of 3 × 3 Matrices)

 
a11 a12 a13
Let A :=  a21 a22 a23 , then det(A) is defined by
a31 a32 a33

a11 a12 a13

a21 a22 a23 :=[a11 a22 a33 + a21 a32 a13 + a31 a12 a23 ]

a31 a32 a33
− [a13 a22 a31 + a23 a32 a11 + a33 a12 a21 ].

Determinants

a11 a12 a13

a21 a22 a23

a31 a32 a33

(−) a11 a12 a13 (+)
(−) a21 a22 a23 (+)
(−) (+)
Figure 9: A diagram for the Sarrus rule.

Determinants
Example 19
We compute by Sarrus rule that

1 5 (−3)

2
3 (−4) =[1 · 3 · 2 + 2 · 6 · (−3) + (−1) · 5 · (−4)]
(−1) 6 2
− [(−3) · 3 · (−1) + (−4) · 6 · 1 + 2 · 5 · 2]
=[6 + (−36) + 20] − [9 + (−24) + 20]
=(−10) − 5 = −15.

Determinants
Before proceeding for the general definition of a square matrix

determinant, we need to introduce two concepts.
Definition 22 (Minors and Cofactors of a Matrix)

Let A := (aij ) be square matrix of size n. For an entry aij , its minor,
which is denoted by Mij , is defined to be the determinant of the matrix
formed by deleting the row and the column where aij is located. For an
entry aij , its cofactor is defined to be Cij := (−1)i+j Mij .

Determinants
 
a11 ··· a1(j−1) a1j a1(j+1) ··· a1n
 
 .. .. .. .. .. .. .. 

 . . . . . . . 

 a
 (i−1)1 · · · a(i−1)(j−1) a(i−1)j a(i−1)(j+1) · · · a(i−1)n 

ai1 · · · ai(j−1) aij ai(j+1) · · · ain
 
 
 
 a(i+1)1
 · · · a(i+1)(j−1) a(i+1)j a(i+1)(j+1) · · · a(i+1)n 

 
 .. .. .. .. .. .. .. 

 . . . . . . . 

an1 ··· an(j−1) anj an(j+1) ··· ann
Figure 10: A diagram for minors and cofactors.

Determinants
Definition 23 (Determinant of n × n Matrices by Cofactor Expansion)

Let A := (aij ) be an n × n matrix. Determinant of A through the k-th
row is defined by
det(A) := ak1 Ck1 + ak2 Ck2 + · · · + akn Ckn .
Similarly, cofactor expansion through the k-th column is defined by
det(A) := a1k C1k + a2k C2k + · · · + ank Cnk .
While computing determinant of a matrix by cofactor expansion, it would

be easier to choose the row or the column, where zeroes appear the most.
If A = diag(ak ) is a diagonal matrix of size n, then det(A) = a1 a2 · · · an .

Determinants
Example 20
We compute that

6 0 (−3) 5

4 13 6 (−8)

(−1) 0 7 4

8 6 0 2

4 6 (−8) 6 (−3) 5
=0(−1)1+2 (−1) 7 4 + 13(−1)2+2 (−1)

7 4
8 0 2 8 0 2

6 (−3) 5 6 (−3) 5
3+2 4+2

+ 0(−1) 4 6 (−8)
+ 6(−1) 4 6 (−8)

8 0 2 (−1) 7 4
=0 · (−1) · 708 + 13 · 1 · (−298) + 0 · (−1) · 48 + 6 · 1 · 674
=(−3874) + 4044 = 170.

Determinants
Theorem 8
Let A and B be two square matrices of the same size. Then,
det(AB) = det(BA) = det(A) det(B).

Determinants
Property 4 (Properties of Determinant)

Determinant of a matrix satisfy the following properties.
1. Interchange of two rows or two columns multiplies the value of the
determinant by (−1).
2. Addition of a multiple of a row or a column to another does not
change the value of the determinant.
3. Multiplication of a row or a column by a non-zero constant c
multiplies the value of the determinant by c.

Determinants
Property 5 (Further Properties of Determinant)
Determinant of a matrix also has the following properties.

1. Transposition of a matrix does not change the value of the
determinant.
2. A zero row or a column renders the value of the determinant zero.
3. Proportional rows or columns render the value of the determinant
zero. In particular, a determinant with two identical rows or columns has
the value zero.
Theorem 9 (Determinant of Orthogonal Matrices)

The determinant of an orthogonal matrix is either 1 or (−1).

Cramer’s Rule
Cramer’s Rule
Theorem 10 (Cramer’s Rule)
Consider the square system
a11 x1 + a12 x2 + · · · + a1n xn =b1

a21 x1 + a22 x2 + · · · + a2n xn =b2
.. (13)
.
an1 x1 + an2 x2 + · · · + ann xn =bn .
Denote by A and b the coefficient matrix and the right-hand side vector,
respectively. For k = 1, 2, · · · , n, denote by Ak the matrix formed by
replacing k-th column of A by b. Then, we have the following.
det(Ak )
1. If det(A) 6= 0, then (13) has a unique solution given by xk = det(A) .
2. If det(A) = 0 and det(Ak ) = 0 for all k, then (13) has infinitely many
solutions.
3. If det(A) = 0 and det(Ak ) 6= 0 for some k, then (13) has no solutions.

Cramer’s Rule
As an immediate consequence of Theorem 10, we can give the following

corollary.
Corollary 1
Consider the square homogeneous system
a11 x1 + a12 x2 + · · · + a1n xn =0

a21 x1 + a22 x2 + · · · + a2n xn =0
.. (14)
.
an1 x1 + an2 x2 + · · · + ann xn =0.
Then, (14) has the trivial solution only if and only if det(A) 6= 0.
Note that in this case, det(Ak ) = 0 for all k by the second case of
Corollary 5, and thus last case of Theorem 10 does not appear.

Cramer’s Rule
Example 21
The system
x + 2y =−3
2x + 6y + 4z = − 6
−x + 2z =1
has a unique solution since the determinant of the coefficient matrix is
(−4). Thus, the solution is given by

(−3) 2 0 1 (−3) 0

(−6) 6 4 2 (−6) 4

1 0 2 (−4) (−1) 1 2
x= = = 1, y = = −2,
(−4) (−4) (−4)
and similarly z = 1.

Cramer’s Rule
Definition 24 (Singular Matrix and Non-Singular Matrix)

Let A be a square matrix of size n. A is said to be singular if A−1 does
not exist. Otherwise, A is said to be non-singular.
Theorem 11 (Existence of the Inverse Matrix)

Let A be a square matrix of size n. A is non-signular if and only if
det(A) 6= 0. Further, A is non-singular if and only if rank(A) = n.
The inverse matrix A−1 of A can be obtained by transforming the

augmented matrix (A | I) into (I | B) by using elementary row operations.
This procedure is known as Gauss-Jordan elimination process and the
matrix B is the inverse of A, i.e., B := A−1 .

Cramer’s Rule
Example 22
Find the inverse matrix of the matrix A given below.
 
1 2 3
A :=  0 1 4 .
5 6 0
By using Gauss-Jordan process for the matrix (A | I), we have

   
1 2 3 1 0 0 1 2 3 1 0 0
 0 1 4 0 1 0 ∼ 0 1 4 0 1 0 
5 6 0 0 0 1 0 (−4) (−15) (−5) 0 1
   
1 2 3 1 0 0 1 2 3 1 0 0
∼ 0 1 4 0 1 0 ∼ 0 1 0 20 (−15) (−4) 
0 0 1 (−5) 4 1 0 0 1 (−5) 4 1

Cramer’s Rule
 
1 2 0 16 (−12) (−3)
∼ 0 1 0 20 (−15) (−4) 
0 0 1 (−5) 4 1
 
1 0 0 (−24) 18 5
∼ 0 1 0 20 (−15) (−4)  .
0 0 1 (−5) 4 1
Hence,  
(−24) 18 5
A−1 :=  20 (−15) (−4)  .
(−5) 4 1


Cramer’s Rule
Before we introduce a formula for the inverse matrix, we need to

introduce a new matrix.
Definition 25 (Adjoint Matrix)
Let A = (aij ) be a square matrix. The adjoint matrix of A is defined by
adj(A) := (Cij )T , where Cij denotes the cofactor of aij .
Theorem 12 (Inverse of a Matrix)

Let A = (aij ) be a square matrix. Then,
1
A−1 := adj(A)
det(A)
provided that det(A) 6= 0.
It should be noted that if A = diag(ak ) is a diagonal

matrix such that no
entry on the diagonal is zero, then A−1 = diag a1k .

Cramer’s Rule
Example 23
Clearly, for
3 1
A :=
2 4
we have det(A) = 10 and
T
4 (−2) 4 (−1)
adj(A) = = .
(−1) 3 (−2) 3
Therefore, we have

−1 1 4 (−1)
A = .
10 (−2) 3

Cramer’s Rule
Example 24
For the matrix  
1 (−1) (−2)
A :=  3 (−1) 1 ,
1 (−3) (−4)
we compute that det(A) = 10. Also, we have
 
7 2 (−3)
adj(A) =  13 (−2) (−7)  .
(−8) 2 2
Thus, the inverse matrix is

 
7 2 (−3)
1 
A−1 = 13 (−2) (−7)  .
10
(−8) 2 2

Cramer’s Rule
Theorem 13
Let A and B be two square matrices of the same size. Then,
(AB)−1 = B −1 A−1 .

Cramer’s Rule
The inverse matrix is useful for solving systems of the form
Ax = b.
Once A−1 is known, we can multiply both sides of the above system by
A−1 and get
x = A−1 b.
Example 25
Show that
x − y − 2z = − 10
3x − y + z =5
x − 3y − 4z =20
has the solution
x = −12, y = −28 and z = 13.

Cramer’s Rule
Remark 1
Some remarks on matrix multiplication are given below.
1. Matrix multiplication is not commutative, i.e., we have in general that
AB 6= BA.
2. AB = 0 does not imply that A = 0 or B = 0 (or BA = 0). For

instance,
1 1 (−1) 1 0 0
= .
2 2 1 (−1) 0 0
3. AB = AC does not generally imply B = C even when A 6= 0.

Cramer’s Rule
Theorem 14 (Cancellation Laws)

Let A, B, C be square matrices of the same size. Then, the following
properties hold.
1. If det(A) 6= 0 and AB = AC , then B = C .
2. If det(A) 6= 0 and AB = 0, then B = 0. Hence, if AB = 0 with
A 6= 0 and B 6= 0, then det(A) = 0 or det(B) = 0.
3. If A is singular, then so are AB and BA.

Definition 26 (Real Inner Product Space)

Let V be a vector space over the field R. V is said to be an inner
product space over R, if there exists a mapping h·, ·i : V × V → R
satisfying the following properties.
1. Positive-Definiteness:
(a) hu, ui ≥ 0 for all u ∈ V .
(b) hu, ui = 0 if and only if u = 0.
2. Symmetry: hu, v i = hv , ui for all u, v ∈ V .
3. Linearity:
(a) hu + v , zi = hu, zi + hv , zi for all u, v , z ∈ V .
(b) hαu, v i = αhu, v i for all u, v ∈ V and all α ∈ R.

Example 26
Let (u1 , u2 ), (v1 , v2 ) ∈ R2 , then
h(u1 , u2 ), (v1 , v2 )i = u1 v1 + u2 v2
satisfies the properties of vector spaces. Indeed, we have
h(u1 , u2 ), (u1 , u2 )i = u12 + u22 ≥ 0,
and it is zero if and only if u1 = 0 and u2 = 0, i.e., (u1 , u2 ) = (0, 0).

Next, we see that
h(u1 , u2 ), (v1 , v2 )i = u1 v1 + u2 v2 = v1 u1 + v2 u2 = h(v1 , v2 ), (u1 , u2 )i. I

Finally, we have
h(u1 , u2 ) + (v1 , v2 ), (z1 , z2 )i = h(u1 + v1 , u2 + v2 ), (z1 , z2 )i

=(u1 + v1 )z1 + (u2 + v2 )z2 = (u1 z1 + u2 z2 ) + (v1 z1 + v2 z2 )
=h(u1 , u2 ), (z1 , z2 )i + h(v1 , v2 ), (z1 , z2 )i
and
hα(u1 ,u2 ), (v1 , v2 )i = h(αu1 , αu2 ), (v1 , v2 )i
=(αu1 )v1 + (αu2 )v2 = α(u1 v1 + u2 v2 )
=αh(u1 , u2 ), (v1 , v2 )i.
Thus, R2 is an inner product space on R.

Example 27
Two examples of inner product spaces are listed below.
1. Rn is an inner product space with
   
* u1 v1 +
 u2   v2 
 ..  ,  ..  := u1 v1 + · · · + un vn .
   
 .   . 
un vn
2. C[a, b] is an inner product space with

Z b
hf , g i := f (ξ)g (ξ)dξ.
a

Definition 27 (Orthogonality)
Let V be an inner product space on R, and u, v ∈ V . The vectors u and
v are called orthogonal provided that hu, v i = 0.
Example 28
1. In R3 , u := (2, 1, −1) and v := (1, −1, 1) are orthogonal.
a+b a+b 2

2. In C[a, b], f (x) := x − 2 and g (x) := x 2 − (a + b)x + 2 are
orthogonal.

Definition 28 (Norm)
Let V be an inner product space on R, and u ∈ V . The norm of u is
defined by p
kuk := hu, ui.
Further, u is said to be a unit vector provided that kuk = 1.
Example 29
1. On Rn , a norm is given by
 
u1

 u2  q
 ..  := u12 + u22 + · · · + un2 .
 
 . 

un
2. On C[a, b], a norm is given by

s
Z b 2
kf k := f (ξ) dξ.
a

Property 6
Inner products and norms satisfy the following properties.
1. Cauchy-Schwarz Inequality: |hu, v i| ≤ kukkv k for all u, v ∈ V .
2. Triangle Inequality: ku + v k ≤ kuk + kv k for all u, v ∈ V .

3. Parallelogram Equality: ku + v k2 + ku − v k2 = 2 kuk2 + kv k2 for
all u, v ∈ V .

Definition 29 (Linear Transformation)

Let V and W be vector spaces over the field R. The mapping
f : V → W is called as a linear transform provided that the following
properties are satisfied.
1. f (u + v ) = f (u) + f (v ) for all u, v ∈ V .
2. f (αu) = αf (u) for all u ∈ V and all α ∈ R.

Example 30
Some examples of linear transforms are listed below.
1. Zero transform, i.e., f (u) = 0.
2. Identity transform, i.e., f (u) = u.
3. Reflection transform, i.e., f (u) = −u.
4. Scaling transform, i.e., f (u) = λu, where λ ∈ R
5. Projection transform.
6. Rotation transform.
7. Differential transform.
8. Integral transform.

Representation Matrix
Let {e 1 , e 2 , · · · , e n } be a basis for Rn . Then for every u ∈ Rn , there

exist scalars α1 , α2 , · · · , αn ∈ R such that
u = α1 e 1 + α2 e 2 + · · · + αn e n .
If f is a linear mapping, then we have
f (u) = α1 f (e 1 ) + α2 f (e 2 ) + · · · + αn f (e n ).
Hence, if we know the values f (e 1 ), f (e 2 ), · · · , f (e n ), then we can

uniquely determine f (u). This idea can be used to write the transform f
as
f (u) = Au,
where A is an m × n matrix provided that f : Rn → Rm .

Example 31

1 1
Consider the basis , of R2 . Let us find the rule of
1 (−1)
the linear transform f : R2 → R3 satisfying
   
5 3
1 1
f =  (−1)  and f =  (−5)  .
1 (−1)
11 1

This implies that the transform f is of the form

x x +y 1 x −y 1
f =f +
y 2 1 2 (−1)

x +y 1 x −y 1
= f + f
2 1 2 (−1)
   
5 3
x +y  x −y 
= (−1)  + (−5) 
2 2
11 1
 
4x + y
=  −3x + 2y  .
6x + 5y

Thus, f has the matrix representation

 
4 1
x x
f =  (−3) 2  .
y y
6 5

Definition 30 (Null Space and Range)

Let f : V → W be a linear transform. The null space of f is defined by
Null(f ) := {u : f (u) = 0}
and the range of f is defined by
Range(f ) := {v : f (u) = v }.
Further,

nullity(f ) := dim Null(f ) and rank(f ) := dim Range(f ) .

Recalling Theorem 5 and the definition of the representation matrix, we

can give the following result.
Theorem 15
Let f : V → W be a linear transform. Then,
rank(f ) + nullity(f ) = dim(V ).

Example 32
Find the rank and the nullity of the transform f : R3 → R2 defined by
 
x
16x + y − 3z
f  y = .
8x − 3y + z
z
We can rewrite f in the form

   
x x
16 1 (−3)
f  y  =  y ,
8 (−3) 1
z z

We see that    
 1 
Null(f ) = x  5  : x ∈ R
7
 
 
 1 
= Span  5  ,
7
 
and thus nullity(f ) = 1. Then, by Theorem 15, we have rank(f ) = 2.

Let A := (aij ) be an n × n matrix and consider the vector equation
Ax = λx. (15)
Here, x is an unknown vector and λ is an unknown scalar. Our task is to

determine the x vectors and λ values for which (15) holds. Geometrically,
we are looking for vectors x for which multiplication by A has the same
effect as multiplication by a scalar λ, i.e., Ax is proportional to x.

Clearly, x = 0 is a solution of (15) for any value of λ since A0 = 0 = λ0.

This is obvious and is of no interest. A value of λ for which (15) has a
non-trivial solution (x 6= 0) is called as an “eigenvalue” (or a
“characteristic value”) of the matrix A.
The non-trivial solutions x of (15) are called as “eigenvectors” (or
“characteristic vectors”) of the matrix A corresponding to the
eigenvalue λ.
The set of all eigenvalues of A is called as the “spectrum” of A. We will
see in examples that the spectrum of A consists of at least one
eigenvalue and at most of n numerically different eigenvalues.

This example illustrates the general case as follows. Expanding (15) in its
components, we get
a11 x1 + a12 x2 + · · · + a1n xn =λx1

a21 x1 + a22 x2 + · · · + a2n xn =λx2
..
.
an1 x1 + an2 x2 + · · · + ann xn =λxn .

Moving the terms on the right-hand side to the left-hand side gives us
(a11 − λ)x1 + a12 x2 + · · · + a1n xn =0

a21 x1 + (a22 − λ)x2 + · · · + a2n xn =0
..
.
an1 x1 + an2 x2 + · · · + (ann − λ)xn =0.
This is written in the matrix form as
(A − λ I)x = 0.

By Cramer’s rule, this linear system of equations has a non-trivial

solution if and only if the corresponding determinant of the coefficient
matrix is zero, i.e.,
D(λ) := det(A − λ I) = 0.
Here, (A − λ I) is called as the “characteristic matrix” and D(λ) is
called as the “characteristic determinant” of A. Clearly, D(λ) gives us
a polynomial of n-th degree in λ, which is called as the “characteristic
polynomial” of A.

Theorem 16 (Eigenvalues)
The eigenvalues of a square matrix A are the roots of the characteristic
equation det(A − λ I) = 0. Hence, an n × n matrix has at least 1
eigenvalue and at most n numerically different eigenvalues.

For obtaining the eigensystem (eigenvalues and eigenvectors) of a square

matrix, we must first find the eigenvalues from the characteristic
equation det(A − λ I) = 0. Then, we should use them in the system
(A − λ I)x = 0 to find the eigenvectors.

Example 33

2 2
The characteristic polynomial of the matrix A := is
1 3

2−λ 2
D(λ) := = λ2 − 5λ + 4 = (λ − 1)(λ − 4)
1 3−λ
whose roots are λ1 := 1 and λ2 := 4. Thus, the eigenvalues of A are 1

and 4, i.e., the spectrum of A is {1, 4}. I

It follows from the system (A − λ I)x = 0 with λ1 = 1 that

1 2 x1 0
= .
1 2 x2 0
Then, we get x1 = −2x2 , which yields

(−2)
x =c ,
1

2
where c is an arbitrary constant. So, we can take as the
(−1)
eigenvector
corresponding
to the eigenvalue 1. We could also take
(−1) 2
1 instead of . However, both are linearly dependent
2
(−1)
and the latter one looks more simple. I

The system (A − λ I)x = 0 with λ2 = 4 becomes

(−2) 2 x1 0
= .
1 (−1) x2 0
Solving this, we get

1
x =c ,
1

1
where c is an arbitrary constant. Hence, is an eigenvector, which
1
corresponds to the second eigenvalue 4.

Theorem 17 (Eigenvectors and Eigenspaces)

Let x and y be eigenvectors of a matrix A corresponding to the same
eigenvalue λ. Then, (x + y ) (with x + y 6= 0) and αx (with αx 6= 0) are
also eigenvectors, which correspond to λ.
Hence, the eigenvectors corresponding to one and the same eigenvalue of
A, together with 0, form a vector space, which is called as the
“eigenspace” of A corresponding to that eigenvalue.

Example 34
 
1 2 1
Let us consider the matrix A :=  0 3 2 . The characteristic
(−1) 1 1
equation of A is D(λ) := −λ3 + 5λ2 − 6λ = −λ(λ − 2)(λ − 3). Thus, A
has the eigenvalues
 λ
1 := 0, λ2 := 2and λ3 :=
 3. Further, we get
 the
1 3 1
eigenvector  (−2)  for λ1 = 0,  2  for λ2 = 2 and  1 
3 (−1) 0
for λ3 = 3.

Example 35
 
1 0 0
Consider the matrix A :=  2 1 0 . The characteristic equation
1 (−2) 3
of A is D(λ) := −λ3 + 5λ2 − 7λ + 3 = −(λ − 1)2 (λ − 3). Thus, A has
the
 eigenvalues
 λ1,2 := 1 and λ3 :=
 3. 
Further, we get the eigenvector
0 0
 1  for the eigenvalue 1 and  0  for the eigenvalue 3. In this
1 1
example, we can find only two eigenvectors.

Theorem 18 (Eigenvalues of a Transpose)

Let A be a square matrix. The matrix A and its transpose AT have the
same eigenvalues.

Theorem 19 (Eigenvalues of Symmetric and Skew-Symmetric Matrices)

Symmetric matrices have real eigenvalues while skew-symmetric matrices
have purely imaginary or zero eigenvalues.

Theorem 20 (Eigenvalues of Orthogonal Matrices)

Let A be an orthogonal square matrix. Then, its eigenvalues are real or
complex conjugate in pairs and have modulus 1.

Diagonalization
Definition 31 (Similar Matrices)

Let A and B be n × n matrices. B is said to be “similar” to A if there
exists some non-singular matrix P such that
B = P −1 AP.

Diagonalization
Theorem 21 (Eigenvalues and Eigenvectors of Similar Matrices)

If A and B are similar matrices, then both have the same eigenvalues.
Further, if x is an eigenvector of A, then y := P −1 x is an eigenvector of
B, which corresponds to the same eigenvalue.

Diagonalization
Theorem 22 (Diagonalization of a Matrix)

Let A be an n × n matrix with n linearly independent eigenvalues. Then,
D := X −1 AX
is a diagonal matrix whose entries on the main diagonal are the

eigenvalues of A. Here, X is the matrix with the eigenvectors as column
vectors.
Further, for m ∈ N, we have
D m = X −1 Am X
or equivalently
Am = X D m X −1 .

Diagonalization
Example 36

4 1
Consider the matrix A := . We see that the eigenvalues
(−8) (−5)

1
of A are (−4) and 3, which yield the eigenvectors and
(−8)

1 1 1
, respectively. Let us define X := , which
(−1) (−8) (−1)

(−1) (−1)
yields X −1 = 17 . Then, we get the diagonal matrix
8 1

(−4) 0
D := X −1 AX = .
0 3
Further, note that

7 7 −1 4840 2653
A = XD X = .
(−21224) (−19037)

Mat2324 1 PDF

Uploaded by

Copyright:

Available Formats

You might also like

Mat2324 1 PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Mat2324 1 PDF

Uploaded by

Copyright:

Available Formats

MAT5101

Applied Mathematics 1/142

Real Inner Product Spaces

Eigenvalues, Eigenvectors and Eigenspaces

Applied Mathematics 2/142

Matrices and Linear Systems

Applied Mathematics 3/142

We will denote by capital bold letters such as A, B, · · · , or by writing the

Applied Mathematics 4/142

Applied Mathematics 5/142

If either m = 1 or n = 1, then a matrix is called as a vector. We will

while if n = 1, then the vector is called a column vector, i.e.,

Applied Mathematics 6/142

Definition 1 (Equality of Matrices)

Applied Mathematics 7/142

Definition 2 (Addition of Matrices)

Applied Mathematics 8/142

Definition 3 (Scalar Multiplication)

Applied Mathematics 9/142

Property 1 (Rules for Matrix Addition and Scalar Multiplication)

Applied Mathematics 10/142

Definition 4 (Matrix Multiplication)

The product C = AB (in this order) of an m × n matrix A = (aij ) times

Applied Mathematics 11/142

In (2), cik is obtained by multiplying each entry in the i-th row of A by

Applied Mathematics 12/142

Figure 1: An illustration of matrix multiplication.

Applied Mathematics 13/142

Note that multiplication in the reverse order is not defined.

Applied Mathematics 14/142

It should be noted that matrix multiplication is in general not

Applied Mathematics 15/142

Our examples show that order of factors in matrix multiplication must

Property 2 (Rules for Matrix Multiplication)

Applied Mathematics 16/142

Definition 5 (Transposition of a Matrix)

As a particular case, transposition converts row vectors to column vectors

Applied Mathematics 17/142

Property 3 (Rules for Matrix Transposition)

Applied Mathematics 18/142

Transposition gives a rise to two useful classes of matrices as follows.

Definition 6 (Symmetric and Skew-Symmetric Matrices)

Note that skew-symmetric matrices have zero as their diagonal entries.

Applied Mathematics 19/142

Any real square matrix A may be written as the sum of a symmetric

Applied Mathematics 20/142

Definition 7 (Upper and Lower Triangular Matrices)

Applied Mathematics 21/142

Definition 8 (Diagonal and Identity Matrices)

The diagonal matrix A whose diagonal entries are ak is denoted by

Definition 9 (Orthogonal Matrices)

Applied Mathematics 23/142

Definition 10 (Linear Systems)

a11 x1 + a12 x2 + · · · + a1n xn =b1

The system is called as linear because x1 , x2 , · · · , xn appear in the first

Applied Mathematics 24/142

From the definition of matrix multiplication, we see that the m equations

where A, x and b are defined as