CHAPTER 3

ORTHONORMAL LISTS IN INNER PRODUCT SPACES

Let V and W be inner product spaces throughout this chapter.
We have been studied that every finite dimensional vector space has a basis of
right length from every linearly independent list in an inner product space we derive a
list of vectors whose lengths are 1 and one vector in list is orthogonal to any other
vector in list, called orthonormal list. We study an algorithm to find such list, call
Gram-Schmidt Procedure. We also study Riesz Representation Theorem: Every linear
functional on a finite dimensional inner product space is represented by the inner
product on it. Lastly, we study some properties on the orthogonal complement of a
subspace of an inner product space and orthogonal projections on this space.
3.1 Orthonormal Bases
In this section, we study orthonormal lists, orthonormal bases, method to find
a orthonormal list from the given linearly independent list, called Gram-Schmidt
Procedure, every finite dimensional inner product space has an orthonormal basis and
every orthonormal list in an inner product space can be extended an orthonormal
basis.
3.1.1 Definition A list of vectors is called orthonormal if each vector in the list has
norm 1 and is orthogonal to all the other vectors in the list. In other words, a list
e1 , , en of vectors in V is orthonormal if

1 if j  k ,
e j , ek  
0 if j  k .

3.1.2 Example (a) The standard basis e1  1, 0 , e2   0,1 in 2

is an orthonormal
list since
e1 , e1  11  0  0  1, e1 , e2  1 0  0 1  0, e2 , e1  0, e2 , e2  1.

 1 1 1   1 1   1 1 2 
(b) v1   ,  , v2    , 0 , v3  
 6 6 6 
, , , , is an orthonormal
 3 3 3  2 2 
list in 3 .
3.1.3 Proposition If e1 , , en is an orthonormal list of vectors in V, then

1e1    n en  1   n
2 2 2
(3.1)

for all 1 , ,n  .

Proof. Since ei , e j  0, for i  j ,

2 2
i ei   j e j  i ei   j e j
2

by Pythagorean Theorem. Then

 29

2 2 2 2
i ei   j e j  i ei   j  i   j
2 2 2
ej

since ei  e j  1. Hence

1e1   2 e2    n en  1e1    n 1en 1   n en

2 2 2

 1e1    n 1en 1   n
2 2

 1e1    n  2 en  2   n 1   n
2 2 2

 1 2   n .
2


3.1.4 Proposition Every orthonormal list of vectors is linearly independent.
Proof. Suppose e1 , , en is an orthonormal list of vectors in V and a1 , , an  are
such that
1e1    n en  0.

Then 1    n  0 by (3.1), which means that all the  j ' s are 0. Thus e1 ,
2 2
, en
is linearly independent. 
3.1.5 Definition An orthonormal basis of V is an orthonormal list of vectors in V
that is also a basis of V.
2
For example, the standard basis e1 , e2 of in Example 3.1.2 (a) is an
orthonormal basis.
3.1.6 Proposition Every orthonormal list of vectors in V with length dim V is an
orthonormal basis of V.
Proof. By Proposition 3.1.4, any such list must be linearly independent. Since it has
the right length, it is a basis. 

 1 1 1 1  1 1 1 1  1 1 1 1
3.1.7 Example We show that  , , ,  ,  , ,  ,   ,  ,  ,  ,  ,
 2 2 2 2  2 2 2 2  2 2 2 2
 1 1 1 1
  , ,  ,  is an orthonormal basis of
4
.
2 2 2 2
We have
2 2 2 2
 1 1 1 1  1  1  1  1
 , , ,               1.
2 2 2 2 2 2 2 2
Similarly, the other three vectors in the list above also have norm 1. Also, we have
 30

 1 1 1 1  1 1 1 1 1 1 1 1
 , , ,  ,  , ,  ,        0.
2 2 2 2 2 2 2 2 4 4 4 4
Similarly, the inner product of any two distinct vectors in the list above also equals 0.
Thus the list above is orthonormal. Because we have an orthonormal list of
length four in the four dimensional vector space 4 , this list is an orthonormal basis
4
of .
In general, given a basis e1 , , en of V and a vector v V , we know that there is
some choice of scalars 1 , ,n  such that

v  1e1    n en .

It is difficult to find 1 , ,  n which satisfy the equation above for an arbitrary

basis of V. The next result shows that it is easy for orthonormal basis.
3.1.8 Theorem Suppose e1 , , en is an orthonormal basis of V and v V . Then

v  v, e1 e1   v, en en (3.2)
and
2 2
v  v, e1   v, en
2
.

Proof. Since e1 , , en is a basis of V, there exist scalars 1 , ,  n such that

v  1e1    n en .
Then for 1  j  n,

v, e j   j ;

hence (3.2) holds. Next we have

2 2 2
v  v, e1 e1   v, en en  v, e1   v, en
2
. 

The algorithm used in the next proof is called the Gram-Schmidt Procedure. It
gives a method to find a orthonormal list from the given linearly independent list with
the same span as the original list.
3.1.9 Theorem Suppose v1 , , vn is a linearly independent list of vectors in V. Let
v1
e1  . For j  2, , n, define e j inductively by
v1

v j  v j , e1 e1   v j , e j 1 e j 1
ej  . (3.3)
v j  v j , e1 e1   v j , e j 1 e j 1

Then e1 , , en is an orthonormal list of vectors in V such that

 31

span v1 , 
, v j  span e1 ,  ,ej 
for j  1, , n.

Proof. We will show by induction on j. For j  1, we must show that span  v1   span
 e1  . If v  span  v1  then v   v1 where   .

By given v   v1  e1  span  e1  . Conversely, if v  span  e1  then v   e1

where   . By given,
 v 
v    1 
 v1 
and so, v  span  v1  . Thus

span  v1   span  e1  .

Suppose 1  j  n and suppose that e1 ,..., e j is an orthonormal list of vectors in V such

that


span v1 , 
, v j 1  span e1 ,  
, e j 1 . (3.4)

Note that v j span v1 ,  

, v j 1 because v1 , , vn is linearly independent. Thus


v j span e1 , 
, e j 1 . Hence we are not dividing by 0 in the definition of e j in given
2
(3.3). Thus e j  e j , e j  1; e j  1.

Let 1  k  j. Then

v j  v j , e1 e1   v j , e j 1 e j 1
e j , ek  , ek
v j  v j , e1 e1   v j , e j 1 e j 1

v j , ek  v j , ek

v j  v j , e1 e1   v j , e j 1 e j 1

 0.
Thus e1 , , e j is an orthonormal list.

By (3.3), we see that v j  span e1 ,  

, e j . Combing this information with (3.4)
shows that
span v1 , 
, v j  span e1 ,  , ej .
 32

Both lists above are linearly independent by hypothesis and Proposition 3.1.4. Thus
both subspaces above have dimension j, and hence they are equal, completing the
proof. 

3.1.10 Example We find an orthonormal basis of  2   , where the inner product is

1

given by p, q   p( x) q  x dx.
1

We will apply the Gram-Schmidt Procedure to the basis 1, x, x 2 . Let v1  1,

v2  x, and v3  x2 .
To get started, with this inner product we have
1

1  1,1   11 dx  2.
2

1

1 1
Thus 1  2, and hence e1   . Now we find e2 :
1 2
1
1 1 1 1 1
v2  v2 , e1 e1  x  x,  x  x dx  x  1  1  x
2 2 2 1 2 4
and
1
1 2
v2  v2 , e1 e1  x   x 2 dx  1  1  .
2 2

1
3 3

x 3
Thus e2   x. We find e3 :
2 2
3

1 1 3 3 1
v3  v3 , e1 e1  v3 , e2 e2  x 2  x 2 ,  x2 , x x  x2  .
2 2 2 2 3

We have
2 2
 1
1
1 8
   x 2   dx  .
2
v3  v3 , e1 e1  v3 , e2 e2  x  2

3 1
 3 45

45  2 1 
Thus e3   x   . Hence
8  3

1 3 45  2 1 
, x, x  
2 2 8  3
is an orthonormal list of length 3 in  2  . Hence the orthonormal list is an
orthonormal basis of  2   by Proposition 3.1.6.
 33

3.1.11 Theorem Every finite dimensional inner product space has an orthonormal
basis.
Proof. Suppose V is finite dimensional. Choose of a basis of V. V has an orthonormal
list with length dim V by using the Gram-Schmidt Procedure. By Proposition 3.1.6,
this orthonormal list is an orthonormal basis of V. 
3.1.12 Theorem Suppose V is finite dimensional. Then every orthonormal list of
vectors in V can be extended to an orthonormal basis of V.
Proof. Suppose e1 , , en is an orthonormal list of vectors in V. Then e1 , , en is
linearly independent by Proposition 3.1.4. Hence this list can be extended to a basis
e1 , , en , v1 , , vm of V by Theorem 1.3.7. By using Gram-Schmidt Procedure to
e1 , , en , v1 , , vm ,we have an orthonormal list
e1 , , en , f1 , , fm .

The list above is an orthonormal basis of V by Proposition 3.1.6. 

3.2 Linear Functionals on Inner Product Spaces
We consider liner maps from V into , called linear functionals. We study
Riesz Representation Theorem: every linear functional on V can be represented by an
inner product on it.
3.2.1 Definition A linear functional on an inner product space V is a linear map from
V to . In other words, a linear functional is an element of L V ,  .

3.2.2 Example The function  : 2

 defined by

  z1 , z2   2 z1  5 z2
is a linear functional on 𝔽2 . We could write this linear functional in the form
  z   z, u

for every z ∈ 𝔽2 , where u   2, 5  .

If v V , then the map that sends v to v, u is a linear functional on V. The

next result shows that every linear functional on V is of this form. This result is honor
of Hungarian mathematician Frigyes Riesz (1880-1956).
3.2.3 Example The function  : P2   2
defined by
1
  p    p  t  cos  t  dt ,
1

is a linear functional on P2   . For p, q  P2   , we have

 34

1
  p  q    p  q  t  cos  t  dt
1

1 1
  p  t  cos  t  dt   q  t  cos  t  dt
1 1

   p  q .

Also, for p  P2   and   , we have

1
  p     p  t  cos  t  dt
1

1
   p  t  cos  t  dt
1

   p  .

It is not obvious that there exists u  P2   such that

  p   p, u

for every p  P2   (we cannot take u  t   cos  t  because that function is not an
element of P2   ).
3.2.4 Theorem (Riesz Representation Theorem) Suppose V is finite dimensional
and  is a linear functional on V. Then there is a unique vector u V such that

  v   v, u
for every v V .
Proof. First we show there exists a vector u V such that   v   v, u for every
v V . Let e1 , , en be an orthonormal basis of V. Then by (3.2),
v  v, e1 e1   v, en en and so

  v     v, e1 e1   v, en en 

 v, e1   e1    v, en   en 

 v,   e1 e1     en en

for every v V . Thus letting

u    e1 e1     en en , (3.5)

we have   v   v, u for every v  V , as desired.

 35

Now we prove that only one vector u  V has the desired behavior. Suppose
u1 , u2  V are such that

  v   v, u1  v, u2
for every v  V . Then
0  v, u1  v, u2  v, u1  u2

for every v  V . Taking v  u1  u2 show that u1  u2  0. The proof is complete. 

3.2.5 Example We find an orthonormal basis of P2   , where the inner product is

1
given by p, q   p  x  q  x  dx .
0

We will apply the Gram-Schmidt Procedure to the basis 1, x, x 2 . Let

v1  1, v2  x and v3  x2 .
To get started with this inner product we have
1
1  1,1   1.1dx  1.
2

Thus 1  1, and hence e1 

1  1. Now we find e :
2
1
1
v2  v2 , e1 e1  x  x,1 1  x   xdx  x  1
0
2

and

  dx  121 .
2 1 2
 x 1   x 1
2
v2  v1 , e1 e1
2 0
2

 
Thus e2  x  1 2 3  3  1  2 x  . We find e3 :
2

v3  v3 , e1 e1  v3 , e2 e2  x 2  x  1 .
6
We have

 1 .
2
v3  v3 , e1 e1  v3 , e2
180
Thus

 
e3  6 5 x 2  x  1  5 1  6 x  6 x 2  .
6
Hence 1, 3  1  2 x  , 5 1  6 x  6 x 2  is an orthonormal basis of P2  .
 36

3.2.6 Example We find a polynomial q  P2 ( ) such that

  1
p  1    p( x)q( x) dx
2 0
for every p  P2 ( ). We will need an orthonormal basis of P2 ( ), where the inner
product of two polynomials in P2 ( ) is defined to be the integral from 0 to 1 of the
product of the two polynomials. An orthonormal basis of P2 ( ) was already
computed in Example 3.2.5. Specifically, let
e1 ( x)  1
e2 ( x)  3(1  2 x)
e3 ( x)  5(1  6 x  6 x 2 ).

Then e1 , e2 , e3 is an orthonormal basis of P2 ( ).

Define a linear functional  on P2 ( ) by

1
 ( p)  p   .
 
2

We seek q  P2 ( ) such that  ( p)  p, q for every p  P2 ( ). By (3.2), we have

q  q, e1 e1  q, e2 e2  q, e3 e3
  (e1 )e1   (e2 )e2   (e3 )e3
1 1 1
 e1   e1  e2   e2  e3   e3
2 2 2
 6
 e1  3(1  1)e2  5 1  3   e3
 4
3
   15 x  15 x 2 .
2
3.2.7 Example We find a polynomial q  P2 ( ) such that
1 1
0
p( x) cos( x)dx  p( x)q( x)dx
0

for every p  P2 ( ). Define a linear functional  on P2 ( ) by

1
 ( p)   p( x) cos( x)dx.
0

We seek q  P2 ( ) such that  ( p)  p, q for every p  P2 ( ), where the inner

product on P2 ( ) is defined as in Example 3.2.5, letting e1 , e2 , e3 be as in Example
3.2.6. By (3.2), we have
 37

q  q, e1 e1  q, e2 e2  q, e3 e3

   e ( x)q( x)dx  e    e (x)q(x)dx  e    e (x)q(x)dx  e

1

0 1 1
1

0 2 2
1

0 3 3

   e ( x) cos( x)dx  e    e ( x) cos( x)dx  e    e ( x) cos( x)dx  e

1 1 1

0 1 1 0 2 2 0 3 3

4 3

2
 3(1  2 x) 
12  24 x
 .
2
3.3 Orthogonal Complements
3.3.1. Definition If U is a subset of V, then the orthogonal complement of U,
denoted U  , is the set of all vectors in V that are orthogonal to every vector in U:

U    v  V : v, u  0 for every u  U .

For example: If U   x1 , 0, 0  : x1   then U    0, x2 , x3  : x2 , x3  . If

U   x1 , x2 , 0  : x1 , x2   then U 
  0, 0, x3  : x3  .
We derive the basic properties of orthogonal complement.
3.3.2 Theorem Let V be an inner product space.
(a) If U is a subset of V, then U  is a subspace of V.

(b) 0  V .


(c) V   0.

(d) If U is a subset of V, then U U   0.

(e) If U and W are subsets of V and U  W , then W   U  .

Proof. (a) Suppose U is a subset of V. Then 0, u  0 for every u U ; thus 0 U  .

Suppose v, w U  . If u U then

v  w, u  v, u  w, u  0  0  0.

Thus v  w U  . Suppose   and v U  . If u U then

 v, u   v, u    0  0.

Thus v U  . Hence U  is a subspace of V.

v, 0  0, which implies v  0 . By definition of


(b) Suppose v V . Then
0 , 0  V . Thus 0  V .
  
 38

(c) Suppose v  V  . Then v, v  0, which implies v  0. Thus V   0.

(d) Suppose U is a subset of V and v U U  . Then v U and v  U  so that

v, v  0. Hence v  0. Thus U U   0. Since 0  U  and 0, 0  0,
0 U U  . Hence 0  U U  .

(e) Suppose U and W are subsets of V and U  W . Suppose v W  . Then v, w  0

for every w W . Thus v, u  0 for every u U . Thus v U  . Hence W   U  .

Recall that if U and W are subspaces of V, then V is the direct sum of U and W
(written V  U  W ) if each element of V can be written in exactly one way as a
vector in U plus a vector in W.
3.3.3 Theorem Suppose U is a finite dimensional subspace of V. Then
V  U U  .
Proof. First, we will show that
V  U U .
To do this, suppose v V . Let e1 , , en be an orthonormal basis of U. Obviously

v  v, e1 e1   v, en en  v  v, e1 e1   v, en en .
u w

Let u and w be defined as in the equation above. Clearly, u U . Since e1 , , en is an

orthonormal list, for each j  1, , n we have

w, e j  v, e j  v, e j  0.

Thus w is orthogonal to every vector in span  e1 , , en  . In other words, w U  .

Thus we have written v  u  w, where u U and w U  . By Theorem 3.3.2(d),
U U   0. Hence by Theorem 1.2.11, V  U  U  . 
3.3.4 Theorem Suppose U is a finite dimensional subspace of V. Then

 

U  U .

Proof. We firstly show that

 

U  U . (3.6)

Suppose u U . Then u , v  0 for every v U  . Since u is orthogonal to every

vector in U  , u  U   . To prove U    U suppose v  U   . By Theorem
  

3.3.3, we can write v  u  w, where u U and w U  . We have v  u  w U  .

 39

Since v  U   and u  U   , we have v  u  U   . Thus v  u U  U 

   


which implies u  v  0. Thus U    U . Hence the proof is complete.




We now define an operator PU for each finite dimensional subspace of V.

3.3.5 Definition Suppose U is a finite dimensional subspace of V. The orthogonal
projection of V onto U is the operator PU  L V  defined as follows: For v V ,
write v  u  w, where u U and w U  . Then PU v  u.

3.3.6 Proposition The orthogonal projection PU is well defined on V.

Proof. Suppose v  v1 where v, v1 V . Then by Theorem 3.3.3 v  u  w and
v1  u1  w1 where u, u1 U and w, w1 U  . So,
u  u1  w1  w U U   0

so that u  u1. Thus PU v  PU v1 so that PU is well defined. 

3.3.7 Example Suppose x V with x  0 and U  span ( x). Then
v, x
PU v  2
x
x
for every v V . For suppose v V , we have

v, x  v, x 
v x  v  x . (3.7)
x
2
 x
2

 
First term of (3.7) is in U. Since

v, x v, x
x, v  2
x  x, v  2
x, x  0,
x x

second term of (3.7) is in U  . Thus PU v equals the first term of (3.7).

3.3.8 Theorem Suppose U is a finite dimensional subspace of V and v V . Then
(a) PU  L(V );

(b) PU u  u for every u U ;

(c) PU w  0 for every w U  ;

(d) range PU  U ;

(e) null PU  U  ;

(f) v  PU v U  ;
 40

(g) PU  PU ;
2

(h) PU v  v ;

(i) for every orthonormal basis e1 , , en of U, PU v  v, e1 e1   v, en en . (3.8)

Proof. (a) To show PU is a linear map on V, suppose v1 , v2 V . Write

v1  u1  w1 and v2  u2  w2
with u1 , u2 U and w1 , w2 U  . Thus PU v1  u1 and PU v2  u2 . Now

v1  v2   u1  u2    w1  w2  ,

where u1  u2 U and w1  w2 U  . Thus

PU  v1  v2   u1  u2  PU v1  PU v2 .

Suppose   and v  u  w where u U and w U  . Then v  u   w

with u U and  w U  . Thus
PU v  u   PU v.

Hence PU is a linear map from V to V.

(b) Suppose u U . Then we write u  u  0 where u U and 0 U  . Hence

PU u  u.

(c) Suppose w U  . Then we write w  0  w where 0 U and w U  . Hence

PU w  0.

(d) range PU  PU v : v V   PU v : v  u  w where u U and w  U   U . 

(e) null PU  v : PU v  0, v V   v : u  0, v  u  w, u U and w  U   U  . 
(f) To show v  PU v U  we must show that v  PU v, u  0 for all u U . For any
v V we write v  u  w where u U and w U  . Then u  PU v and so
v  u  v  PU v  w. Since w, u  0 for all u  U , v  PU v, u  0. Hence
v  PU v U  .

(g) If v  u  w with u U and w U  then

 P  v  P  P v  P u  u  P v.
U
2
U U U U

Hence PU 2  PU .

(h) If v  u  w with u U and w U  , then

 41

PU v  u  u  w  v
2 2 2 2 2

where the last equation comes from the Pythagorean Theorem.

(i) For every v V we write
v  v, e1 e1   v, en en  v  v, e1 e1   v, en en . (3.9)
Then the sum of first n terms of (3.9) is in U. Since
v, e1 e1   v, en en , v  v, e1 e1   v, en en

 v, e1 e1 , v   v, en en , v  v, e1 v, e1   v, en v, en  0,

the sum of the last n  1 terms of (3.9) is in U  . Thus

PU v  v, e1 e1   v, en en .

3.3.9 Theorem If P  L V  is such that P 2  P and every vector in null P is

orthogonal to very vector in range P , then P is orthogonal Projection.
Proof. Suppose P  L V  is such that P 2  P and every vector in null P is
orthogonal to every vector in range P . Let U  range P . We will show that P
equals the orthogonal projection PU . To do this, suppose v V .Then

v  Pv   v  Pv  .

Clearly, Pv  range P  U . Also, P  v  Pv   Pv  P 2v  0, which means that

v  Pv  null P . Thus v  Pv is orthogonal to every vector in U . In other words,
v  Pv  P  . Thus the equation above writes V as the sum of a vector in U and a
vector in U  . In this decomposition, the vector in U equals, by definition, PU v .
Hence PU v  Pv so that PU  P . 
We find the distance from a point v V to a given subspace U of V. That is, we
find a point u in U such that v  u is as small as possible.

3.3.10 Theorem Suppose U is a finite dimensional subspace of V, v V and u U .

Then
v  PU v  v  u .

Furthermore, the inequality above is an equality if and only if u  PU v.

Proof. We have

v  PU v  v  PU v  PU v  u
2 2 2
(3.10)
 42

 v  PU v  PU v  u
2
by Pythagorean Theorem

and hence v  PU v  v  u .

The inequality in  3.10  is equality  PU v  u  0

 PU v  u, PU v  u  0

 PU v  u  0. 

O
Figure 2.1 is the closest point in U to v

3.3.11 Example In 2
, let U  span  1,1 ,  2,1  . We find u U such that
u  1,5  is as small as possible. First, we find orthonormal basis of U by applying
the Gram-Schmidt Procedure to 1,1 ,  2,1 , getting

 1 1   1 1 
e1   ,  , e2   , .
 2 2  2 2
Thus e1 , e2 is an orthonormal basis of U. By Theorem 3.3.10 and Theorem 3.3.8(i),
the closest point u U to 1,5 is

1,5 , e1 e1  1,5  , e2 e2 ,

which equals 1,5  .

3.3.12 Example In 4
, let U  span  1,1, 0, 0  , 1,1,1, 2   . We find u U such that
u  1, 2,3, 4  is as small as possible. First, we find orthonormal basis of U by
applying the Gram-Schmidt Procedure to 1,1, 0, 0  , 1,1,1, 2  getting

 1 1   1 2 
e1   , , 0, 0  , e2   0, 0, , .
 2 2   5 5
 43

Thus e1 , e2 is an orthonormal basis of U. By Theorem 3.3.10 and Theorem 3.3.8(i),

the closest point u U to 1, 2,3, 4  is

1, 2,3, 4  , e1 e1  1, 2,3, 4  , e2 e2 ,

 3 3 11 22 
which equals  , , ,  .
2 2 5 5 