MA102 Linear Algebra

Lecture-1: Vectors in Rn and Cn

RA, RKS, MGPP, BD

Department of Mathematics
IIT Guwahati

March-June 2023

1 / 19
Outline

Topics:
The vector spaces Rn and Cn
Inner product, angle, and orthogonality

2 / 19
Vectors in Rn
We define Rn to be the set of all ordered n-tuples of real numbers. Thus an n-tuple in
Rn (called n-vector) is of the form
 
v1
 ..
row vector: ~v = [v1 , . . . , vn ] or column vector: v =  .


vn

3 / 19
Vectors in Rn
We define Rn to be the set of all ordered n-tuples of real numbers. Thus an n-tuple in
Rn (called n-vector) is of the form
 
v1
 ..
row vector: ~v = [v1 , . . . , vn ] or column vector: v =  .


vn
We always write an n-vector in Rn as a column vector. Thus
  
 v1
 

R :=  ...
n
 : v1 , . . . , vn ∈ R .
 
 
vn
 

3 / 19
Vectors in Rn
We define Rn to be the set of all ordered n-tuples of real numbers. Thus an n-tuple in
Rn (called n-vector) is of the form
 
v1
 ..
row vector: ~v = [v1 , . . . , vn ] or column vector: v =  .


vn
We always write an n-vector in Rn as a column vector. Thus
  
 v1
 

R :=  ...
n
 : v1 , . . . , vn ∈ R .
 
 
vn
 

   >
v1 v1
Transpose: [v1 , . . . , vn ]> =  ...  and  ...  = [v1 , . . . , vn ].
   

vn vn
3 / 19
Vectors in Cn
We define Cn to be the set of all ordered n-tuples of complex numbers. Thus an
n-tuple in Cn (called n-vector) is of the form
 
v1
row vector: ~v = [v1 , . . . , vn ] or column vector: v =  ... 
 

vn

4 / 19
Vectors in Cn
We define Cn to be the set of all ordered n-tuples of complex numbers. Thus an
n-tuple in Cn (called n-vector) is of the form
 
v1
row vector: ~v = [v1 , . . . , vn ] or column vector: v =  ... 
 

vn
We always write an n-vector in Cn as column vector. Thus
  
 v1
 

n  .. 
C :=  .  : v1 , . . . , vn ∈ C
 
vn
 

4 / 19
Vectors in Cn
We define Cn to be the set of all ordered n-tuples of complex numbers. Thus an
n-tuple in Cn (called n-vector) is of the form
 
v1
row vector: ~v = [v1 , . . . , vn ] or column vector: v =  ... 
 

vn
We always write an n-vector in Cn as column vector. Thus
  
 v1
 

n  .. 
C :=  .  : v1 , . . . , vn ∈ C
 
vn
 

Conjugate transpose:   Here z̄ ∗ complex conjugate of z ∈ C.

is the
v̄1 v1
[v1 , . . . , vn ]∗ =  ...  and  ..  = [v̄ , . . . , v̄ ].
  
.  1 n
v̄n vn
4 / 19
Algebraic properties of vectors in Fn
Define addition and scalar multiplication on Fn (F = R or C) as follows:
         
u1 v1 u1 + v1 u1 αu1
 ..   ..   ..  .   . 
 . + . =  and α  ..  =  ..  for α ∈ F.

.
un vn un + vn un αun

5 / 19
Algebraic properties of vectors in Fn
Define addition and scalar multiplication on Fn (F = R or C) as follows:
         
u1 v1 u1 + v1 u1 αu1
 ..   ..   ..  .   . 
 . + . =  and α  ..  =  ..  for α ∈ F.

.
un vn un + vn un αun
This produces new vectors from old vectors.

5 / 19
Algebraic properties of vectors in Fn
Define addition and scalar multiplication on Fn (F = R or C) as follows:
         
u1 v1 u1 + v1 u1 αu1
 ..   ..   ..  .   . 
 . + . =  and α  ..  =  ..  for α ∈ F.

.
un vn un + vn un αun
This produces new vectors from old vectors. For u, v, w in Fn and scalars α, β in F,
the following hold:
1 Commutativity: u + v = v + u

2 Associativity: (u + v) + w = u + (v + w)

3 Identity: u + 0 = u

4 Inverse: u + (−u) = 0

5 / 19
Algebraic properties of vectors in Fn
Define addition and scalar multiplication on Fn (F = R or C) as follows:
         
u1 v1 u1 + v1 u1 αu1
 ..   ..   ..  .   . 
 . + . =  and α  ..  =  ..  for α ∈ F.

.
un vn un + vn un αun
This produces new vectors from old vectors. For u, v, w in Fn and scalars α, β in F,
the following hold:
1 Commutativity: u + v = v + u

2 Associativity: (u + v) + w = u + (v + w)

3 Identity: u + 0 = u

4 Inverse: u + (−u) = 0

5 Distributivity : α(u + v) = αu + αv

6 Distributivity : (α + β)u = αu + βu

7 Associativity: α(βu) = (αβ)u

8 Identity: 1u = u.

5 / 19
Examples of vectors
Standard
 vectors: The vectors
> >  >
e1 := 1 0 · · · 0 , e2 := 0 1 0 · · · 0 , . . . , en := 0 · · · 0 1 are
called standard vectors or canonical vectors in Rn and Cn .

6 / 19
Examples of vectors
Standard
 vectors: The vectors
> >  >
e1 := 1 0 · · · 0 , e2 := 0 1 0 · · · 0 , . . . , en := 0 · · · 0 1 are
called standard vectors or canonical vectors in Rn and Cn .
Features vectors. A feature vector collects together n different quantities that pertain
to a single thing or object. The entries of a feature vector are called the features or
attributes.

6 / 19
Examples of vectors
Standard
 vectors: The vectors
> >  >
e1 := 1 0 · · · 0 , e2 := 0 1 0 · · · 0 , . . . , en := 0 · · · 0 1 are
called standard vectors or canonical vectors in Rn and Cn .
Features vectors. A feature vector collects together n different quantities that pertain
to a single thing or object. The entries of a feature vector are called the features or
attributes.
For instance, a 5-vector x := [x1 , x2 , x3 , x4 , x5 ]> could give the age, height, weight,
blood pressure, and temperature of a patient admitted to a hospital.

6 / 19
Examples of vectors
Standard
 vectors: The vectors
> >  >
e1 := 1 0 · · · 0 , e2 := 0 1 0 · · · 0 , . . . , en := 0 · · · 0 1 are
called standard vectors or canonical vectors in Rn and Cn .
Features vectors. A feature vector collects together n different quantities that pertain
to a single thing or object. The entries of a feature vector are called the features or
attributes.
For instance, a 5-vector x := [x1 , x2 , x3 , x4 , x5 ]> could give the age, height, weight,
blood pressure, and temperature of a patient admitted to a hospital.
Feature differences. If x and y are n-vectors that give n feature values for two items
then the difference d := x − y gives the difference in feature values for the two objects.

6 / 19
Examples of vectors
Standard
 vectors: The vectors
> >  >
e1 := 1 0 · · · 0 , e2 := 0 1 0 · · · 0 , . . . , en := 0 · · · 0 1 are
called standard vectors or canonical vectors in Rn and Cn .
Features vectors. A feature vector collects together n different quantities that pertain
to a single thing or object. The entries of a feature vector are called the features or
attributes.
For instance, a 5-vector x := [x1 , x2 , x3 , x4 , x5 ]> could give the age, height, weight,
blood pressure, and temperature of a patient admitted to a hospital.
Feature differences. If x and y are n-vectors that give n feature values for two items
then the difference d := x − y gives the difference in feature values for the two objects.
Word count vector. An n-vector w can represent the number of times each word in a
dictionary of n words appears in a document.

6 / 19
Examples of vectors
Standard
 vectors: The vectors
> >  >
e1 := 1 0 · · · 0 , e2 := 0 1 0 · · · 0 , . . . , en := 0 · · · 0 1 are
called standard vectors or canonical vectors in Rn and Cn .
Features vectors. A feature vector collects together n different quantities that pertain
to a single thing or object. The entries of a feature vector are called the features or
attributes.
For instance, a 5-vector x := [x1 , x2 , x3 , x4 , x5 ]> could give the age, height, weight,
blood pressure, and temperature of a patient admitted to a hospital.
Feature differences. If x and y are n-vectors that give n feature values for two items
then the difference d := x − y gives the difference in feature values for the two objects.
Word count vector. An n-vector w can represent the number of times each word in a
dictionary of n words appears in a document.
For instance, the word count vector [25, 2, 0]> means that the first dictionary word
appears 25 times, the second one twice, and the third one not at all.
6 / 19
Inner product
Angle, Length, and Distance can all be described by using the notion of inner product
(dot product) of two n-vectors.

Definition: If u := [u1 , . . . , un ]> and v := [v1 , . . . , vn ]> are n-vectors then the inner
product hu, vi is defined by

hu, vi := u1 v1 + u2 v2 + · · · + un vn when u, v ∈ Rn

7 / 19
Inner product
Angle, Length, and Distance can all be described by using the notion of inner product
(dot product) of two n-vectors.

Definition: If u := [u1 , . . . , un ]> and v := [v1 , . . . , vn ]> are n-vectors then the inner
product hu, vi is defined by

hu, vi := u1 v1 + u2 v2 + · · · + un vn when u, v ∈ Rn
hu, vi := u1 v̄1 + u2 v̄2 + · · · + un v̄n when u, v ∈ Cn .

The inner product hu, vi is also called dot product and is written as u • v.

7 / 19
Inner product
Angle, Length, and Distance can all be described by using the notion of inner product
(dot product) of two n-vectors.

Definition: If u := [u1 , . . . , un ]> and v := [v1 , . . . , vn ]> are n-vectors then the inner
product hu, vi is defined by

hu, vi := u1 v1 + u2 v2 + · · · + un vn when u, v ∈ Rn
hu, vi := u1 v̄1 + u2 v̄2 + · · · + un v̄n when u, v ∈ Cn .

The inner product hu, vi is also called dot product and is written as u • v.

Example: If u := [1, 2, −3]> and v := [−3, 5, 2]> then

hu, vi = 1 · (−3) + 2 · 5 + (−3) · 2 = 1.

7 / 19
Inner product
f1 · · · fn ∈ Rn be a feature vector of an
 
Weights, features, and score. Let
 f :=
object and w := w1 · · · wn ∈ Rn be a weight vector. Then the inner product

hf, wi = w1 f1 + · · · + wn fn

is the sum of the feature values, scaled by the weights, and is called a score.

8 / 19
Inner product
f1 · · · fn ∈ Rn be a feature vector of an
 
Weights, features, and score. Let
 f :=
object and w := w1 · · · wn ∈ Rn be a weight vector. Then the inner product

hf, wi = w1 f1 + · · · + wn fn

is the sum of the feature values, scaled by the weights, and is called a score.

Examples:
Credit score: Let f be a feature vector associated with a loan applicant (e.g., age,
income, . . . ). Then we might interpret hf, wi as a credit score, where wi is the
weight given to feature fi in forming the score.

8 / 19
Inner product
f1 · · · fn ∈ Rn be a feature vector of an
 
Weights, features, and score. Let
 f :=
object and w := w1 · · · wn ∈ Rn be a weight vector. Then the inner product

hf, wi = w1 f1 + · · · + wn fn

is the sum of the feature values, scaled by the weights, and is called a score.

Examples:
Credit score: Let f be a feature vector associated with a loan applicant (e.g., age,
income, . . . ). Then we might interpret hf, wi as a credit score, where wi is the
weight given to feature fi in forming the score.
Co-occurrence. Let x and y be Boolean n-vectors (each entry is either 0 or 1) that
describe occurrence. Then the inner product hx, yi gives the total number of
co-occurrences.

8 / 19
Inner product
f1 · · · fn ∈ Rn be a feature vector of an
 
Weights, features, and score. Let
 f :=
object and w := w1 · · · wn ∈ Rn be a weight vector. Then the inner product

hf, wi = w1 f1 + · · · + wn fn

is the sum of the feature values, scaled by the weights, and is called a score.

Examples:
Credit score: Let f be a feature vector associated with a loan applicant (e.g., age,
income, . . . ). Then we might interpret hf, wi as a credit score, where wi is the
weight given to feature fi in forming the score.
Co-occurrence. Let x and y be Boolean n-vectors (each entry is either 0 or 1) that
describe occurrence. Then the inner product hx, yi gives the total number of
co-occurrences.
For x := [0, 1, 1, 1, 1, 1, 1]> and y := [1, 0, 1, 0, 1, 0, 0]> , we have hx, yi = 2, which
is the number of common occurrences.
8 / 19
Properties of inner product
Theorem: Let u, v, and w be vectors in Cn and let α ∈ C. Then
1 hu, ui ≥ 0 and hu, ui = 0 ⇐⇒ u = 0.

2 hu, vi = hv, ui

3 hu + v, wi = hu, wi + hv, wi

4 hαu, vi = αhu, vi.

9 / 19
Properties of inner product
Theorem: Let u, v, and w be vectors in Cn and let α ∈ C. Then
1 hu, ui ≥ 0 and hu, ui = 0 ⇐⇒ u = 0.

2 hu, vi = hv, ui

3 hu + v, wi = hu, wi + hv, wi

4 hαu, vi = αhu, vi.

Definition: The norm (or length) of a vector v := [v1 , . . . , vn ]> in Cn is a nonnegative

number kvk defined by
p q
kvk := hv, vi = |v1 |2 + · · · + |vn |2 .

9 / 19
Properties of inner product
Theorem: Let u, v, and w be vectors in Cn and let α ∈ C. Then
1 hu, ui ≥ 0 and hu, ui = 0 ⇐⇒ u = 0.

2 hu, vi = hv, ui

3 hu + v, wi = hu, wi + hv, wi

4 hαu, vi = αhu, vi.

Definition: The norm (or length) of a vector v := [v1 , . . . , vn ]> in Cn is a nonnegative

number kvk defined by
p q
kvk := hv, vi = |v1 |2 + · · · + |vn |2 .

Theorem (Cauchy-Schwarz Inequality): Let u and v be n-vectors. Then

|hu, vi| ≤ kuk kvk.

Proof for Rn : p(t) := ku + tvk2 = kuk2 + 2thu, vi + kvk2 t 2 ≥ 0 for all t ∈ R. Hence
discriminant of p(t) is non-positive which yields the result. 
9 / 19
Properties of Norm
Theorem: Let u and v be vectors in Cn and let α ∈ C. Then
1 Positive definite: kuk ≥ 0 and kuk = 0 ⇐⇒ u = 0

2 Positive homogeneity: kαuk = |α| kuk

3 Triangle inequality: ku + vk ≤ kuk + kvk.

10 / 19
Properties of Norm
Theorem: Let u and v be vectors in Cn and let α ∈ C. Then
1 Positive definite: kuk ≥ 0 and kuk = 0 ⇐⇒ u = 0

2 Positive homogeneity: kαuk = |α| kuk

3 Triangle inequality: ku + vk ≤ kuk + kvk.

Proof: Using Cauchy-Schwarz in equality, we have

ku + vk2 = kuk2 + 2 Re(hu, vi) + kvk2 ≤ kuk2 + 2|hu, vi| + kvk2

≤ kuk2 + 2kuk kvk + kvk2 = (kuk + kvk)2 .
10 / 19
Unit vectors
Definition: A vector v in Cn or Rn is called a unit vector if kvk = 1. If u is a nonzero
1
vector then v := kuk u is a unit vector in the direction of u.

11 / 19
Unit vectors
Definition: A vector v in Cn or Rn is called a unit vector if kvk = 1. If u is a nonzero
1
vector then v := kuk u is a unit vector in the direction of u. Indeed,
1
kvk = k(1/kuk)uk = kuk = 1.
kuk
The vector v is referred to as a normalization of u.

11 / 19
Unit vectors
Definition: A vector v in Cn or Rn is called a unit vector if kvk = 1. If u is a nonzero
1
vector then v := kuk u is a unit vector in the direction of u. Indeed,
1
kvk = k(1/kuk)uk = kuk = 1.
kuk
The vector v is referred to as a normalization of u.
 
2 √ √
Example: Let u :=  −1 . Then kuk = 4 + 1 + 9 = 14 and
3
   √ 
2 2/√14
1 1
v := √ u = √  −1  =  −1/√14  .
14 14 3 3/ 14

11 / 19
Unit vectors
Definition: A vector v in Cn or Rn is called a unit vector if kvk = 1. If u is a nonzero
1
vector then v := kuk u is a unit vector in the direction of u. Indeed,
1
kvk = k(1/kuk)uk = kuk = 1.
kuk
The vector v is referred to as a normalization of u.
 
2 √ √
Example: Let u :=  −1 . Then kuk = 4 + 1 + 9 = 14 and
3
   √ 
2 2/√14
1 1
v := √ u = √  −1  =  −1/√14  .
14 14 3 3/ 14
Standard unit vectors: The vectors e1 := [1, 0, 0]> , e2 := [0, 1, 0]> and e3 := [0, 0, 1]>
are unit vectors in R3 and are called standard unit vectors. The canonical vectors
e1 , e2 , . . . , en in Rn are standard unit vectors.
11 / 19
Distance
Distance: The distance d(u, v) between two vectors u := [u1 , . . . , un ]> and
v := [v1 , . . . , vn ]> in Rn or Cn is defined by
q
d(u, v) := ku − vk = |u1 − v1 |2 + · · · + |un − vn |2 .

12 / 19
Angle between two vectors in R2

Consider the triangle in R2 with sides u, v and u − v. Let θ be the angle between u and
v. Then by the law of cosines there is unique θ ∈ [0, π] such that
ku − vk2 = kuk2 + kvk2 − 2kuk kvk cos θ.

13 / 19
Angle between two vectors in R2

Consider the triangle in R2 with sides u, v and u − v. Let θ be the angle between u and
v. Then by the law of cosines there is unique θ ∈ [0, π] such that
ku − vk2 = kuk2 + kvk2 − 2kuk kvk cos θ.
Expanding ku − vk2 = kuk2 − 2hu, vi + kvk2 gives us
hu, vi
hu, vi = kuk kvk cos θ =⇒ cos θ = .
kuk kvk

13 / 19
Angle between two n-vectors
Definition: Let u and v be nonzero n-vectors. Then
hu, vi
cos θ := =⇒ hu, vi = kuk kvk cos θ for θ ∈ [0, π] when u, v ∈ Rn
kuk kvk

14 / 19
Angle between two n-vectors
Definition: Let u and v be nonzero n-vectors. Then
hu, vi
cos θ := =⇒ hu, vi = kuk kvk cos θ for θ ∈ [0, π] when u, v ∈ Rn
kuk kvk
|hu, vi| π
cos θ := =⇒ |hu, vi| = kuk kvk cos θ for θ ∈ [0, ] when u, v ∈ Cn .
kuk kvk 2

14 / 19
Angle between two n-vectors
Definition: Let u and v be nonzero n-vectors. Then
hu, vi
cos θ := =⇒ hu, vi = kuk kvk cos θ for θ ∈ [0, π] when u, v ∈ Rn
kuk kvk
|hu, vi| π
cos θ := =⇒ |hu, vi| = kuk kvk cos θ for θ ∈ [0, ] when u, v ∈ Cn .
kuk kvk 2
   
1 0
Example: Let u := 0 and v := 1 . Then hu, vi = 1 · 0 + 0 · 1 + 1 · 1 = 1.
  
1 1

14 / 19
Angle between two n-vectors
Definition: Let u and v be nonzero n-vectors. Then
hu, vi
cos θ := =⇒ hu, vi = kuk kvk cos θ for θ ∈ [0, π] when u, v ∈ Rn
kuk kvk
|hu, vi| π
cos θ := =⇒ |hu, vi| = kuk kvk cos θ for θ ∈ [0, ] when u, v ∈ Cn .
kuk kvk 2
   
1 0
Example: Let u := 0 and v := 1 . Then hu, vi = 1 · 0 + 0 · 1 + 1 · 1 = 1. We have
  
1 1
√ √ √ √
kuk = 1 + 1 = 2 and kvk = 1 + 1 = 2. Hence

hu, vi 1 1
cos θ = = √ √ = =⇒ θ = π/3 radians. 
kuk kvk 2 2 2

14 / 19
Orthogonal vectors
Definition: Two n-vectors u and v are said to be orthogonal to each other if
hu, vi = 0. We write u ⊥ v when hu, vi = 0.

15 / 19
Orthogonal vectors
Definition: Two n-vectors u and v are said to be orthogonal to each other if
hu, vi = 0. We write u ⊥ v when hu, vi = 0. If, in addition, kuk = kvk = 1 then u and
v are called orthonormal.

15 / 19
Orthogonal vectors
Definition: Two n-vectors u and v are said to be orthogonal to each other if
hu, vi = 0. We write u ⊥ v when hu, vi = 0. If, in addition, kuk = kvk = 1 then u and
v are called orthonormal.
Remark: The zero vector 0 is orthogonal to all vectors in Rn as h0, vi = 0 for all
v ∈ Rn .

15 / 19
Orthogonal vectors
Definition: Two n-vectors u and v are said to be orthogonal to each other if
hu, vi = 0. We write u ⊥ v when hu, vi = 0. If, in addition, kuk = kvk = 1 then u and
v are called orthonormal.
Remark: The zero vector 0 is orthogonal to all vectors in Rn as h0, vi = 0 for all
v ∈ Rn .
Example: The vectors u := [1, 1, −2]> and v := [3, 1, 2]> in R3 are orthogonal as
hu, vi = 1 · 3 + 1 · 1 + (−2) · 2 = 0.

15 / 19
Orthogonal vectors
Definition: Two n-vectors u and v are said to be orthogonal to each other if
hu, vi = 0. We write u ⊥ v when hu, vi = 0. If, in addition, kuk = kvk = 1 then u and
v are called orthonormal.
Remark: The zero vector 0 is orthogonal to all vectors in Rn as h0, vi = 0 for all
v ∈ Rn .
Example: The vectors u := [1, 1, −2]> and v := [3, 1, 2]> in R3 are orthogonal as
hu, vi = 1 · 3 + 1 · 1 + (−2) · 2 = 0.
Pythagoras’ Theorem: Let u and v be n-vectors. Then

hu, vi = 0 ⇐⇒ ku + vk2 = kuk2 + kvk2 when u, v ∈ Rn

15 / 19
Orthogonal vectors
Definition: Two n-vectors u and v are said to be orthogonal to each other if
hu, vi = 0. We write u ⊥ v when hu, vi = 0. If, in addition, kuk = kvk = 1 then u and
v are called orthonormal.
Remark: The zero vector 0 is orthogonal to all vectors in Rn as h0, vi = 0 for all
v ∈ Rn .
Example: The vectors u := [1, 1, −2]> and v := [3, 1, 2]> in R3 are orthogonal as
hu, vi = 1 · 3 + 1 · 1 + (−2) · 2 = 0.
Pythagoras’ Theorem: Let u and v be n-vectors. Then

hu, vi = 0 ⇐⇒ ku + vk2 = kuk2 + kvk2 when u, v ∈ Rn

hu, vi = 0 =⇒ ku + vk2 = kuk2 + kvk2 when u, v ∈ Cn

15 / 19
Orthogonal vectors
Definition: Two n-vectors u and v are said to be orthogonal to each other if
hu, vi = 0. We write u ⊥ v when hu, vi = 0. If, in addition, kuk = kvk = 1 then u and
v are called orthonormal.
Remark: The zero vector 0 is orthogonal to all vectors in Rn as h0, vi = 0 for all
v ∈ Rn .
Example: The vectors u := [1, 1, −2]> and v := [3, 1, 2]> in R3 are orthogonal as
hu, vi = 1 · 3 + 1 · 1 + (−2) · 2 = 0.
Pythagoras’ Theorem: Let u and v be n-vectors. Then

hu, vi = 0 ⇐⇒ ku + vk2 = kuk2 + kvk2 when u, v ∈ Rn

hu, vi = 0 =⇒ ku + vk2 = kuk2 + kvk2 when u, v ∈ Cn

Proof: We have ku + vk2 = kuk2 + 2 hu, vi + kvk2 = kuk2 + kvk2 ⇐⇒ hu, vi = 0

when u, v ∈ Rn .

15 / 19
Orthogonal vectors
Definition: Two n-vectors u and v are said to be orthogonal to each other if
hu, vi = 0. We write u ⊥ v when hu, vi = 0. If, in addition, kuk = kvk = 1 then u and
v are called orthonormal.
Remark: The zero vector 0 is orthogonal to all vectors in Rn as h0, vi = 0 for all
v ∈ Rn .
Example: The vectors u := [1, 1, −2]> and v := [3, 1, 2]> in R3 are orthogonal as
hu, vi = 1 · 3 + 1 · 1 + (−2) · 2 = 0.
Pythagoras’ Theorem: Let u and v be n-vectors. Then

hu, vi = 0 ⇐⇒ ku + vk2 = kuk2 + kvk2 when u, v ∈ Rn

hu, vi = 0 =⇒ ku + vk2 = kuk2 + kvk2 when u, v ∈ Cn

Proof: We have ku + vk2 = kuk2 + 2 hu, vi + kvk2 = kuk2 + kvk2 ⇐⇒ hu, vi = 0

when u, v ∈ Rn . We have ku + vk2 = kuk2 + 2 Re(hu, vi) + kvk2 when u, v ∈ Cn .

15 / 19
Orthogonal vectors
Definition: Two n-vectors u and v are said to be orthogonal to each other if
hu, vi = 0. We write u ⊥ v when hu, vi = 0. If, in addition, kuk = kvk = 1 then u and
v are called orthonormal.
Remark: The zero vector 0 is orthogonal to all vectors in Rn as h0, vi = 0 for all
v ∈ Rn .
Example: The vectors u := [1, 1, −2]> and v := [3, 1, 2]> in R3 are orthogonal as
hu, vi = 1 · 3 + 1 · 1 + (−2) · 2 = 0.
Pythagoras’ Theorem: Let u and v be n-vectors. Then

hu, vi = 0 ⇐⇒ ku + vk2 = kuk2 + kvk2 when u, v ∈ Rn

hu, vi = 0 =⇒ ku + vk2 = kuk2 + kvk2 when u, v ∈ Cn

Proof: We have ku + vk2 = kuk2 + 2 hu, vi + kvk2 = kuk2 + kvk2 ⇐⇒ hu, vi = 0

when u, v ∈ Rn . We have ku + vk2 = kuk2 + 2 Re(hu, vi) + kvk2 when u, v ∈ Cn .
Hence hu, vi = 0 =⇒ ku + vk2 = kuk2 + kvk2 . 
15 / 19
Vectors in information retrieval
Problem: Given a few key words, retrieve relevant information from a large
database.

16 / 19
Vectors in information retrieval
Problem: Given a few key words, retrieve relevant information from a large
database.

Document vectors: Document vectors are used in information retrieval. Consider

the five documents.

16 / 19
Vectors in information retrieval
Problem: Given a few key words, retrieve relevant information from a large
database.

Document vectors: Document vectors are used in information retrieval. Consider

the five documents.
Doc. 1: The Google matrix G is a model of the Internet.
Doc. 2: Gij is nonzero if there is a link from web page j to i.
Doc. 3: The Google matrix G is used to rank all web pages.
Doc. 4: The ranking is done by solving a matrix eigenvalue
problem.
Doc. 5: England dropped out of the top 10 in the FIFA ranking.

16 / 19
Vectors in information retrieval
Problem: Given a few key words, retrieve relevant information from a large
database.

Document vectors: Document vectors are used in information retrieval. Consider

the five documents.
Doc. 1: The Google matrix G is a model of the Internet.
Doc. 2: Gij is nonzero if there is a link from web page j to i.
Doc. 3: The Google matrix G is used to rank all web pages.
Doc. 4: The ranking is done by solving a matrix eigenvalue
problem.
Doc. 5: England dropped out of the top 10 in the FIFA ranking.

The blue colored texts are the key words or terms. The set of terms is called a
Dictionary. Counting the frequency of terms in each document, we obtain
document vectors.
16 / 19
Term-document matrix
Term Doc. 1 Doc. 2 Doc. 3 Doc. 4 Doc. 5
eigenvalue 0 0 0 1 0
England 0 0 0 0 1
FIFA 0 0 0 0 1
Google 1 0 1 0 0
Internet 1 0 0 0 0
link 0 1 0 0 0
matrix 1 0 1 1 0
page 0 1 1 0 0
rank 0 0 1 1 1
web 0 1 1 0 1

17 / 19
Term-document matrix
Term Doc. 1 Doc. 2 Doc. 3 Doc. 4 Doc. 5
eigenvalue 0 0 0 1 0
England 0 0 0 0 1
FIFA 0 0 0 0 1
Google 1 0 1 0 0
Internet 1 0 0 0 0
link 0 1 0 0 0
matrix 1 0 1 1 0
page 0 1 1 0 0
rank 0 0 1 1 1
web 0 1 1 0 1

Each document is a vector in R10 and is represented by a column of the

term-document matrix.

17 / 19
Query vector

Suppose that we want to find all documents that are relevant to the query
ranking of web pages. This is represented by a query vector, constructed in the
way as the document vectors, using the same dictionary:
>
v := 0 0 0 0 0 0 0 1 1 1 ∈ R10 .


18 / 19
Query vector

Suppose that we want to find all documents that are relevant to the query
ranking of web pages. This is represented by a query vector, constructed in the
way as the document vectors, using the same dictionary:
>
v := 0 0 0 0 0 0 0 1 1 1 ∈ R10 .


Thus the query itself is a document. The information retrieval task can now be
formulated as a mathematical problem.

18 / 19
Query vector

Suppose that we want to find all documents that are relevant to the query
ranking of web pages. This is represented by a query vector, constructed in the
way as the document vectors, using the same dictionary:
>
v := 0 0 0 0 0 0 0 1 1 1 ∈ R10 .


Thus the query itself is a document. The information retrieval task can now be
formulated as a mathematical problem.

Problem: Find the document vectors (columns of the term of document matrix)
that are close (in some sense) to the query vector v.

18 / 19
Query matching (use of dot product)
Query matching is the process of finding all documents that are relevant to a
particular query v. The cosine of angle between two vectors is often used to
determine relevant documents:
|hdj , vi|
cos θj := > tol
kvk kdj k

where dj is the j-th document vector (j-th column of the term-document matrix)
and tol is a predefined tolerance. Thus cos θj > tol ⇒ dj is relevant.

19 / 19
Query matching (use of dot product)
Query matching is the process of finding all documents that are relevant to a
particular query v. The cosine of angle between two vectors is often used to
determine relevant documents:
|hdj , vi|
cos θj := > tol
kvk kdj k

where dj is the j-th document vector (j-th column of the term-document matrix)
and tol is a predefined tolerance. Thus cos θj > tol ⇒ dj is relevant.

For the document vectors d1 , . . . , d5 and the query (”ranking of web pages”)
vector v, the cosines measures of the query and the original data are given by

[0, 0.6667, 0.7746, 0.3333, 0.3333]>

which shows that Doc 2 and Doc 3 are most relevant.

19 / 19