QMT581: MULTIVARIATE METHODS

CHAPTER 1: Data Array and Matrix

Algebra
Ida Rosmini Othman
rosmini@tmsk.uitm.edu.my

Department of Statistics Ida Rosmini Othman 1

Application of Multivariate Techniques

WHAT IS MULTIVARIATE
ANALYSIS
Statistical methodology to analyze data
with measurements on many variables.
controllable factors

"
input "
output
Process

"
uncontrollable factors

Department of Statistics Ida Rosmini Othman 2

1
 Application of Multivariate Techniques

THE NEED TO STUDY

MULTIVARIATE

Explanation of social or physical

phenomenon must be tested by gathering
and analyzing data.
Complexities of most phenomena require
an investigator to collect observations on
many different variables.

Department of Statistics Ida Rosmini Othman 3

Application of Multivariate Techniques

APPLICATION EXAMPLES

Is one product better than the other?

Which factor is the most important to
determine the performance of a system?
How to classify the results into clusters?
What are the relationships between
variables?

Department of Statistics Ida Rosmini Othman 4

2
 Application of Multivariate Techniques

MAJOR USES OF MULTIVARIATE

ANALYSIS

Data reduction or structural simplification.

Sorting and grouping.
Investigation of the dependence among
variables.
Prediction.
Hypothesis construction and testing.

Department of Statistics Ida Rosmini Othman 5

The Organization of Data

ARRAY
Xjk = measurement of the kth variable on
the jth item.
Var 1 Var 2 … Var k … Var p
Item 1 x11 x12 … x1k … x1p
Item 2 x21 x22 … x2k … x2p
…

… …
Item j xj 1 xj 2 … xjk … xjp
…

… …
Item n xn 1 xn 2 … xnk … xnp
Department of Statistics Ida Rosmini Othman 6

3
 The Organization of Data

ARRAY

x11 x12 … x1k … x1p

x21
… x22 … x2k … x2p

…
X = … …
xj 1 xj 2 … xjk … xjp
…

…
… …
xn 1 xn 2 … xnk … xnp

Department of Statistics Ida Rosmini Othman 7

The Organization of Data

ARRAY

Example 1.1: Four receipts from a university.

Variable 1 (dollar sales): 42 52 48 58
Variable 2 (number of books): 4 5 4 3

x11= 42 x21= 52 x31= 48 x41= 58 42 4

x12= 4 x22= 5 x32= 4 x42= 3 X = 52 5
48 4
58 3

Department of Statistics Ida Rosmini Othman 8

4
 Some Basics of Matrix and Vector Algebra

VECTORS
Can be transpose

⎡ x1 ⎤
⎢x ⎥
x = ⎢ 2⎥
⎢#⎥
⎢ ⎥
⎢⎣ xn ⎥⎦

x' = [ x1 , x2 ," , xn ]

Department of Statistics Ida Rosmini Othman 9

Some Basics of Matrix and Vector Algebra

VECTORS
Can be expanded
through constant
multiplication

⎡ cx1 ⎤
⎢cx ⎥
cx = ⎢ 2 ⎥
⎢ # ⎥
⎢ ⎥
⎣⎢cxn ⎦⎥

Department of Statistics Ida Rosmini Othman 10

5
 Some Basics of Matrix and Vector Algebra

VECTORS

Can be added

⎡ x1 ⎤ ⎡ y1 ⎤
⎢x ⎥ ⎢ y ⎥
x+ y = ⎢ 2⎥ + ⎢ 2⎥
⎢#⎥ ⎢#⎥
⎢ ⎥ ⎢ ⎥
⎢⎣ xn ⎥⎦ ⎢⎣ yn ⎥⎦
⎡ x1 + y1 ⎤
⎢x + y ⎥
=⎢ 2 2⎥
⎢ # ⎥
⎢ ⎥
⎢⎣ xn + yn ⎥⎦

Department of Statistics Ida Rosmini Othman 11

Some Basics of Matrix and Vector Algebra

VECTORS

Has both direction (positive & negative)

and length

Department of Statistics Ida Rosmini Othman 12

6
 Some Basics of Matrix and Vector Algebra

VECTORS

Length : Lx = x12 + x22

Length : Lx = x12 + x22 +"+ xn2 = x' x

Department of Statistics Ida Rosmini Othman 13

Some Basics of Matrix and Vector Algebra

VECTORS

Multiply vector X with scalar, c changes

a) length

Length : Lx = cx12 + cx22 + "+ cxn2 = c Lx

b) direction if c<0

Department of Statistics Ida Rosmini Othman 14

7
 Some Basics of Matrix and Vector Algebra

VECTORS : ANGLE

x, y
cos (θ ) =
Lx L y
x, y
=
x' x y ' y

Department of Statistics Ida Rosmini Othman 15

Some Basics of Matrix and Vector Algebra

EXAMPLE 2.1

x ' = [1, 3, 2 ] an d y ' = [ − 2,1, − 1 ]

Find:
a) 3x d) Length of y
b) x + y e) Angle between x and y
c) Length of x f) Check that length of 3x
is 3 times the length of x

Department of Statistics Ida Rosmini Othman 16

8
 Some Basics of Matrix and Vector Algebra

EXAMPLE 2.1

⎡1 ⎤ ⎡3⎤
a) 3x = 3 ⎢ 3 ⎥ = ⎢9 ⎥
⎢ ⎥ ⎢ ⎥
⎢⎣ 2 ⎥⎦ ⎢⎣ 6 ⎥⎦

⎡ 1 ⎤ ⎡ − 2 ⎤ ⎡ − 1⎤
b) x + y = ⎢⎢ 3 ⎥⎥ + ⎢⎢ 1 ⎥⎥ = ⎢⎢ 4 ⎥⎥
⎢⎣ 2 ⎥⎦ ⎢⎣ − 1 ⎥⎦ ⎢⎣ 1 ⎥⎦

Department of Statistics Ida Rosmini Othman 17

Some Basics of Matrix and Vector Algebra

EXAMPLE 2.1

c) Length of x d) Length of y

⎡1⎤ ⎡ −2 ⎤
x' x = [1 3 2] ⎢⎢3⎥⎥ =14 y' y = [ −2 1 −1] ⎢⎢ 1 ⎥⎥ = 6
⎢⎣2⎥⎦ ⎢⎣ −1⎥⎦
Lx = 14 = 3.742 Lx = 6 = 2.449

Department of Statistics Ida Rosmini Othman 18

9
 Some Basics of Matrix and Vector Algebra

EXAMPLE 2.1
e) Angel between x and y

⎡ −2 ⎤
x' y = [1 3 2 ] ⎢⎢ 1 ⎥⎥ = − 1
⎢⎣ − 1 ⎥⎦
x,y −1
cos (θ ) = = = − 0.109
' ' 14 6
xx yy
θ = 96.3D
Department of Statistics Ida Rosmini Othman 19

Some Basics of Matrix and Vector Algebra

EXAMPLE 2.1
f) Check that length of 3x is 3 times the length of x
⎡3⎤
3 x = ⎢⎢9 ⎥⎥
⎢⎣ 6 ⎥⎦
⎡3⎤
3 x 3 x = [3 9 6] ⎢⎢9 ⎥⎥ = 126
'

⎢⎣6 ⎥⎦
L3 x = 126
3Lx = 3 14 = 126
∴ L3 x = 3Lx
Department of Statistics Ida Rosmini Othman 20

10
 Some Basics of Matrix and Vector Algebra

MATRICES
⎡0 3 1⎤ ⎡1 −2 −3⎤
A=⎢ ⎥ B=⎢ ⎥
⎣1 −1 1⎦ ⎣2 5 1 ⎦
⎡0 1⎤
Transpose : A' = ⎢⎢3 −1⎥⎥
⎢⎣1 1 ⎥⎦
⎡0 12 4⎤
Multiply by cons tan t : 4A = ⎢
⎣ 4 −4 4⎥⎦
⎡0 3 1⎤ ⎡1 −2 −3⎤ ⎡1 1 −2⎤
Addition : A + B = ⎢ ⎥+⎢ =
⎣1 −1 1⎦ ⎣ 2 5 1 ⎥⎦ ⎢⎣3 4 2 ⎥⎦
Department of Statistics Ida Rosmini Othman 21

Some Basics of Matrix and Vector Algebra

MATRICES
Multiplication :
⎡ −2 ⎤
⎡3 −1 2⎤ ⎡2 0⎤
A=⎢ B = ⎢⎢ 7 ⎥⎥ C =⎢
⎣1 5 4 ⎥⎦ ⎣1 − 1⎥⎦
⎢⎣ 9 ⎥⎦
⎡ −2 ⎤
⎡3 −1 2 ⎤ ⎢ ⎥ ⎡5⎤
A B =⎢ 7 =
( 2×3 ) ( 3×1) ⎣ 1 5 4 ⎥⎦ ⎢ ⎥ ⎢ 69 ⎥
⎣ ⎦
⎢⎣ 9 ⎥⎦ ( 2 ×1)
⎡2 0 ⎤ ⎡3 −1 2⎤ ⎡6 −2 4 ⎤
A =⎢ =
− 1⎥⎦ ⎢⎣ 1 4 ⎥⎦ ⎢⎣ 2 − 2 ⎥⎦
C
( 2×2 ) ( 2×3 ) ⎣ 1 5 −6
( 2× 3 )

Department of Statistics Ida Rosmini Othman 22

11
 Some Basics of Matrix and Vector Algebra

MATRICES

A square matrix is said to be symmetric if A=A’

⎡3 5⎤
Symmetric : ⎢
⎣5 − 2 ⎥⎦
⎡3 6⎤
Not symmetric : ⎢
⎣4 − 2 ⎥⎦

Department of Statistics Ida Rosmini Othman 23

Some Basics of Matrix and Vector Algebra

MATRICES

Inverse for matrix A = ⎡ 3 5⎤ is

⎢ 4 1⎥
⎣ ⎦

⎡ −.2 .4 ⎤
A −1 = ⎢ ⎥
⎣ .8 −.6 ⎦

Department of Statistics Ida Rosmini Othman 24

12
 Some Basics of Matrix and Vector Algebra

MATRICES
⎡ a11 0 0 0 0 ⎤
⎢0 a22 0 0 0 ⎥⎥ ⎡ 1 ⎤
⎢ ⎢a 0 0 0 0 ⎥
Diagonal : ⎢ 0 0 a33 0 0 ⎥ ⎢ 11 ⎥
⎢ ⎥ ⎢ 1 ⎥
⎢0 0 0 a44 0 ⎥
⎢ 0 0 0 0 ⎥
⎢⎣ 0 0 0 0 a55 ⎥⎦ ⎢ a22 ⎥
⎢ 1 ⎥
Inverse : ⎢ 0 0 0 0 ⎥
⎢ a33 ⎥
⎢ 1 ⎥
⎢ 0 0 0 0 ⎥
⎢ a44 ⎥
⎢ 1 ⎥
⎢ 0 0 0 0 ⎥
⎢⎣ a55 ⎥⎦

Department of Statistics Ida Rosmini Othman 25

Some Basics of Matrix and Vector Algebra

MATRICES
A square matrix is said to be orthogonal if
QQ ' = Q ' Q = I
Q −1 = Q '
⎡cos θ − sin θ ⎤ ⎡ cosθ sin θ ⎤ ⎡1 0 ⎤
⎢ sin θ =
⎣ cos θ ⎥⎦ ⎢⎣ − sin θ cos θ ⎥⎦ ⎢⎣0 1 ⎥⎦
−1
⎡ cos θ sin θ ⎤ ⎡ cosθ − sin θ ⎤
⎢ − sin θ =⎢
⎣ cos θ ⎥⎦ ⎣ sin θ cos θ ⎥⎦
Department of Statistics Ida Rosmini Othman 26

13
 CLASS EXERCISE

Exercise 2.2, 2.3 (a-c), 2.5, 2.6 (a)

Department of Statistics Ida Rosmini Othman 27

Department of Statistics Ida Rosmini Othman 28

14