Lecture 2. About matrices.

1. A matrix is a rectangular array of scalars such

as

 a11 a12 ... a1n 

a a22 .. a2n 
A   21  having m rows & n
 .. .. .. .. 
 
 am1 am2 .. amn 

columns. The ith row and jth column entry of this

matrix is aij . We often denote a matrix as
Amn  [ aij ]

Matrix elements may be constants (real, complex,

ordinal), variables like functions of time, etc.

Row of a matrix. Column of a matrix. Square

matrix. Rectangular matrix. Diagonal elements.

A typical square matrix must have same number

of rows and columns.
  on off on on 
 off off 
 2 3  on off
e.g. A   4 5 B=  
   on off on off 
 
 off on off on 

All square matrices are rectangular matrices, but

not all rectangular matrices are square.

 1 1 0 1 1 
e.g. A25  
 1 1 1 0 0 

A Diagonal matrix is the following.

 d11 0 0 
D 0 d 22 0  . Notice that all dij  0, if i  j
 
 0 0 d 33 


An Identity matrix is a diagonal matrix with all its

diagonal elements equal to 1.

1 0 0
I  0 1 0 
 
0 0 1
 
Trace of a matrix Tr( A ) is the sum of all its
n
diagonal elements. Tr( A )   aii .
i 1

A matrix A is strictly diagonally dominant if for

each row i, | aij | < |aii |
j i

Transpose of a matrix A is At whose rows and

columns are columns and rows of A .
 2 0
 2 1 3 
e.g. A    , At
  1 3 
0 3 2  
 3 2
 

The (ij)th element of At is a ji .

A complex matrix has complex elements in it.

 2i 0 2 
e.g. C33   1  i i 1  i 
 
 2  i 3i 3 


Given a complex number z  a  ib , its conjugate

is a complex number z  a  ib obtained by
reversing the sign of its imaginary part.
A conjugate transpose matrix D obtained from
the original complex matrix C is such that dij  c ji .

Thus, conjugate transpose of C above is

 2  i 1  i 2  i 
C t  D33  0 i 3i 
 
 2 1 i 3 


A matrix C H is Hermitian if it is equal to its

conjugate transpose. i.e.

If cij  c ji then C is Hermitian. A Hermitian matrix

will have real numbers in its diagonal.

 1 i   p -i 
e.g. i 1 ,  i 1
   

Also are the Pauli matrices

0 1 0 i 1 0 
1    ,  2   i 0  ,  3   0 1 
 1 0     

A typical 3  3 Hermitian matrix would be

  2 1i 2i 
H  1 i 3 2  3i 
 
 2i 2  3i 6 


A square matrix A is skew-Hermitian if aij   a ji

A matrix could often be configured with positional

dependencies. For instance, a Hilbert matrix H is
1
a square matrix in which ijh  . These
i  j 1
type of matrices are often used to test matrix
algorithms.

2. An upper triangular matrix U has the form that

all items below the diagonal are zero, i.e.

uij  aij i  j
= 0, otherwise

 3 1 2 0 9 
 0 2 2 1 4 
 
e.g. U   0 0 3 0 1 
 
 0 0 0 1 9 
0 0 0 0 4 
 
A lower triangular matrix L has all items above
the diagonal zero.

lij  aij i  j
= 0, otherwise

e.g.

3 0 0 0 0
1 2 0 0 0
 
L   3 2 3 0 0
 
 8 3 1 1 0
 1 1 2 1 4 


A matrix is Hessenberg (upper or lower) if it is

'almost' triangular with an additional off-diagonal
band just below or above the main diagonal. e.g.

1 0 -1 5 3
2 -2 3 4 6
b= -1 1 2 0 7
0 1 2 -3 -4
1 -1 2 -2 3
and its Hessenberg form is:

1.0000 -1.6330 3.2889 -4.6345 0.1941

-2.4495 -1.6667 -0.0384 4.7074 -1.9307
0 2.8964 0.3091 7.9878 -5.8211
0 0 4.7644 0.2614 -1.8186
0 0 0 2.2094 1.0962

Another important type of matrices is a sparse

matrix. It is a matrix where most of the elements
are 0 (e.g. only 10% of the elements have non-zero
values, say). The issue here is storage of such
matrices. Normal approach is inefficient.

e.g.

0 2 0 0 1 
5 0 0 0 3 
 
A  0 0 0 0 6 
 
2 0 0 0 0 
0 0 0 1 0 

Its storage has to be designed, preferably, in some
form of list structure such as

1,2,2 1,5,1 2,1,5 2,5,3

3,5,-6 4,1,2 5,4,1 NULL

Finally, a block matrix is a matrix that is defined

using smaller blocks (of rectangular) matrices. It
is advantageous in computation. e.g.

A B
X   where each element unit is a matrix
 C D 
such as
 1 1 
2 3   4 3 2  2 2 
A ,B  ,C 
4 1   4 0 5 
  
 3 2

 
 3 0 3 
D   2 2 0 
 
 1 3 4 

  2 3 4 3 2
4 1 4 0 5
 
with the original X   1 1 3 0 3 
 
 2 2 2 2 0 
 3 2 1 3 0 


3. Matrix addition & scalar multiplication. Matrix

multiplication.

Given Amn and a Bmn , we may define Cmn as

Cmn  Amn  kBmn

if cij  aij  kbij i  [1,m] & j  [1,n]

2 0 3  -2 0 -3 
   
Thus, A   1 1 9  , -A =  -1 1 -9 
4 3 5  -4 -3 -5 
   

Summation Symbols one more time!

Given two vectors u and v , we may compute
expressions like:

a. uj
j  u1  u2  u3  ...  un on an n-component

vector
n 1

b.  ( j  1 )u
j 1
j  2u1  3u2  4u3  ...  nun1

n
c. u v
k 3
k k 2  u3 v1  u4 v2  ...  un vn 2

Similarly, on matrices

d. ai
ij  a1 j  a2 j  a3 j  ...  amj

 a   ( a
i j j
ij
i j
i2  ai 3  ai4  ...  ain )
e. = a12  ( a13  a23 )  ( a14  a24  a34 )
+ ... + (a1n  a2n  ...  amn )

n
f. a
k 1
ik kjb  ( ai1b1 j  ai 2b2 j  ...  ainbnj )
This is an inner product of two vectors:
i-th row of a and the j-th column of b .

 ai. | b. j  
 b1 j 
b 
 2j 
 b3 j 
(ai1 ai2 ai3 ... aij ... ain )  
 ... 
 ... 
 
 bnj 

3. 1 Matrix multiplication

If Am p is multiplied from the right by B pn the

p

result is Cmn such that the cij   aik bkj

k 1
Observe. Am p can multiply Bqn from the left if
p  q . It can multiply Bqn from the right if m  n .

e.g.

 1 2 3  1 0 1 
A   2 1 1  and B   2 2 3 
   
 2 0 2   1 1 2 
   

Then

c23  a21b13  a22b23  a23b33

= 2  (-1)+(-1)  3+1  2 =-3

and the entire matrix c is

 2 1 11 
C   1 3 3 
 
 4 2 6 
 
Note that in matrix multiplication AB  BA
usually. If they are equal they are said to be
commutative. In our case,

 2 1 11  3 2 1
AB   1 3 3  , BA=  -8 6 10 
   
 4 2 6   -3 -3 2 
   

Also, note for a square matrix Ann ,

AI  IA  A if I is a n  n identity matrix.

The product of block matrices of same structure

and shape with diagonal block elements as square
matrices yields a block matrix of same structure.

e.g. Product of two block diagonal matrices

 A1 0 0  B1 0 0   A1 B1 
0 A2 0  0 B2 0  A2 B2 
    
0 0 A3  B3   A3 B3 
  0 0

also, for general block-matrices

 A1 A2  B1 B2   A1 B1  A2 B2 A1 B2  A2 B4 
A 
 3 A4 
 B3

B4   A3 B1  A4 B3 A3 B2  A4 B4 

Note that the diagonal blocks must be square

matrices.

3.2 Question: If AB  I , could we conclude that

BA  I ?

AB  I  ABA  A (multiplying from the right). If

BA  X then AX  A implying X  I .

Therefore, AB  I  BA

Another question: Is it possible for AB  0 (the

zero matrix, with all elements 0) even though
neither A nor B is zero? Leave it to you as an
exercise!

4. Inverse of a matrix.

Suppose, AB  I . B is then inverse of A and is

denoted by A1 assuming A is invertible.
Accordingly, AA1  I . Since, AB  I  BA  I
, we see A1 A  I as well.
Determination of inverse of a matrix is a challenge
for a numerical analyst. This is a significant
problem for us.

Given C  AB , C 1  B 1 A1 . That it is so can be

seen via this construct

C 1C  B 1 A1 AB  I . This means for any product

C  A1 A2 A3 ...Ai , C 1  Ai1 Ai11 ...A21 A11 . In all these
applications, we assume the matrices are
invertible.

If D is a diagonal matrix like

 d11 0 0 0 
 0 d 22 0 0 
D   then
 0 0 .. 0 
 
 0 0 0 d nn 

 1 / d11 0 0 0 
 0 1 / d 22 0 0 
1
D   
 0 0 .. 0 
 
 0 0 0 1 / d nn 
A real matrix A is orthogonal if At  A1 .
Orthonormal matrices constitute an important
class of matrices in physical sciences.

An inverse of an orthogonal matrix is orthogonal;

product of two orthogonal matrices is orthogonal.

Orthogonal matrices preserve inner products as

can be seen from this:

Suppose Q is orthogonal. Let

| x  = Q|x> and |y'> = Q|y> . Then

 y | =  y | Q t and
 y | x  = <y|Q t Q | x  = <y|x>

Also, an orthogonal matrix preserves the norm of

a vector, i.e. || Qx ||  ||x|| . This is useful when
we want to solve an equation like A| x   |b 
by its equivalent problem Q | A| x  Q | b 

Recall our discussion on inner product space.

Accordingly, rows (or columns) of an orthogonal
matrix form an orthogonal basis in R n with usual
inner products either 1 or 0.
e.g.

0.4507 -0.5172 0.0236 -0.4721 0.5531

0.4465 0.2753 -0.3655 0.6273 0.4447
x = 0.5096 0.6817 0.0726 -0.4827 -0.1930
0.3984 -0.3959 -0.5234 0.0386 -0.6396
0.4231 -0.1878 0.7659 0.3860 -0.2236

Take any row or column of x, and see that its

norm is 1.0. Dot product of any two rows or two
columns must be 0.0.

4.1 How do we determine A1 given A ? Consider a

simple case.

 a11 a12 
a. A    we want its inverse
a
 21 a 22 

 b11 b12 
B  such that AB  I
b b
 21 22 

 a11 a12  b11 b12   1 0 

Now      implies that
 a21 a22  b21 b22   0 1 
a11b11  a12b21  1
a21b12  a22b22  1
a11b12  a12b22  0
a21b11  a22b21  0

These equations give us the following:

Let | A| a11a22  a12 a21 . Then

a a a a
b11  22 , b12  12 , b21  21 , b22  11
| A| | A| | A| | A|

b. Suppose we like to obtain A1 of the matrix A

where A is the following:

 2 1 1   x1 x2 x3 
A   0 1 1  Assume, A1   y1 y2 y3 
   
 1 1 2  z z2 z3 
   1

 2 1 1   x1 x2 x3 
AA1   0 1 1   y1 y2 y3   I 3
  
 1 1 2  z z2 z3 
  1

Then, we have nine sets of equations to solve for

the unknowns. (Tedious but could be done with
patience, etc.). Eventually, one gets
  0.5 0.5 0 
A1   0.5 2.5 1 
 
 0.5 1.5 1 
 

c. There are other better computationally feasible

ways to solve the matrix inversion problem.