Matrices

Part I
Introduction
• A matrix is a vector taken to the next level.
• Matrices are highly versatile mathematical objects.
• They can store sets of equations, geometric transformations, the
positions of particles over time, financial records, and myriad other
things.
• In data science, matrices are sometimes called data tables, in which
rows correspond to observations (e.g., customers) and columns
correspond to features (e.g., purchases).
Creating and Visualizing Matrices in NumPy
• Matrices are indicated using bold-faced capital letters, like matrix A or
M.
• The size of a matrix is indicated using (row, column) convention.
• For example, the following matrix is 3 × 5 because it has three rows
and five columns:
Creating and Visualizing Matrices in NumPy
• You can refer to specific elements of a matrix by indexing the row
and column position: the element in the 3rd row and 4th column of
matrix A is indicated as a3,4
• Important reminder: math uses 1-based indexing whereas Python
uses 0-based indexing. Thus, element a3,4 is indexed in Python as
A[2,3]
Creating and Visualizing Matrices in NumPy
• Special Matrices
• matrices can be described using a relatively small number of
characteristics, which creates “families” or categories of matrices

• Following is a list of some common special matrices and Python code

to create them
Creating and Visualizing Matrices in NumPy
• Special Matrices
• Random numbers matrix
• This is a matrix that contains numbers drawn at random from some
distribution, typically Gaussian.
Creating and Visualizing Matrices in NumPy
• Special Matrices
• Square versus nonsquare
• A square matrix has the same number of rows as columns; in other
words, the matrix is in ℝN × N . A nonsquare matrix, also sometimes
called a rectangular matrix, has a different number of rows and
columns
• Rectangular matrices are called tall if they have more rows than
columns and wide if they have more columns than rows.
Creating and Visualizing Matrices in NumPy
• Special Matrices
• Diagonal
• The diagonal of a matrix is the elements starting at the top-left and
going down to the bottom-right. A diagonal matrix has zeros on all
the off-diagonal elements

• The NumPy function np.diag() has two behaviors depending on the

inputs: input a matrix and np.diag will return the diagonal elements
as a vector; input a vector and np.diag will return a matrix with those
vector elements on the diagonal
Creating and Visualizing Matrices in NumPy
• Special Matrices
• Triangular
• A triangular matrix contains all zeros either above or below the main
diagonal.
• The matrix is called upper triangular if the nonzero elements are
above the diagonal and lower triangular if the nonzero elements are
below the diagonal.
• NumPy has dedicated functions to extract the upper (np.triu()) or
lower (np.tril()) triangle of a matrix
Creating and Visualizing Matrices in NumPy
• Special Matrices
• Identity
• The identity matrix is a square diagonal matrix with all diagonal
elements having a value of 1. It is indicated using the letter I

• You can create an identity matrix in Python using np.eye()

Creating and Visualizing Matrices in NumPy
• Special Matrices
• Zeros
• The zeros matrix is comparable to the zeros vector: it is the matrix of
all zeros. Like the zeros vector, it is indicated using a bold-faced zero: 0

• The zeros matrix is created using the np.zeros() function.

Creating and Visualizing Matrices in NumPy
• Special Matrices
Matrix Math: Addition, Scalar Multiplication,
Hadamard Multiplication

• Mathematical operations on matrices fall into two categories:

intuitive and unintuitive.

• In general, the intuitive operations can be expressed as element-wise

procedures
Addition and Subtraction
• You add two matrices by adding their corresponding elements. Here’s
an example:

• matrix addition is defined only between two matrices of the same

size.
Scalar-matrix multiplication
• If A is an m X n matrix and c is a scalar, then the scalar multiple cA is
the mXn matrix obtained by multiplying each entry of A by c.
Hadamard Multiplications
• Hadamard multiplication involves multiplying two matrices element-
wise

• In NumPy, Hadamard multiplication can be implemented using the

np.multiply() function.
• But it’s often easier to implement using an asterisk between the two
matrices: A*B
Hadamard Multiplications
Standard Matrix Multiplication
• You know that matrix sizes are written out as M × N—rows by
columns.
• Two matrices multiplying each other can have different sizes, so let’s
refer to the size of the second matrix as N × K.
• When we write out the two multiplicand matrices with their sizes
underneath, we can refer to the “inner” dimensions N and the
“outer” dimensions M and K.
Standard Matrix Multiplication
• Here’s the important point: matrix multiplication is valid only when
the “inner” dimensions match, and the size of the product matrix is
defined by the “outer” dimensions
Standard Matrix Multiplication
• More formally, matrix multiplication is valid when the number of
columns in the left matrix equals the number of rows in the right
matrix, and the size of the product matrix is defined by the number of
rows in the left matrix and the number of columns in the right matrix.
• Example of matrix multiplication
Standard Matrix Multiplication
• Example
Standard Matrix Multiplication
Standard Matrix Multiplication
• Ann and Bert are planning to go shopping for fruit for the next week.
They each want to buy some apples, oranges, and grapefruit, but in
differing amounts.
• Table below lists what they intend to buy. There are two fruit markets
nearby Sams and Theos and their prices are given in the Table.
• How much will it cost Ann and Bert to do their shopping at each of
the two markets?
Standard Matrix Multiplication
Standard Matrix Multiplication
• Solution:
• If Ann shops at Sam's, she will spend
6(0.10) + 3(0.40) + 10(0.10)= $2.80
• If she shops at Theo's, she will spend
6(0.15) + 3(0.30) + 10(0.20)= $3.80
• Bert will spend at Sams
4(0.10)+ 8(0.40) + 5(0.10) = $4.10
• And at Theos
4(0.15) + 8(0.30) + 5(0.20) = $4.00.
Standard Matrix Multiplication
• The "dot product form" of these calculations suggests that matrix
multiplication is at work here. If we organize the given information
into a demand matrix D and a price matrix P, we have
Matrices
Part II
Matrix-Vector Multiplication
• Mutliplying a matrix and a vector is simply matrix multiplication
where one “matrix” is a vector

• Rule: A matrix can be right-multiplied by a column vector but not a

row vector, and it can be left-multiplied by a row vector but not a
column vector.
Matrix-Vector Multiplication
• Let
Matrix-Vector Multiplication
• That is clear from inspecting matrix sizes: an M × N matrix can be
premultiplied by a 1xM matrix (a row vector) or postmultiplied by an
N ×1 matrix (a column vector)

• The result of matrix-vector multiplication is always a vector, and the

orientation of that vector depends on the orientation of the
multiplicand vector: premultiplying a matrix by a row vector produces
another row vector, while postmultiplying a matrix by a column
vector produces another column vector
Linear weighted combinations
• Previously, we calculated linear weighted combinations by having
separate scalars and vectors, and then multiplying them individually

• You can use matrix-vector multiplication to implement it.

• Put the individual vectors into a matrix, and put the weights into
corresponding elements of a vector. Then multiply
Partitioned Matrices
• It will often be convenient to regard a matrix as being composed of a
number of smaller submatrices.

• By introducing vertical and horizontal lines into a matrix, we can

partition it into blocks.
Partitioned Matrices
• For example, consider the matrix

where I is the 3X3

identity matrix,
B is 3 X 2,
O is the 2X3 zero
matrix,
and C is 2 X 2.

In this way, we can view A as a 2 X 2 matrix whose entries are

themselves matrices.
Partitioned Matrices
• Suppose A is m X n and B is n X r, so the product AB exists. If we
partition B in terms of its column vectors, as B = [b1, b2, …, bn], then

• This result is an immediate consequence of the definition of matrix-

vector multiplication.
• The form on the right is called the matrix-column representation of
the product (postmultiplied).
Partitioned Matrices
• Example
Partitioned Matrices
• Remark:
• Observe that the matrix-column representation of AB allows us to
write each column of AB as a linear combination of the columns of A
with entries from B as the coefficients. For example,
Partitioned Matrices
• Suppose A is m X n and B is n X r, so the product AB exists. If we
partition A in terms of its row vectors, as
Partitioned Matrices
• Once again, this result is a direct consequence of the definition of
matrix-vector multiplication.
• The form on the right is called the row-matrix representation of the
product (premultiplying).

• Classwork
• Using the row-matrix representation, compute AB for the matrices in
the previous example
Partitioned Matrices
Partitioned Matrices
• The definition of the matrix product AB uses the natural partition of
A into rows and B into columns; this form might well be called the
row-column representation of the product.

• We can also partition A into columns and B into rows; this form is
called the column-row representation of the product
Partitioned Matrices

• Notice that the sum resembles a dot product expansion;

• Thus, each aiBi is called outer products, and the summation of them
all is called the outer product expansion of AB.
Partitioned Matrices
• Example: Compute the outer product expansion of previous AB
matrices

• The outer product are

Partitioned Matrices
• Therefore, the outer product expansion of AB is
The Transpose of a Matrix
• The transpose of an m X n matrix A is the n X m matrix AT obtained by
interchanging the rows and columns of A That is, the ith column of AT
is the ith row of A for all i.
The Transpose of a Matrix
• The transpose is sometimes used to give an alternative definition of
the dot product of two vectors in terms of matrix multiplication. If
The Transpose of a Matrix
• The transpose is also used to define a very important type of square
matrix: a symmetric matrix.

• Definition:
• A square matrix A is symmetric if AT = A that is, if A is equal to its own
transpose
The Transpose of a Matrix
• Example:
Exercises

compute the indicated matrices (if possible).

Exercises
• write the given system of linear equations as a matrix equation of the
form Ax = b.
Exercises
1. Use the matrix-column representation of the product to write each
column of AB as a linear combination of the columns of A.
2. Use the row-matrix representation of the product to write each row
of AB as a linear combination of the rows of B.
3. Compute the outer product expansion of AB.
Advanced Topics in Matrices
Linear combinations of matrices.
• Example:

• Solution: We want to find scalars c1, c2, and c3 such that c1A1 + c2A2 +
c3A3 = B. Thus,
Linear combinations of matrices.
• a)

• The left hand side of the equation can be written as:

Linear combinations of matrices.
• Comparing entries and using the definition of matrix equality, we
have four linear equations:

• so c1 = 1, c2 = -2, and c3 = 3. Thus, A1 - 2A2 + 3A3 = 5,

Linear combinations of matrices.
• b) This time we want to solve

• Proceeding as in part (a), we obtain the linear system

Linear combinations of matrices.

• The last row implies that there is no solution. Therefore, in this case,
C is not a linear combination of A1, A2, and A3.
Linear combinations of matrices.

• Linear independence also makes sense for matrices. We say that

matrices A1, A2, . . . , Ak of the same size are linearly independent if
the only solution of the equation

c1A1 + c2A2+-- + ckAk = O

• is the trivial one: c1 = c2 =...= ck = 0

• Otherwise, A1, A2,..., Ak are called linearly dependent.

Linear combinations of matrices.
• Example: Determine whether the matrices A1, A2, and A3 in the
previous example are linearly independent.

• Solution: we want to solve the equation c1A1 + c2A2 + c3A3 = O. Writing

out the matrices, we have
Linear combinations of matrices.

• Thus, c1 = c2 = c3 = 0, and we conclude that the matrices A1, A2, and A3

are linearly independent
The Inverse of a Matrix
• If A is an n X n matrix, an inverse of A is an n X n matrix A' with the
property that
AA' = I and A'A = I
• where I = In is the n X n identity matrix. If such an A' exists, then A is
called invertible.
The Inverse of a Matrix
• Example:
The Inverse of a Matrix
• Show that the following matrices are not invertible
The Inverse of a Matrix

• Row Reduction yields 0 = -2

• Thus, there is no solution
• We deduce that no such matrix B' exists; that is, B is not invertable
The Inverse of a Matrix
• Theorem:
• If A is an invertible n X n matrix, then the system of linear equations
given by Ax = b has the unique solution x = A-1b for any b in Rn

• So, how can we find its inverse when it is invertible?

• We will give a general procedure shortly, but the situation for 2 X 2

matrices is sufficiently simple to warrant being singled out.
The Inverse of a Matrix

• The expression ad - bc is called the determinant of A, denoted det A.

The Inverse of a Matrix
• The formula for the inverse of

(when it exists) is thus

times the matrix obtained by interchanging the entries on the main

diagonal and changing the signs on the other two entries.
The Inverse of a Matrix
• Example:
The Inverse of a Matrix
• Example:
The Gauss-Jordan Method for Computing the
Inverse
• We can perform row operations on A and I (Identity matrix)
simultaneously by constructing a "super-augmented matrix" [A I ].
• Theorem:
• Let A be a square matrix. If a sequence of elementary row operations
reduces A to I, then the same sequence of elementary row operations
transforms I into A-1.
The Gauss-Jordan Method for Computing the
Inverse
• Example:
The Gauss-Jordan Method for Computing the
Inverse
• Example:
The Gauss-Jordan Method for Computing the
Inverse
• Solution:

• At this point, we see that it is not possible to reduce A to I, since there is a

row of zeros on the left-hand side of the augmented matrix. Consequently, A
is not invertible.
The Gauss-Jordan Method for Computing the
Inverse
• Example:

• Solution: