Multiplying Matrices

n Two matrices, A with (p x q) matrix and B with (q x r)

matrix, can be multiplied to get C with dimensions p x r,
using scalar multiplications

1
Multiplying Matrices

2
Ex2:Matrix-chain multiplication
n We are given a sequence (chain) <A1, A2, ..., An> of n
matrices to be multiplied, and we wish to compute the
product. A1.A2…An.

n Matrix multiplication is associative, and so all

parenthesizations yield the same product,

(A1 (A2 (A3 A4))) ,

(A1 ((A2 A3) A4)) ,
((A1 A2) (A3 A4)) ,
((A1 (A2 A3)) A4) ,
(((A1 A2) A3) A4).

3
 Matrix-chain multiplication – cont.
n We can multiply two matrices A and B only if they are
compatible the number of columns of A must equal the
number of rows of B. If A is a p × q matrix and B is a q ×
r matrix, the resulting matrix C is a p × r matrix.

n The time to compute C is dominated by the number of

scalar multiplications, which is p q r.

4
Matrix-chain multiplication – cont.
n Ex: consider the problem of a chain <A1, A2, A3> of
three matrices, with the dimensions of 10 × 100, 100
× 5, and 5 × 50, respectively.

n If we multiply according to ((A1 A2) A3), we perform

10 · 100 · 5 = 5000 scalar multiplications to compute
the 10 × 5 matrix product A1 A2, plus another 10 · 5 ·
50 = 2500 scalar multiplications to multiply this matrix
byA3 for a total of 7500 scalar multiplications

n If instead , we multiply as (A1 (A2 A3)), we perform

100 · 5 · 50 = 25,000 scalar multiplications to
compute the 100 × 50 matrix product A2 A3, plus
another 10 · 100 · 50 = 50,000 scalar multiplications
to multiply A1 by this matrix, for a total of 75,000
scalar multiplications. Thus, computing the product
according to the first parenthesization is 10 times
faster.
5
Matrix-chain multiplication – cont.
n Matrix multiplication is associative
q q (AB)C = A(BC)

n The parenthesization matters

n Consider A X B X C X D, where
q A is 30x1, B is 1x40, C is 40x10, D is 10x25

n Costs:
q ((AB)C)D = 1200 + 12000 + 7500 = 20700
q (AB)(CD) = 1200 + 10000 + 30000 = 41200
q A((BC)D) = 400 + 250 + 750 = 1400

6
Matrix Chain Multiplication (MCM)
Problem
n Input:
q Matrices A1, A2, …, An, each Ai of size pi-1 x pi,

n Output:
q Fully parenthesised product A1 x A2 x … x An
that minimizes the number of scalar
multiplications.

n Note:
q In MCM problem, we are not actually multiplying
matrices
q Our goal is only to determine an order for
multiplying matrices that has the lowest cost
7
Matrix Chain Multiplication (MCM)
Problem
n Typically, the time invested in determining
this optimal order is more than paid for by the
time saved later on when actually performing
the matrix multiplications

n So, exhaustively checking all possible

parenthesizations does not yield an efficient
algorithm

8
 Counting the Number of Parenthesizations
n Denote the number of alternative parenthesizations of a
sequence of (n) matrices by P(n) then a fully
parenthesized matrix product is given by:

! (A1 (A2 (A3 A4))) , If n = 4 <A1,A2,A3,A4> then the number of

! (A1 ((A2 A3) A4)) , alternative parenthesis is =5
! ((A1 A2) (A3 A4)) , P(4) =P(1) P(4-1) + P(2) P(4-2) + P(3) P(4-3)
! ((A1 (A2 A3)) A4) , = P(1) P(3) + P(2) P(2) + P(3) P(1)= 2+1+2=5
! (((A1 A2) A3) A4). P(3) =P(1) P(2) + P(2) P(1) = 1+1=2
P(2) =P(1) P(1)= 1

9
Counting the Number of Parenthesizations

10
Step 1: Optimal Sub-structure

11
Step 1: Optimal Sub-structure

12
Step 1: Optimal Sub-structure

13
Step 1: Optimal Sub-structure

14
Step 2: Recursive Solution
n Next we define the cost of optimal solution
recursively in terms of the optimal solutions to
sub-problems

n For MCM, sub-problems are:

q problems of determining the min-cost of a
parenthesization of Ai Ai+1…Aj for 1 <= i <= j <= n

n Let m[i, j] = min # of scalar multiplications

needed to compute the matrix Ai...j.
q For the whole problem, we need to find m[1, n]

15
Step 2: Recursive Solution
n Since Ai..j can be obtained by breaking it into
Ai..k Ak+1..j, We have (each Ai of size pi-1 x
pi)
q m[i, j] = m[i, k] + m[k+1, j] + pi-1pkpj

n There are j-i possible values for k: i <= k <= j

n Since the optimal parenthesization must use

one of these values for k, we need only check
them all to find the best.

16
Step 2: Recursive Solution

17
Step 3: Computing the Optimal Costs

18
Elements of Dynamic Programming

19
Overlapping Sub-problems

20
Step 3: Computing the Optimal Costs
The following code assumes that matrix Ai has
dimensions pi-1 x pi for i =1,2, …, n

21
Step 3: Computing the Optimal Costs

22
Step 3: Computing the Optimal Costs

23
Step 3: Computing the Optimal Costs

24
Step 3: Computing the Optimal Costs

25
Step 3: Computing the Optimal Costs

26
Step 3: Computing the Optimal Costs

27
Step 4: Constructing an Optimal Solution
n Although Matrix-Chain-Order determines the
optimal number of scalar multiplications
needed to compute a matrix-chain product, it
does not directly show how to multiply the
matrices

n Using table s, we can construct an optimal

solution

28
Step 4: Constructing an Optimal Solution

29
Step 4: Constructing an Optimal Solution

30
Step 4: Constructing an Optimal Solution

31
Step 4: Constructing an Optimal Solution

32