Strassen's Matrix

Multiplication
 Basic Matrix Multiplication

Suppose we want to multiply two matrices of size N

x N: for example A x B = C.

C11 = a11b11 + a12b21

C12 = a11b12 + a12b22

C21 = a21b11 + a22b21 2x2 matrix multiplication can be

accomplished in 8 multiplication.(2log28 =23)
C22 = a21b12 + a22b22
 Basic Matrix Multiplication
printf("multiply of the matrix=\n");    
for(i=0;i<n;i++)    
{    
for(j=0;j<n;j++)     algorithm
{    
mul[i][j]=0;    
for(k=0;k<n;k++)    
{    
mul[i][j]= mul[i][j]+a[i][k]*b[k][j];    
}}}      

N
Time analysis Ci , j   ai ,k bk , j
k 1
N N N
Thus T ( N )   c  cN 3  O( N 3 )
i 1 j 1 k 1
 Divide-and-Conquer
• Divide-and conquer is a general algorithm design
paradigm:
– Divide: divide the input data S in two or more disjoint
subsets S1, S2, …
– Recur: solve the subproblems recursively
– Conquer: combine the solutions for S1, S2, …, into a
solution for S
• The base case for the recursion are subproblems of
constant size
• Analysis can be done using recurrence equations
Divide and Conquer Matrix Multiply
A  B = R
A0 A1 B0 B1 A0B0+A1B2 A0B1+A1B3
 =
A2 A3 B2 B3 A2B0+A3B2 A2B1+A3B3

•Divide matrices into sub-matrices: A0 , A1, A2 etc

•Use blocked matrix multiply equations
•Recursively multiply sub-matrices
Divide and Conquer Matrix Multiply

A  B = R

a0  b0 = a 0  b0

• Terminate recursion with a simple base case

 Strassen’s Algorithm
• We can use a Divide and Conquer solution to solve matrix
multiplication by separating a matrix into 4 quadrants:

X
=

Then we know have:

a a  b b12  c c 
A   11 12  B   11  C   11 12 
 a21 a22   b21 b22   c21 c22 

if C  AB c11  a11b11  a12 b21

c12  a11b12  a12b22
, then we have the c21  a21b11  a22b21
following:
c22  a21b12  a22b22
8 (n/2 * n/2 matrix multiples) + 4 (n/2 * n/2 matrix additions)
T(n) = 8T(n/2) + O(n2)
If we solve using the master theorem we still have O(n3)
 Strassen’s Algorithm
• Strassen showed how two matrices can be
multiplied using only 7 multiplications and
18 additions/subtraction op:
– Consider calculating the following 7 products:
• q1 = (a11 + a22) * (b11 + b22)
• q2 = (a21 + a22) * b11
• q3 = a11*( b12 – b22)
• q4 = a22 * (b21 – b11)
• q5 = (a11 + a12) * b22
• q6 = (a21 – a11) * (b11 + b12)
• q7 = (a12 – a22) * (b21 + b22)

– It turns out that

• c11 = q1 + q4 – q5 + q7
• c12 = q3 + q5
• c21 = q2 + q4
• c22 = q1 + q3 – q2 + q6
 Strassen’s Algorithm

Mult Add Recurrence Relation Runtime

Regular 8 4 T(n) = 8T(n/2) + O(n2) O(n3)
Strassen 7 18 T(n) = 7T(n/2) + O(n2) O(n log27) = O(n2.81)
 Strassen’s Algorithm
• I have no idea how Strassen came up with these
combinations.
– He probably realized that he wanted to determine each
element in the product using less than 8 multiplications.
• From there, he probably just played around with it.

• If we let T(n) be the running time of Strassen's

algorithm, then it satisfies the following recurrence
relation:
– T(n) = 7T(n/2) + O(n2)
• It's important to note that the hidden constant in the O(n2) term
is larger than the corresponding constant for the standard divide
and conquer algorithm for this problem.
– However, for large matrices this algorithm yields an
improvement over the standard one with respect to time.
 Strassen Algorithm
    Algorithm Strassen(n, a, b, q)
begin
            If n = threshold then compute
         C = a * b is a conventional matrix.
            Else
              Partition a into four sub matrices a11, a12, a21, a22.
              Partition b into four sub matrices b11, b12, b21, b22.
   Strassen ( n/2, a11 + a22, b11 + b22, q1)
                     Strassen ( n/2, a21 + a22, b11, q2)
                     Strassen ( n/2, a11, b12 – b22, q3)
                     Strassen ( n/2, a22, b21 – b11, q4)
                     Strassen ( n/2, a11 + a12, b22, q5)
                     Strassen (n/2, a21 – a11, b11 + b22, q6)
                     Strassen (n/2, a12 – a22, b21 + b22, q7)
                    C11 = q1+q4-q5+q7
C12= q3+q5
C21=q2+q4  
C22=q1+q3-q2-q6  
           end if
                return (C)
end.
Time Analysis