Stassen's
Stassen s Matrix Multiplication
Matrix Multiplication
Dr. Bibhudatta Sahoo
Communication & Computing Group
Department of CSE, NIT Rourkela
Email: [email protected], 9937324437, 2462358
Multiplication of Large Integers
Consider the problem of multiplying two (large) n‐digit
integers represented by arrays of their digits such as:
A = 12345678901357986429 B = 87654321284820912836
The grade‐school algorithm:
a1 a2 … an
b1 b2 … bn
(d10) d11d12
12 … d
… d1n
(d20) d21d22 … d2n
… … … … … … …
(dn0) dn1dn2 … ddnn
Efficiency: Θ(n2) single‐digit multiplications
Divide and conquer 2
First Divide‐‐and
First Divide and‐‐Conquer Algorithm
A small example: A ∗ B where A = 2135 and B = 4014
A = (21∙102 + 35), B = (40 ∙102 + 14)
So, A ∗ B = (21 ∙102 + 35) ∗ (40 ∙102 + 14)
= 21 ∗ 40 ∙104 + (21 ∗ 14 + 35 ∗ 40) ∙102 + 35 ∗ 14
In general, if A = A1A2 and B = B1B2 (where A and B are n‐digit, A1,
A2, B1, B2 are n/
n/2
/2‐digit
g numbers), ),
A ∗ B = A1 ∗ B1∙10n + (A1 ∗ B2 + A2 ∗ B1) ∙10n/2 + A2 ∗ B2
Recurrence for the number of one‐digit
one digit multiplications M(n):
M(n) = 4M(n/2)+c n,
M(1) = 1
M(1) 1
Solution: M(n) = n2
Divide and conquer 3
Second Divide‐‐and
Second Divide and‐‐Conquer Algorithm
A ∗ B = A1 ∗ B1∙10n + (A1 ∗ B2 + A2 ∗ B1) ∙10n/2 + A2 ∗ B2
The idea is to decrease the number of multiplications from 4 to
3:
((A1 + A2 ) ∗ ((B1 + B2 ) = A1 ∗ B1 + ((A
A1 ∗ B2 + A2 ∗ B1) + A2 ∗ B2,
i.e., (A1 ∗ B2 + A2 ∗ B1) = (A1 + A2 ) ∗ (B1 + B2 ) ‐ A1 ∗ B1 ‐ A2 ∗ B2,
which requires only 3 multiplications at the expense of (4‐1)
extra add/sub.
Recurrence for the number of multiplications M(n):
M(n) = 3M(n/2), M(1) = 1
( ) log
Solution: M(n) = 3 l 2n = nlog 3 ≈ n1.585
l 23 1 585
Divide and conquer 4
Example of Large‐
Example of Large‐Integer Multiplication
2135 ∗ 4014
( ) ( )
= (21*10^2 + 35) * (40*10^2 + 14)
= (21*40)*10^4 + c1*10^2 + 35*14
where c1 = (21+35)*(40+14) ‐
( )*( ) 21*40 ‐
* 35*14, and
*
21*40 = (2*10 + 1) * (4*10 + 0)
= (2*4)*10^2 + c2*10 + 1*0
where c2 = (2+1)*(4+0) ‐
h 2 (2 1)*(4 0) 2*4 ‐
2*4 1*0, etc.
1*0 t
This process requires 9 digit multiplications as opposed to 16.
This process requires 9 digit multiplications as opposed to 16.
Divide and conquer 5
Sample problems
• Complex multiplication: If we multiply two complex
numbers a+bi and c+di in the standard way, it
requires four multiplications and two additions of
real numbers. Since multiplications are more
expensive
i than
h additions
ddi i ( d subtractions),
(and b i ) iti pays
to minimize the number of multiplications if one
would allow large numbers.
numbers Find an algorithm that
uses only three multiplications (but more additions)
to multiply
p y two complex
p numbers.
Divide and conquer 6
Matrix Multiplication
Let A and B to be two n*n matrices
matrices.. We wish to compute their
product C=AB, n = 2k ; for some nonnegative integer k
Divide and conquer 7
Complexity (on uniprocessor)
• Best known lower bound: Ω(n2)
(assume m = Θ(n) and k = Θ(n) )
• Straightforward algorithm: O(n3).
• Strassen's algorithm: O(nlog 7) = O(n2.81). [1969]
[1969]
• Best known sequential algorithm: O(n2.376) [1990]
[1990]
• The best algorithm for this problem is still open.
Divide and conquer 8
Strassen’s Matrix Multiplication
• History
Volker Strassen is a German mathematician
born in 1936. He is well known for his works on
probability, but in the computer science and
algorithms he’s mostly recognized because of
his algorithm for matrix multiplication that’s still
one of the main methods that outperforms the
general matrix multiplication algorithm.
Strassen firstly published this algorithm in 1969
and proved that the n3 algorithm isn’t the
optimal one. Actually the given solution by
Strassen is slightly better, but his contribution is
enormous because this resulted in many more
researches about matrix multiplication that led
to some faster approaches, i.e. the
Coppersmith‐‐Winograd
Coppersmith algorithm with
O(n2,3737).
Divide and conquer 9
Matrix Multiplication
Divide and conquer 10
The Straightforward Method
• It takes O(
O(mm⋅n⋅k) = O(n3) time.
Divide and conquer 11
Conventional Matrix Multiplication
8 multiplications
4 additions
• Brute‐force algorithm
c00 c01 a00 a01 b00 b01
= **
c10 c11 a10 a11 b10 b11
a00 * b00 + a01 * b10 a00 * b01 + a01 * b11
=
a10 * b
* b00 + a + a11 * b
* b10 a10 * b
* b01 + a
+ a11 * b
* b11
Efficiency class in general: Θ (n3)
Divide and conquer 12
A Divide & Conquer
A Divide & Conquer approach
approach
Divide and conquer 13
Strassen’s Matrix Multiplication
• Strassen’s algorithm for two 2x2 matrices (1969):
c00 c01 a00 a01 b00 b01
=
= *
c10 c11 a10 a11 b10 b11
m1 + m4 ‐ m5 + m7 m3 + m5
=
m2 + m4 m1 + m3 ‐ m2 + m6
• m1 = (a
(a00 + a
+ a11)
) * (b00 + b
+ b11)
• m2 = (a10 + a11) * b00 This requires 7 multiplications and
• m3 = a00 * (b01 ‐ b11)
• m4 = a11 * (b10 ‐ b00) 10 additions
• m5 = (a
( 00 + a01) *
) * b11
• m6 = (a10 ‐ a00) * (b00 + b01)
• m7 = (a01 ‐ a11) * (b10 + b11)
Divide and conquer 14
Strassen's algorithm
• T(n) 7T(n/2) +O(n2)
T(n) = 7T(n/2) +O(n )
= O(nlog 7) = O(n2.81)
Divide and conquer 15
Time Analysis
Divide and conquer 16
Time Analysis
• With constant a and b
• Using master’ s theorem
• as a > bk
Divide and conquer 17
Recurrence for the number of additions required
by Strassens’s
by Strassens
by Strassens s algorithm
• Solve the recurrence relation ?
Divide and conquer 18
Strassen Algorithm
void matmul(int *A, int *B, int *R, int n) {
if (n == 1) {
Divide matrices in
(*R) += (*A) * (*B); sub-matrices and
} else { recursively multiply
matmul(A, B, R, n/4); sub-matrices
matmul(A, B+(n/4), R+(n/4), n/4);
matmul(A+2*(n/4), B, R+2*(n/4), n/4);
matmul(A+2*(n/4), B+(n/4), R+3*(n/4), n/4);
matmul(A+(n/4), B+2*(n/4), R, n/4);
matmul(A+(n/4) B+3*(n/4),
matmul(A+(n/4), B+3*(n/4) R+(n/4),
R+(n/4) n/4);
matmul(A+3*(n/4), B+2*(n/4), R+2*(n/4), n/4);
matmul(A+3*(n/4), B+3*(n/4), R+3*(n/4), n/4);
}
Divide and conquer 19
Performance analysis
Divide and conquer 20
Algorithms for Multiplying Two n
Algorithms for Multiplying Two n×n Matrices
Divide and conquer 21
Limitations
• Generally Strassen’s Method is not preferred for practical
applications for following reasons.
1. The constants used in Strassen’s method are high and for a
typical application Naive method works better.
2 For Sparse matrices,
2. matrices there are better methods especially
designed for them.
3. The sub‐matrices in recursion take extra space.
4. Because of the limited precision of computer arithmetic on
non‐integer values, larger errors accumulate in Strassen’s
algorithm than in Naive Method .
Divide and conquer 22
Sample problems
1 Use Strassens
1. Strassens’ss matrix multiplication algorithm to multiply
the matrices
2. Program Strassens’s matrix multiplication algorithm as well
as classical matrix multiplication
p to ((show that)) determine
when Strassens’s matrix multiplication method outperforms
the classical one.
3. Solve the recurrence for the number of additions required by
Strassens’s algorithm assuming that n is power of 2.
4. Practical implementation of Strassens’s matrix multiplication
algorithm usually switch to the brute force method after
matrix sizes become smaller than some crossover ppoint. Run
an experiment to determine such crossover point on your
computer
Divide and system.
conquer 23
Sample problems
5. Why is it important that Strassen’s algorithm does not use
commutativity in the multiplication of 2 X2 matrices?
6. Show how to multiply two complex number X=a+ib and
Y=c+id using only three multiplications.
7 [a] Show that how to multiply two numbers by solving five
7.
problems that are roughly one third of the original size.
[b] Generalized this problem to obtain an O(N1+∈) algorithm
for any constant ∈ > 0.
[c]Is the algorithm in part (b) better than O(N log N).
Divide and conquer 24
Sample problems
8. Using the "ordinary" method for multiplying 4‐by‐4
matrices, how many multiplications of numbers will be
needed? Give a reason for your answer,
answer more than just
that your answer is given by a formula.
9. Use Strassen's method for multiplyingp y g matrices to see
how many multiplications of numbers would be
required to multiply 4‐by‐4 matrices in this case. [Hint:
you must not only use Strassen
Strassen'ss basic method,
method but also
the divide‐and‐conquer strategy.] Give a reason for your
answer. For full credit you must explain in some detail
how Strassen
Strassen'ss basic result allows one to multiply the 4‐
4
by‐4 matrices using the number of multiplications that
you indicate.
Divide and conquer 25
Sample problems
10. Matrix multiplication Strassen's algorithm multiplies
two n×n‐matrices in time O(n2.808) by decomposition of
2×2‐block matrices. It is faster than O(n3) because it
computes seven multiplications instead of eight to form
the product matrix.
matrix Another idea is to make the
decomposition of 3×3‐block matrices instead. A
researcher at NADA tried a couple of years ago to find
the minimum number of multiplications needed to
multiply two 3× 3‐matrices. He managed to get almost
22 multiplications.
multiplications If he had succeeded,
succeeded what would
have been the time complexity to the multiplication of
two n× n matrices?
Divide and conquer 26
Problems on Page no 194‐
Problems on Page no 194‐195
Divide and conquer 27
Thanks for Your Attention!
Divide and conquer 28
Divide and conquer 29
