Algorithm Design & Analysis

SE 208
Unit 4 -4 Dynamic Programming
Matrix Chain Multiplication
Divyashikha Sethia
Divyashikha@dtu.ac.in
DTU
 Matrix-chain Multiplication
• Suppose we have a sequence or chain
A1, A2, …, An of n matrices to be
multiplied
– That is, we want to compute the product
A1A2…An

• There are many possible ways

(parenthesizations) to compute the
product

Divyashikha Sethia (DTU)

11-2
Matrix-chain Multiplication …contd

• Example: consider the chain A1, A2, A3,

A4 of 4 matrices
– Let us compute the product A1A2A3A4
• There are 5 possible ways:
1. (A1(A2(A3A4)))
2. (A1((A2A3)A4))
3. ((A1A2)(A3A4))
4. ((A1(A2A3))A4)
5. (((A1A2)A3)A4)
Divyashikha Sethia (DTU)
11-3
Matrix-chain Multiplication …contd

• To compute the number of scalar

multiplications necessary, we must know:
– Algorithm to multiply two matrices
– Matrix dimensions

• Can you write the algorithm to multiply

two matrices?

Divyashikha Sethia (DTU)

11-4
 Algorithm to Multiply 2 Matrices
Input: Matrices Ap×q and Bq×r (with dimensions p×q and q×r)
Result: Matrix Cp×r resulting from the product A·B

MATRIX-MULTIPLY(Ap×q , Bq×r)
1. for i ← 1 to p
2. for j ← 1 to r
3. C[i, j] ← 0
4. for k ← 1 to q
5. C[i, j] ← C[i, j] + A[i, k] · B[k, j]
6. return C
Scalar multiplication in line 5 dominates time to compute
CNumber of scalar multiplications = pqr
Divyashikha Sethia (DTU)
11-5
Matrix-chain Multiplication …contd

• Example: Consider three matrices

A10100, B1005, and C550
• There are 2 ways to parenthesize
– ((AB)C) = D105 · C550
• AB  10·100·5=5,000 scalar multiplications Total:
• DC  10·5·50 =2,500 scalar multiplications 7,500
– (A(BC)) = A10100 · E10050
• BC  100·5·50=25,000 scalar multiplications
• AE  10·100·50 =50,000 scalar multiplications
Total:
Divyashikha Sethia (DTU)
75,000 11-6
Matrix-chain Multiplication …contd

• Matrix-chain multiplication problem

– Given a chain A1, A2, …, An of n matrices,
where for i=1, 2, …, n, matrix Ai has
dimension pi-1pi
– Parenthesize the product A1A2…An such that
the total number of scalar multiplications is
minimized
• Brute force method of exhaustive search
takes time exponential in n
Divyashikha Sethia (DTU)
11-7
 Dynamic Programming Approach
• The structure of an optimal solution
– Let us use the notation Ai..j for the matrix that
results from the product Ai Ai+1 … Aj
– An optimal parenthesization of the product
A1A2…An splits the product between Ak and
Ak+1 for some integer k where1 ≤ k < n
– First compute matrices A1..k and Ak+1..n ; then
multiply them to get the final matrix A1..n

Divyashikha Sethia (DTU)

11-8
Dynamic Programming Approach
…contd

– Key observation: parenthesizations of the

subchains A1A2…Ak and Ak+1Ak+2…An must
also be optimal if the parenthesization of the
chain A1A2…An is optimal (why?)

– That is, the optimal solution to the problem

contains within it the optimal solution to
subproblems

Divyashikha Sethia (DTU)

11-9
 Dynamic Programming Approach
…contd

• Recursive definition of the value of an

optimal solution
– Let m[i, j] be the minimum number of scalar
multiplications necessary to compute Ai..j
– Minimum cost to compute A1..n is m[1, n]
– Suppose the optimal parenthesization of Ai..j
splits the product between Ak and Ak+1 for
some integer k where i ≤ k < j

Divyashikha Sethia (DTU)

11-10
Dynamic Programming Approach
…contd

– Ai..j = (Ai Ai+1…Ak)·(Ak+1Ak+2…Aj)= Ai..k · Ak+1..j

– Cost of computing Ai..j = cost of computing
Ai..k + cost of computing Ak+1..j + cost of
multiplying Ai..k and Ak+1..j
– Cost of multiplying Ai..k and Ak+1..j is pi-1pk pj

– m[i, j ] = m[i, k] + m[k+1, j ] + pi-1pk pj

for i ≤ k < j
– m[i, i ] = 0 for i=1,2,…,n

Divyashikha Sethia (DTU)

11-11
 Dynamic Programming Approach
…contd

– But… optimal parenthesization occurs at

one value of k among all possible i ≤ k < j
– Check all these and select the best one

0 if i=j
m[i, j ] =
min {m[i, k] + m[k+1, j ] + pi-1pk pj } if i<j
i ≤ k< j

Divyashikha Sethia (DTU)

11-12
 Dynamic Programming Approach
…contd

• To keep track of how to construct an

optimal solution, we use a table s
• s[i, j ] = value of k at which Ai Ai+1 … Aj is
split for optimal parenthesization
• Algorithm: next slide
– First computes costs for chains of length l=1
– Then for chains of length l=2,3, … and so on
– Computes the optimal cost bottom-up

Divyashikha Sethia (DTU)

11-13
Algorithm to Compute Optimal Cost
Input: Array p[0…n] containing matrix dimensions and n
Result: Minimum-cost table m and split table s

MATRIX-CHAIN-ORDER(p[ ], n)
for i ← 1 to n Takes O(n3) time
m[i, i] ← 0
Requires O(n 2) space
for l ← 2 to n
for i ← 1 to n-l+1
j ← i+l-1
m[i, j] ← 
for k ← i to j-1
q ← m[i, k] + m[k+1, j] + p[i-1] p[k] p[j]
if q < m[i, j]
m[i, j] ← q
s[i, j] ← k
return m and s
Divyashikha Sethia (DTU)
11-14
 Constructing Optimal Solution
• Our algorithm computes the minimum-
cost table m and the split table s
• The optimal solution can be constructed
from the split table s
– Each entry s[i, j ]=k shows where to split the
product Ai Ai+1 … Aj for the minimum cost

Divyashikha Sethia (DTU)

11-15
 Example
• Show how to multiply Matrix Dimension
this matrix chain
A1 30×35
optimally
A2 35×15
• Solution on the board A3 15×5
– Minimum cost 15,125
A4 5×10
– Optimal parenthesization
((A1(A2A3))((A4 A5)A6)) A5 10×20
A6 20×25

Divyashikha Sethia (DTU)

11-16
given the sequence {4, 10, 3, 12, 20, and 7}. The matrices have size 4 x 10, 10 x 3,
3 x 12, 12 x 20, 20 x 7. We need to compute M [i,j], 0 ≤ i, j≤ 5. We know M [i, i] = 0
for all i

Divyashikha Sethia (DTU)

11-17
proceed with working away from the diagonal.
We compute the optimal solution for the
product of 2 matrices

Divyashikha Sethia (DTU)

11-18
 {p= 4, 10, 3, 12, 20, and 7}.

Divyashikha Sethia (DTU)

11-19
 product of 3 matrices
M [1, 3] = M1 M2 M3

( M1 x M2) + M3 OR M1+ (M2x M3)

Divyashikha Sethia (DTU)

11-20
product of 3 matrices

Divyashikha Sethia (DTU)

11-21
Divyashikha Sethia (DTU)
11-22
Divyashikha Sethia (DTU)
11-23
Product of 4 matrices

Divyashikha Sethia (DTU)

11-24
Divyashikha Sethia (DTU)
11-25
Divyashikha Sethia (DTU)
11-26
Product of 5 matrices

Divyashikha Sethia (DTU)

11-27
Divyashikha Sethia (DTU)
11-28
 Example
mat={2,4,6,8,6}

Divyashikha Sethia (DTU)

11-29
 mat={2,4,6,8,6}

Divyashikha Sethia (DTU)

11-30
 mat={2,4,6,8,6}

Divyashikha Sethia (DTU)

11-31
Divyashikha Sethia (DTU)
11-32
Divyashikha Sethia (DTU)
11-33
Divyashikha Sethia (DTU)
11-34