Design and Analysis of Algorithms

Dynamic Programming
(Matrix Chain Multiplication)

Dr. D. P. Acharjya
Professor, SCOPE
Office: SJT Annex 201E
Email: debiprasannaacharjya@vit.ac.in

2/10/2024 Dr. D. P. Acharjya 1

Matrix Multiplication
 Let A = [aij]mn and B = [bjk]n p be two matrices. The
multiplication of A and B is defined as:
C = AB = [cik]mp
 For Example:
 b11 b12 
 a11 a13 
B  b21 b22 
a12
A
 a21 a22 a23  23
b31 b32  32

 a11b11  a12b21  a13b31 a11b12  a12b22  a13b32 

C
 a21b11  a22b21  a23b31 a21b12  a22b22  a23b32  22
 Number of scalar multiplications = 23 2 = 12
2/10/2024 Dr. D. P. Acharjya 2
Matrix Chain Multiplication
 If A = [aij]mn and B = [bjk]n p be two matrices. Then
the number of scalar multiplications in
C = AB is (m  n  p).
 Given a sequence of matrices A1, A2, A3, …, An, where
order of Ai is (pi-1  pi) for i = 1, 2, 3, …, n to be
multiplied in such a manner that the total number of
scalar multiplications is minimum.
 For example, if the chain of matrices is A1, A2, A3, A4,
the product A1A2A3A4 can be fully parenthesized in:
(A1 (A2 (A3 A4))), (A1((A2 A3) A4)), ((A1A2)(A3A4)) ,
((A1(A2A3))A4), or (((A1A2) A3) A4)
2/10/2024 Dr. D. P. Acharjya 3
Continued …
 Consider three matrices A1 = [aij]25, A2 = [bij]56, and
A3 = [cij]610 be three matrices.
 These three matrices can be multiplied in two different
ways such as A1(A2A3) or (A1A2)A3.
 First Case: A1(A2A3)
 The order of [A2A3]510
 Scalar multiplications in A2A3 = 56 10 = 300
 The order of [A1A2A3] 210 = [A1] 25[A2A3] 510
 Scalar multiplications [A1]25[A2A3]510=2510= 100
 Total scalar multiplications in [A1A2A3]
= 300+100 = 400
2/10/2024 Dr. D. P. Acharjya 4
Continued …
 Second Case: (A1A2)A3
 The order of [A1A2]26
 Scalar multiplications in A1A2 = 25 6 = 60
 The order of [A1A2A3] 210 = [A1A2] 26[A3] 610
 Scalar multiplications [A1A2]26[A3] 610 = 2610 =
120
 Total scalar multiplications in [A1A2A3]
= 60+120 = 180
The optimal number of scalar multiplication is: 180
The optimal Solution: (A1A2)A3
2/10/2024 Dr. D. P. Acharjya 5
Recurrence Relation
 Let m[i, j] be the notion for scalar multiplications
when multiplying matrices Ai to Aj in sequence
 Therefore:

i min {m[i, k ]  m[k  1, j ]  pi 1 pk p j } when i  j
m[i, j ]    k  j
0 when i  j

 Where k stands for the number of matrices before
the partition.

2/10/2024 Dr. D. P. Acharjya 6

Analysis of Recurrence Relation
 Assume the earlier Ex: [A1]25, [A2]56, and [A3]610
 Here, (p0p1) = 25; (p1p2) = 56; (p2p3) = 610
 First Case: A1(A2A3)
 The order of [A2A3]510
m[2,3]  m[2, 2]  m[3,3]  p1 p2 p3
 0  0  5  6 10  300
 Now, in [A1A2A3] = [A1][A2A3]
m[1,3]  m[1,1]  m[2,3]  p0 p1 p3
 0  300  2  5 10  400
2/10/2024 Dr. D. P. Acharjya 7
Continued …
 Assume the earlier Ex: [A1]25, [A2]56, and [A3]610
 Here, (p0p1) = 25; (p1p2) = 56; (p2p3) = 610
 Second Case: (A1A2)A3
 The order of [A1A2]26
m[1, 2]  m[1,1]  m[2, 2]  p0 p1 p2
 0  0  2  5  6  60
 Now, in [A1A2A3] = [A1A2]A3
m[1,3]  m[1, 2]  m[3,3]  p0 p2 p3
 60  0  2  6 10  180
2/10/2024 Dr. D. P. Acharjya 8
Continued …
 Finally,
m[1,1]  m[2,3]  p0 p1 p3

 0  300  2  5 10  400
m[1,3]  Min 
m[1, 2]  m[3,3]  p0 p2 p3
 60  0  2  6 10  180
 Therefore,
[A1A2A3] = [A1A2]A3

2/10/2024 Dr. D. P. Acharjya 9

Numerical Illustration
 Consider [A1]23, [A2]36, [A3]65, and [A4]57
 Here, (p0p1) = 23; (p1p2) = 36; (p2p3) = 65;
and (p3p4) = 57
 Create a table of size (4  4)
m[1,1]  0 m 1 2 3 4

m[2, 2]  0 1 0
2 0
m[3,3]  0 3 0
m[4, 4]  0 4 0

In all these cases i  j

2/10/2024 Dr. D. P. Acharjya 10
Continued …
 Compute m[1, 2], m[2, 3], m[3, 4]
m[1, 2]  m[1,1]  m[2, 2]  p0 p1 p2
 0  0  2  5  6  60
Here, i  1, j  2, and k  1
m 1 2 3 4
1 0 60
2 0
3 0
4 0

2/10/2024 Dr. D. P. Acharjya 11

Continued …
 m[2,3]  m[2, 2]  m[3,3]  p1 p2 p3
 0  0  3  6  5  90
Here, i  2, j  3, and k  2 m 1 2 3 4
1 0 60

m[3, 4]  m[3,3]  m[4, 4] 2 0 90

3 0 210
 p2 p3 p4
4 0
 0  0  6  5  7  210
Here, i  3, j  4, and k  3
2/10/2024 Dr. D. P. Acharjya 12
Continued …

m[1,1]  m[2,3]  p0 p1 p3

 0  90  2  3  5  120
m[1,3]  Min 
m[1, 2]  m[3,3]  p0 p2 p3
 60  0  2  6  5  120
m 1 2 3 4
Here, i  1, j  3, and k  1, 2 1 0 60 120
2 0 90
3 0 210
4 0

2/10/2024 Dr. D. P. Acharjya 13

Continued …

m[2, 2]  m[3, 4]  p1 p2 p4
 0  210  3  6  7  336

m[2, 4]  Min 
m[2,3]  m[4, 4]  p1 p3 p4
 90  0  3  5  7  195
m 1 2 3 4
Here, i  2, j  4, and k  2,3 1 0 60 120
2 0 90 195
3 0 210
4 0

2/10/2024 Dr. D. P. Acharjya 14

Continued …
 m[1, 4] 
m[1,1]  m[2, 4]  p0 p1 p4

 0  195  2  3  7  237

m[1, 2]  m[3, 4]  p0 p2 p4 m 1 2 3 4
Min  1 0 60 120 190
 60  210  2  6  7  354 2 0 90 195
m[1, 3]  m[4, 4]  p0 p3 p4
 3 0 210
 120  0  2  5  7  190
 4 0
Here, i  1, j  4, and k  1, 2, 3
 Solution is: ((A1(A2A3))A4) or (((A1A2)A3)A4)

2/10/2024 Dr. D. P. Acharjya 15