Unit #3

Divide and Conquer Algorithm

Tejas Chauhan
Divide and Conquer
› Recursive in structure
› Divide the problem into sub-problems that are
similar to the original but smaller in size

› Conquer the sub-problems by solving them

recursively. If they are small enough, just solve
them in a straightforward manner.

› Combine the solutions to create a solution to the

original problem

Thursday, March 21, 2024 Tejas Chauhan 2

An Example: Merge Sort
Sorting Problem:
Sort a sequence of n elements into increasing order.

› Divide: Divide the n-element sequence to be

sorted into two subsequences of n/2 elements
each
› Conquer: Sort the two subsequences recursively
using merge sort.
› Combine: Merge the two sorted subsequences to
produce the sorted answer.
Thursday, March 21, 2024 Tejas Chauhan 3
Analysis of Merge Sort
› Running time T(n) of Merge Sort:
› Divide: computing the middle takes (1)
› Conquer: solving 2 sub-problems takes 2T(n/2)
› Combine: merging n elements takes (n)
› Total:
T(n) = (1) if n = 1
T(n) = 2T(n/2) + (n) if n > 1
T(n) = (n log n)

Thursday, March 21, 2024 Tejas Chauhan 4

Recurrence Relations
› Recurrence Relation:
› Equation or an inequality that characterizes a
function by its values on smaller inputs.

› Recurrence relations arise when we analyze

the running time of iterative or recursive
algorithms.
› Ex: Divide and Conquer.
T(n) = (1) if n  c
T(n) = aT(n/b) + D(n) + C(n) otherwise

Thursday, March 21, 2024 Tejas Chauhan 5

Recurrence Relations
› Solution Methods:
› Substitution Method.
› Recursion-tree Method.
› Master Method.

Thursday, March 21, 2024 Tejas Chauhan 6

Substitution Method
› Guess the form of the solution, then
use mathematical induction to show it
correct.
› Substitute guessed answer for the function when
the inductive hypothesis is applied to smaller
values – hence, the name.
› Works well when the solution is easy to
guess.
› No general way to guess the correct solution.

Thursday, March 21, 2024 Tejas Chauhan 7

Example – Exact Function
Recurrence: T(n) = 1 if n = 1
T(n) = 2T(n/2) + n if n > 1
Guess: T(n) = n log n + n.
Induction:
Basis: n = 1  n logn + n = 1 = T(n).
Hypothesis: T(k) = k log k + k for all k < n.
Inductive Step: T(n) = 2 T(n/2) + n
= 2 ((n/2)log(n/2) + (n/2)) + n
= n (log(n/2)) + 2n
= n log n – n + 2n
= n log n + n
Thursday, March 21, 2024 Tejas Chauhan 8
Recursion-tree Method
› Making a good guess is sometimes difficult
with the substitution method.

› Use recursion trees to devise good guesses.

Thursday, March 21, 2024 Tejas Chauhan 9

Recursion Tree – Example
› Running time of Merge Sort:
T(n) = (1) if n = 1
T(n) = 2T(n/2) + (n) if n > 1
› Rewrite the recurrence as
T(n) = c if n = 1
T(n) = 2T(n/2) + cn if n > 1
c > 0: Running time for the base case and time
per array element for the divide and combine
steps.

Thursday, March 21, 2024 Tejas Chauhan 10

Recursion Tree for Merge Sort
For the original problem, Each of the size n/2
we have a cost of cn, problems has a cost of cn/2
plus two subproblems plus two subproblems,
each of size (n/2) and each costing T(n/4).
running time T(n/2). cn

cn Cost of divide and

merge.

cn/2 cn/2

T(n/2) T(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
Cost of sorting
subproblems.
Thursday, March 21, 2024 Tejas Chauhan 11
 Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.
cn cn

cn/2 cn/2 cn

log n
cn/4 cn/4 cn/4 cn/4 cn

c c c c c c cn
Total : cn logn+cn
Thursday, March 21, 2024 Tejas Chauhan 12
Recursion Tree for Merge Sort
Continue expanding until the problem size reduces to 1.
cn
There are log n+1 levels,
height is log n.

cn/2 cn/2 Total cost

= sum of costs at each level
= (log n+1)cn
cn/4 cn/4 cn/4 cn/4 = cn logn + cn
= (n logn).

c c c c c c
Thursday, March 21, 2024 Tejas Chauhan 13
Recursion Trees – Caution Note
› Recursion trees only generate guesses.
› Verify guesses using substitution method.

› If careful when drawing out a recursion tree

and summing the costs, can be used as
direct proof.

Thursday, March 21, 2024 Tejas Chauhan 14

Recursion Tree – Example
Solve T(n) = T(n/4) + T(n/2) + n2:

n
2 n2
5 2
(n/4)2
T(n/4) T(n/2)
(n/2) 2 n
16
25 2
(n/16)2
T(n/16) (n/8)2
T(n/8) (n/8)
T(n/8)2
T(n/4)
(n/4)2 n
256

…
…


2 5  5 
2
 5 
3

Total = n 1         ... 
Q(1) 16  16   16  
 
= Q(n2)
Thursday, March 21, 2024 Tejas Chauhan 15
The Master Method
› The master method applies to recurrences of
the form
T(n) = a T(n/b) + f (n) ,
› where a ≥ 1, b > 1, and f is asymptotically
positive.

Thursday, March 21, 2024 Tejas Chauhan 16

 Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

…
…

ah = alogbn nlogbaT (1)

T (1) = nlogba

Thursday, March 21, 2024 Tejas Chauhan 17

Three common cases
Compare f (n) with nlogba:
1. f (n) = O(nlogba – e) for some constant e > 0.
• f (n) grows polynomially slower than nlogba
(by an ne factor).
Solution: T(n) = Q(nlogba) .

Thursday, March 21, 2024 Tejas Chauhan 18

 Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

…
…

CASE 1: The weight increases nlogbaT (1)

T (1) geometrically from the root to the
leaves. The leaves hold a constant
fraction of the total weight. Q(nlogba)

Thursday, March 21, 2024 Tejas Chauhan 19

Three common cases
Compare f (n) with nlogba:

2. f (n) = Q(nlogba lgkn) for some constant k ≥ 0.

• f (n) and nlogba grow at similar rates.
Solution: T(n) = Q(nlogba lgk+1n).

Thursday, March 21, 2024 Tejas Chauhan 20

 Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

…
…

CASE 2: (k = 0) The weight is nlogbaT (1)

T (1) approximately the same on each
of the logbn levels. Q(nlogbalg n)

Thursday, March 21, 2024 Tejas Chauhan 21

Three common cases
Compare f (n) with nlogba:
3. f (n) = W(nlogba + e) for some constant e > 0.
• f (n) grows polynomially faster than nlogba (by
an ne factor),
Solution: T(n) = Q(f (n)).

Thursday, March 21, 2024 Tejas Chauhan 22

 Idea of master theorem
Recursion tree:
f (n) f (n)
a
f (n/b) f (n/b) … f (n/b) a f (n/b)
h = logbn a
f (n/b2) f (n/b2) … f (n/b2) a2 f (n/b2)

…
…

CASE 3: The weight decreases nlogbaT (1)

T (1) geometrically from the root to the
leaves. The root holds a constant
fraction of the total weight. Q(f (n))

Thursday, March 21, 2024 Tejas Chauhan 23

Examples

Ex. T(n) = 4T(n/2) + n

a = 4, b = 2  nlogba = n2; f (n) = n.
CASE 1: f (n) = O(n2 – e) for e = 1.
 T(n) = Q(n2).

Ex. T(n) = 4T(n/2) + n2

a = 4, b = 2  nlogba = n2; f (n) = n2.
CASE 2: f (n) = Q(n2lg0n), that is, k = 0.
 T(n) = Q(n2lg n).

Thursday, March 21, 2024 Tejas Chauhan 24

Examples
Ex. T(n) = 4T(n/2) + n3
a = 4, b = 2  nlogba = n2; f (n) = n3.
CASE 3: f (n) = W(n2 + e) for e = 1
and 4(cn/2)3 ≤ cn3 (reg. cond.) for c = 1/2.
 T(n) = Q(n3).

Thursday, March 21, 2024 Tejas Chauhan 25

The Master Theorem
Theorem:
Let a  1 and b > 1 be constants, let f(n) be a function, and
Let T(n) be defined on nonnegative integers by the recurrence
T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b.

T(n) can be bounded asymptotically in three cases:

1. If f(n) = O(nlogba–) for some constant  > 0,
then T(n) = (nlogba).
2. If f(n) = (nlogbalgkn) for some constant k ≥ 0,
then T(n) = (nlogbalgk+1n).
3. If f(n) = (nlogba+) for some constant  > 0,
then T(n) = (f(n)).

Thursday, March 21, 2024 Tejas Chauhan 26

 Applications of
Divide and Conquer

Thursday, March 21, 2024

Tejas Chauhan 27
Applications of Divide and Conquer
› Merge Sort
› Quick Sort
› Binary Search
› Multiplication of large Integers
› Max-Min Problem
› Matrix Multiplication
› Exponential

Thursday, March 21, 2024 Tejas Chauhan 28

Multiplication of large integers
› 981 x 1234 = ?
› divide the integer in two portion left and right.
(12 and 34)
› 1234 = 102 x 12 + 34 = 1234
› 0981 x 1234: consider following things,
› w = 09, x = 81, y = 12 and z = 34
› 0981 x 1234 = (102w + x ) x (102y + z)
=104 wy + 102(wz + xy) + xz
=1080000 + 1278000 + 2754
= 1210554
Thursday, March 21, 2024 Tejas Chauhan 29
Multiplication of large integers
› The above procedure still needs four half-size
multiplications; wy, wz, xy and xz.

› The key observation is that there is no need to

compute both wz and xy; all we really need is the
sum of these two terms.

› Is it possible to obtain wz + xy at the cost of single

multiplication? Our equeation is like

› Result = (w + x) x (y + z) = wy + (wz + xy) + xz

Thursday, March 21, 2024 Tejas Chauhan 30
Multiplication of large integers
› There is one mathematical formula that we are
going to apply here
› (wz + xy) = (w + x) x (y + z) – wy – xz

› p = wy = 09 x 12 = 108
› q = xz = 81 x 34 = 2754
› r = (w + x) x (y + z) = 90 x 46 = 4140

› 0981 x 1234 = 104 p + 102(r-p-q) + q

= 1080000 + 127800 + 2754
= 1210554
Thursday, March 21, 2024 Tejas Chauhan 31
Multiplication of large integers

0981 x 1234

09 x 12 81 x 34 90 x 46

0x1 9x2 9x3

› T(n) = 3T(n/2) + g(n)

Thursday, March 21, 2024 Tejas Chauhan 32

Multiplication of large integers
Procedure Multiply(X,Y)
n=max(sizeof X, sizeof Y)
if n=1: return XY

xL, xR = leftmost n / 2, rightmost n / 2, bits of X

yL, yR = leftmost n / 2, rightmost n / 2, bits of Y

P1 = Multiply(xL, yL)
P2 = Multiply(xR, yR)
P3 = Multiply(xL+xR, yL+yR)
return P1x2n + (P3-P1-P2)x2n/2 + P2

Thursday, March 21, 2024 Tejas Chauhan 33

Max-Min Problem
› Obvious strategy (Needs 2n - 3 compares)
› Find MAX (n - 1 compares)
› Find MIN of the remaining elements (n - 2)
› Nontrivial strategy
› Split the array in half
› Find the MAX and MIN of both halves
› Compare the 2 MAXes and compare the 2 MINs
to get overall MAX and MIN.

Thursday, March 21, 2024 Tejas Chauhan 34

 Matrix Multiplication
0 1 2 3 4 0 1 2 3 4 0 1 2 3 4
0
1
2 X =
3
4
A B C
C[1][1] = A[1][0]*B[0][1]+ C[3][2] = A[3][0]*B[0][2]+
A[1][1]*B[1][1]+ A[3][1]*B[1][2]+
A[1][2]*B[2][1]+ A[3][2]*B[2][2]+
A[1][3]*B[3][1]+ A[3][3]*B[3][2]+
A[1][4]*B[4][1] A[3][4]*B[4][2]

C[i][j] = sum(A[i][k]*B[k][j]) for k=0 … n

Thursday, March 21, 2024 Tejas Chauhan 35
Matrix Multiplication
Procedure Matrix-Multiply(A,B)
if columns[A] ≠ rows[B]
then error “incompatible dimensions”
else for i = 1 to rows[A] do
for j = 1 to columns[B] do
C[i][j] = 0
for k = 1 to columns[A] do
C[i][j] = C[i][j] + A[i][k]*B[k][j]
return C

› T(n) = O(n3)

Thursday, March 21, 2024 Tejas Chauhan 36

Strassen’s Matrix Multiplication
0 1 0 1 0 1

0
X =
1

X Y XY

 A B E F   AE  BG AF  BH 
XY       
C D  G H  CE  DG CF  DH 

T(n) = 8T(n/2) + O(n2)

Thursday, March 21, 2024 Tejas Chauhan 37
Strassen’s Matrix Multiplication
P1 = A(F-H) P5 = (A+D)(E+H)
P2 = (A+B)H P6 = (B-D)(G+H)
P3 = (C+D)E P7 = (A-C)(E+F)
P4 = D(G-E)

 P5  P4  P2  P6 P1  P2 
XY   
 P3  P4 P1  P5  P3  P7 

T(n) = 7T(n/2) + O(n2) O(nlog27) ≈ O(n2.81)

Thursday, March 21, 2024 Tejas Chauhan 38

Exponentiation
› Let a and n be two integers.
› We wish to compute x = an.
› If n is small, the obvious algorithm is:

Function expoSeq(a, n) {
ra
for i  1 to n-1 do r  a*r
return r
}

› T(n) = O(n)
Thursday, March 21, 2024 Tejas Chauhan 39
Exponentiation
› With divide and conquer:

› an = return a if n = 1
2
return (a )
n/2
if n is even
return a * an-1 otherwise

›
2
a = a * a = a * (a ) …….
29 28 14

› Which involves only three multiplications and four

squaring instead of the 28 multiplication.
Thursday, March 21, 2024 Tejas Chauhan 40
Exponentiation
a29 = a * a28
= a * (a14)2
= a * ((a7)2)2
= a * (a * (a6)2)2
= a * (a * ((a3)2)2)2
= a * (a * ((a * a2)2)2)2

› Which involves only three multiplications and

four squaring instead of the 28 multiplication.
Thursday, March 21, 2024 Tejas Chauhan 41
Exponentiation
› Pseudo code for recursive exponentiation :

Function expoDC(a, n) {
if n = 1 then return a
if n is even then return [expoDC(a, n/2)]2
return a * expoDC(a, n - 1)
}

› T(n) = O(log n)

Thursday, March 21, 2024 Tejas Chauhan 42

 THANK YOU.

Thursday, March 21, 2024 Tejas Chauhan 43