Divide and Conquer

Introduction
•The most-well known algorithm design strategy:

• Divide instance of problem into two or more smaller instances

•Solve smaller instances recursively

•These sub problems are solved independently

•Obtain solution to original (larger) instance by combining these solutions

•Combining all the solutions of sub problems into a solution of the whole.
•If the sub problems are large enough then divide and conquer is
reapplied

•The generated sub problems are usually of the same nature as the
original problem - hence recursive algorithms are used in Divide and
Conquer
 Divide and Conquer

a problem of size n
(instance)

subproblem 1 subproblem 2
of size n/2 of size n/2

a solution to a solution to
subproblem 1 subproblem 2

a solution to It general leads to a

the original problem
recursive algorithm!
 Divide and Conquer
• A control abstraction of Divide and Conquer is as given below:
• Algorithm DC(P)
• {
• If P is too small then
• return solution of P
• else
• {
• Divide (P) and obtain P1,P2,P3,…Pn
• where n>= 1
• Apply DC to each subproblem
• return combine (DC(P1),DC(P2),….DC(Pn));
• }
• }
 Divide and Conquer
• The computing time of the DC procedure is given by the recurrence
relation.

• Tn = g(n) if n is small
• T(n1) + T(n2) +…….T(nn) + F(n)
• Where T(n) is the time for Divide and Conquer of size n. g(n) is the
computing time reqd. to solve small inputs. F(n) is the time reqd. for
dividing the problem P and combining the solutions to sub problems.

• In the following slides we will be discussing some of the applications

of D and C such as Binary search, MinMax, Quicksort, Merge sort
and Radix sort.
 General Divide-and-Conquer Recurrence

T(n) = aT(n/b) + f (n) where f(n)  (nd), d  0

Master Theorem: If a < bd, T(n)  (nd)

If a = bd, T(n)  (nd log n)
If a > bd, T(n)  (nlog b a )

Examples: T(n) = 4T(n/2) + n  T(n)  ?

T(n) = 4T(n/2) + n2  T(n)  ?
T(n) = 4T(n/2) + n3  T(n)  ?
 Binary Search
• It is an efficient searching method.
• Requires that the elements of the array be sorted.
• An element which is to be searched from the list of elements stored
in array A[0….n-1] is called KEY element.
• Let A[m] be the mid element of array A. Then there are 3 conditions
that need to be tested while searching the array:
• 1. If KEY=A[m] then desired element is present in the list.
• 2. Otherwise if KEY < A[m] then search the left sub list
• 3. Otherwise if KEY > A[m] then search the right sub list.

A[0]…A[m-1] A[m] A[m+1…A[n-1]

Search here if Search here if Search here if

key < A[m] key = A[m] key > A[m]
 Divide and Conquer
0 1 2 3 4 5 6

10 20 30 40 50 60 70

Low Mid High

The key element is 60. (element to be searched).

To get middle element we will apply a formula as
M=(low+high)/2
M=(0+6)/2=3
Is A[3]=key No
A[3]=40 and 40 < 60 which is the key.
So check in right sub list
M=(4+6)/2=5
Is A[5]=key Yes
SO key element 60 is present in list at position 5.
 Divide and Conquer
Algorithm Binsearch(A[0..n-1],key)
//Problem Description: searching an element using Binary search method
//Input: An array A from which the key element is to be searched.
//Output: returns the index of an array element if it is equal to key otherwise
It returns -1.
Low 0
High n-1
While (low < high) do
{
m (low +high)/2 //mid of the array is obtained
If (key = A[m]) then
return m;
Else if (key < A[m]) then
high m-1; // search left sub list
Else
low m+1; // search right sub list
}
Return -1; // if element is not in list
 Analysis of Binary Search Algorithm
• The basic operation in Binary search is the comparison of Key
element (search element) with array elements.
• We need to count the number of times the key element is compared
with the array elements.
• The comparison is also called a 3 way comparison because
comparison is made whether Key element is equal to , greater than
or less than A[m].
• Best Case: if after one comparison mid element is key element then
Best case efficiency is 1
• Cbest (n) = 1
 Analysis of Binary Search Algorithm
Worst Case : the worst case efficiency is when the algorithm compares all the
Array elements for searching the desired element.
In this method one comparison is made and based on this comparison the array
Is divided each time into n/2 sub lists. Hence the worst case time complexity is
given by:

C worst (n) = C worst(n/2) + 1

Time reqd to One comparison

Compare left or Made with middle
Right sublist element
Analysis of Binary Search Algorithm
Analysis of Binary Search Algorithm
Analysis of Binary Search Algorithm
Analysis of Binary Search Algorithm
 Analysis of Binary Search Algorithm

• Time Complexity of Binary Search

Best Case Average Case Worst Case

1 Log n to the base 2 Log n to the base 2

Both Recursive and Non-recursive implementations of Binary Search are

possible.
Time efficiency
worst-case recurrence: Cw (n) = 1 + Cw( n/2 ), Cw (1) = 1
solution: Cw(n) = log2(n+1)

This is VERY fast: e.g., Cw(106) = 20

Optimal for searching a sorted array

Limitations: must be a sorted array (not linked list)

Min-Max Algorithm
Min-Max Algorithm
Min-Max Algorithm
Min-Max Algorithm
Min-Max Algorithm
Min-Max Algorithm
Min-Max Algorithm
Min-Max Algorithm
Min-Max Algorithm
Master Theorem
Master Theorem
Merge Sort
Merge Sort
Merge Sort
Merge Sort
Merge Sort
Merge Sort
Merge Sort
Merge Sort
Merge Sort
Merge Sort
Merge Sort
Merge Sort
Quick Sort
Quick Sort
Quick Sort
Quick Sort
Quick Sort
Quick Sort
Quick Sort
Quick Sort
Quick Sort
Quick Sort
Quick Sort
Quick Sort
Randomized Quick Sort
Randomized Quick Sort
 Conventional Matrix Multiplication

C00 c01 a00 a01 * b00 b01

=
C10 c11 a10 a11 b10 b11

a00 * b00 + a01 * b10 a00 * b01 + a01 * b11

=
a10 * b00 + a11 * b10 a10 * b01 + a11 * b11

8 multiplications 4 additions Efficiency class in general:  (n )

3
 Strassen’s Matrix Multiplication

• Strassen’s algorithm for two 2x2 matrices (1969):

c00 c01 a00 a01 b00 b01

c10 c11 = a10 a11 * b10 b11

m1 + m4 - m5 + m7 m3 + m5
=
m2 + m4 m1 + m3 - m2 + m6

m1 = (a00 + a11) * (b00 + b11)

m2 = (a10 + a11) * b00
m3 = a00 * (b01 - b11)
7 multiplications
m4 = a11 * (b10 - b00)
m5 = (a00 + a01) * b11
m6 = (a10 - a00) * (b00 + b01)
18 additions
m7 = (a01 - a11) * (b10 + b11)
 Strassen’s Matrix Multiplication

c00 c01 a00 a01 b00 b01

c10 c11 a10 a11 b 10 b11

M1 + M4 - M5 + M7 M3 + M5
=
M2 + M4 M1 + M3 - M2 + M6
 Formulae for Strassen’s Algorithm

M1 = (A00 + A11)  (B00 + B11)

M2 = (A10 + A11)  B00

M3 = A00  (B01 - B11)

M4 = A11  (B10 - B00)

M5 = (A00 + A01)  B11

M6 = (A10 - A00)  (B00 + B01)

M7 = (A01 - A11)  (B10 + B11)

 Complexity Analysis of Conventional Matrix Multiplication
For I = 0 to n-1 do
For j = 0 to n-1 do
c[i][j] 0
For k 0 to n-1 do
c[i][j] c[i][j] + a[i][k] * b[k][j]
Return (c)
The basic operation only depends on the input size. There is no best
case, worst case or average case efficiencies.
The sum can be denoted as
M(n) =outermost loop* inner loop*innermost loop
M(n) = (for loop using i)*(for loop using j) * (for loop using k)
N-1 N-1 N-1
M(n) = Σ Σ Σ
1
i=0 j=0 k=0

=(n-1)-0+1=n
N-1
M(n)= Σ n^2 = n^3 M(n) = θ (n3)
i=0