Master Theorem for Divide and Conquer Recurrences

The Master Theorem is a tool used to solve recurrence relations that arise in the analysis of divide-and-conquer algorithms. The theorem gives asymptotic bounds on T(n) in terms of standard notations like Θ, O, and Ω. The Master Theorem provides a systematic way of solving recurrence relations of the following form

where:

n is the size of the problem
a ≥ 1 is the number of subproblems
n/b is the size of each subproblem (b > 1)
f(n) is the cost of work done outside the recursive calls (dividing the problem and combining results)

\footnotesize\begin{aligned}&\textbf{Master Theorem — Three Cases} \\&T(n) = aT(n/b) + f(n), \text{ where } a \ge 1 \text{ and } b > 1 \\\hline \\&\textbf{Case 1:} \\&\text{If } f(n) = O(n^{\log_b a - \epsilon}) \text{ for some constant } \epsilon > 0, \text{ then:} \\&T(n) = \Theta(n^{\log_b a}) \\\\&\textbf{Case 2:} \\&\text{If } f(n) = \Theta(n^{\log_b a} \log^k n) \text{ for some constant } k \ge 0, \text{ then:} \\&T(n) = \Theta(n^{\log_b a} \log^{k+1} n) \\\\&\textbf{Case 3:} \\&\text{If } f(n) = \Omega(n^{\log_b a + \epsilon}) \text{ for some constant } \epsilon > 0, \text{ and if } \\&a f(n/b) \le c f(n) \text{ for some constant } c < 1, \text{ then:} \\&T(n) = \Theta(f(n))\end{aligned}

There are the following three cases:

If f(n) = O(n^c) where c < Log_ba then T(n) = Θ(n^Log_b^a)
If f(n) = Θ(n^c) where c = Log_ba then T(n) = Θ(n^cLog n)
If f(n) = Ω(n^c) where c > Log_ba then T(n) = Θ(f(n))

The above recurrence relation accusers when an algorithm has the following type of recursive structure.

CSS

function f(input x of size n)
    if (n < k)
        solve x directly and return
    else
        divide x into a subproblems of size n/b
        recursively solve each subproblem
        combine the results

In this structure:

The problem is divided into a number of subproblems.
Each subproblem has size n/b
The work done to divide the problem and combine results is represented by f(n)

Hence, the running time can be expressed as:

T(n) = aT(n/b) + f(n)

When Master Theorem Cannot Be Applied

The Master Theorem does not apply in the following cases:

If T(n) is not monotonic
Example: T(n) = sin⁡(n)
If f(n) is not a polynomial or does not fit the required form
Example:T(n) = 2T(n/2) + 2ⁿ

Examples

Example 1: Binary Search

T(n) = T(n/2) + O(1)

a = 1, b = 2, k = 0, p = 0
a = bᵏ → Case 2(a)

T(n) = Θ(logn)

Example 2: Merge Sort

T(n) = 2 T(n/2) + O(n)

a = 2, b = 2, k = 1, p = 0
a = bᵏ → Case 2(a)

T(n) = Θ(n log ⁡n)

Example 3

T(n) = 3T(n/2) + n²

a = 3, b = 2, k = 2
a < bᵏ → Case 3(a)

T(n) = Θ(n²)

How does the theorem work?

The master method is mainly derived from the recurrence tree method. If we draw the recurrence tree of T(n) = aT(n/b) + f(n), we can see that the work done at the root is f(n), and work done at all leaves is O(n^c) where c is Log_ba. And the height of the recurrence tree is Log_bn

In the recurrence tree method, we calculate the total work done. If the work done at leaves is polynomially more, then leaves are the dominant part, and our result becomes the work done at leaves (Case 1). If work done at leaves and root is asymptotically the same, then our result becomes height multiplied by work done at any level (Case 2). If work done at the root is asymptotically more, then our result becomes work done at the root (Case 3).

Key Points

The Master Theorem is specifically designed for divide-and-conquer recurrences
It applies only to recurrences of the form: T(n) = aT(n/b) + f(n)
It provides a quick and reliable way to compute asymptotic time complexity
Not all recurrences can be solved using this method
For unsupported cases, we either use Substitution Method or Recurrence Tree Method

Advanced Version of Master Theorem

Not all recurrence relations can be solved with the use of the master theorem i.e. if

T(n) is not monotone, ex: T(n) = sin n
f(n) is not a polynomial, ex: T(n) = 2T(n/2) + 2ⁿ

The advanced version of the Master Theorem can handle recurrences with multiple terms and more complex functions.

This theorem is an advance version of master theorem that can be used to determine running time of divide and conquer algorithms if the recurrence is of the following form :-

where n = size of the problem
a = number of subproblems in the recursion and a >= 1
n/b = size of each subproblem
b > 1, k >= 0 and p is a real number.

Then,

if a > b^k, then T(n) = θ(n^log_b^a)
if a = b^k, then
(a) if p > -1, then T(n) = θ(n^log_b^a log^p+1n)
(b) if p = -1, then T(n) = θ(n^log_b^a loglogn)
(c) if p < -1, then T(n) = θ(n^log_b^a)
if a < b^k, then
(a) if p >= 0, then T(n) = θ(n^k log^pn)
(b) if p < 0, then T(n) = θ(n^k)

Example 1: T(n) = 3T(n/2) + log²n
a = 3, b = 2, k = 0, p = 2
b^k = 1. So, a > b^k [Case 1]
T(n) = θ(n^log_b^a )
T(n) = θ(n^log₂³)

Example 2: T(n) = 2T(n/2) + nlog²n
a = 2, b = 2, k = 1, p = 2
b^k = 2. So, a = b^k [Case 2.(a)]
T(n) = θ(n^log_b^alog^p+1n )
T(n) = θ(n^log₂²log³n)
T(n) = θ(nlog³n)

Example 3: T(n) = 2ⁿT(n/2) + nⁿ
This recurrence can't be solved using above method since function is not of form T(n) = aT(n/b) + θ(n^k log^pn)

Practice Questions