Lecture 5

Recursion

EECS 281: Data Structures & Algorithms

 Recursion Basics
• A recursive function calls itself to
accomplish its task
• At least one base case is required
• At least one recursive case is required
• Each subsequent call has simpler
(smaller) input
• Single recursion can be used on lists
• Multiple recursion can be used on trees
2
 Job Interview Question
• Implement this function
// returns x^n
int power(int x, uint32_t n);

• The obvious solution uses n - 1 multiplications

– 28 = 2*2* ... *2
• Less obvious: O(log n) multiplications
– Hint: 28 = ((22)2)2
– How does it work for 27 ?
• Write both solutions iteratively and recursively

3
 Ideas
• Obvious approach uses a subproblem "one step smaller"

1 𝑛 == 0
𝑥𝑛 = ቊ
𝑥 ∗ 𝑥 𝑛−1 𝑛>0

• Less obvious approach splits the problem into two halves

1 𝑛 == 0
𝑥 𝑛 = ቐ 𝑥 𝑛Τ2 ∗ 𝑥 𝑛Τ2 𝑛 > 0, even
𝑥 ∗ 𝑥 𝑛 Τ2 ∗ 𝑥 𝑛 Τ2 𝑛 > 0, odd

4
 Two Recursive Solutions
Solution #1 Solution #2
1 int power(int x, uint32_t n) { 1 int power(int x, uint32_t n) {
2 if (n == 0) 2 if (n == 0)
3 return 1; 3 return 1;
4 4
5 return x * power(x, n – 1); 5 int result = power(x, n / 2);
6 } // power() 6 result *= result;
7 if (n % 2 != 0) // n is odd
Recurrence: T(n) = T(n-1) + c
8 result *= x;
Complexity: Θ(n)
9
10 return result;
11 } // power()
Recurrence: T(n) = T(n / 2) + c
Complexity: Θ(log n) 5
 Tail Recursion
• When a function is called, it gets a stack frame, which
stores the local variables
• A simply recursive function generates a stack frame for
each recursive call
• A function is tail recursive if there is
no pending computation at each
recursive step
– "Reuse" the stack frame rather
than create a new one
• Tail recursion and iteration are
equivalent

http://xkcd.com/1270/ 6
 Recursion and the Stack
1 int factorial(int n) {
2 if (n == 0)
3 return 1;
4 return n * factorial(n - 1);
5 } // factorial()
6
7 int main(int, char *[]) {
8 factorial(5);
9 } // main()

7
 The Program Stack (1)
• When a function call is made
1a. All local variables are saved in a special storage
called the program stack
2a. Then argument values are
pushed onto the program stack
• When a function call is received
2b. Function arguments are popped off the stack
• When return is issued within a function
3a. The return value is pushed onto the program stack
• When return is received at the call site
3b. The return value is popped off the the program stack
1b. Saved local variables are restored
8
 The Program Stack (2)
• Program stack supports nested function calls
– Six nested calls = six sets of local variables

• There is only one program stack (per thread)

– NOT the program heap, (where dynamic memory is allocated)

• Program stack size is limited

– The number of nested function calls is limited

• Example: a bottomless (buggy) recursion function will

exhaust program stack very quickly
9
 Recursion vs. Tail Recursion
1 int factorial(int n) {
2 if (n == 0) Recursive
3 return 1; Θ(n) time complexity
Θ(n) space complexity
4 return n * factorial(n - 1);
(uses n stack frames)
5 } // factorial()

6 int factorial(int n, int res = 1) {

Tail recursive*
7 if (n == 0)
Θ(n) time complexity
8 return res; Θ(1) space complexity
9 return factorial(n - 1, res * n); (reuses 1 stack frame)
10 } // factorial()
* The default argument is used to seed the res parameter. Alternatively, the
"helper function" pattern could be used.
10
Logarithmic Tail Recursive power()
1 int power(int x, uint32_t n, int result = 1) {
2 if (n == 0)
3 return result;
4 else if (n % 2 == 0) // even
5 return power(x * x, n / 2, result);
6 else // odd
7 return power(x * x, n / 2, result * x);
8 } // power()

Θ(log n) time complexity

Θ(1) space complexity

11
 Practical Considerations
• Program stack is limited in size
– It's actually pretty easy to exhaust this!
e.g. Computing the length of a very long vector using a
"linear recursive" function with Θ(n) space complexity

• For a large data set

– ”Simple" recursion is a bad idea
– Use tail recursion or iterative algorithms instead

• This doesn't mean everything should be tail recursive

– Some problems can't be solved in Θ(1) space!
12
 Recurrence Relations
• A recurrence relation describes the way a problem
depends on a subproblem.
– A recurrence can be written for a computation:

𝑛 1 𝑛 == 0
𝑥 =ቊ
𝑥 ∗ 𝑥 𝑛−1 𝑛>0
– A recurrence can be written for the time taken:
𝑐0 𝑛 == 0
𝑇 𝑛 = ቊ𝑇 𝑛 − 1 + 𝑐
1 𝑛>0
– A recurrence can be written for the amount of memory used*:
𝑐0 𝑛 == 0
𝑀 𝑛 = ቊ𝑀 𝑛 − 1 + 𝑐
1 𝑛>0
*Non-tail recursive

13
A Logarithmic Recurrence Relation
𝑐0 𝑛 == 0
𝑇 𝑛 = ቐ𝑇 𝑛 + 𝑐
1
 Θ(log 𝑛)
2 𝑛>0

• Fits the logarithmic recursive implementation of

power()
– The power to be calculated is divided into two halves
and combined with a single multiplication
• Also fits Binary Search
– The search space is cut in half each time, and the
function recurses into only one half

14
 Common Recurrences
Recurrence Example Big-O
Solution
T(n) = T(n / 2) + c Binary Search O(log n)
T(n) = T(n - 1) + c Linear Search O(n)
T(n) = 2T(n / 2) + c Tree Traversal O(n)
T(n) = T(n - 1) + c1 * n + c2 Selection/etc. Sorts O(n2)
T(n) = 2T(n / 2) + c1 * n + c2 Merge/Quick Sorts O(n log n)

15
 Solving Recurrences
• Substitution method
1. Write out T(n), T(n - 1), T(n - 2)
2. Substitute T(n - 1), T(n - 2) into T(n)
3. Look for a pattern
4. Use a summation formula
• Another way to solve recurrence equations
is the Master Method (AKA Master
Theorem)
• Recursion tree method
16
 Recurrence Thought Exercises
• What if a recurrence cuts a problem into two
subproblems, and both subproblems were
recursively processed?
• What if a recurrence cuts a problem into three
subproblems and…
– Processes one piece
– Processes two pieces
– Processes three pieces
• What if there was additional, non-constant work
after the recursion?
17
 Master Theorem
Let T(n) be a monotonically increasing function
that satisfies:
𝑛
𝑇 𝑛 = 𝑎𝑇 + 𝑓(𝑛)
𝑏
𝑇 1 = 𝑐0 𝑜𝑟 𝑇 0 = 𝑐0

where a ≥ 1, b > 1. If 𝑓 𝑛 ∈ Θ 𝑛𝑐 , then:

Θ 𝑛log𝑏 𝑎 𝑖𝑓 𝑎 > 𝑏 𝑐
𝑇 𝑛 ∈ ൞Θ 𝑛𝑐 log 𝑛 𝑖𝑓 𝑎 = 𝑏 𝑐
Θ 𝑛𝑐 𝑖𝑓 𝑎 < 𝑏 𝑐
18
 Exercise 1
𝑛 3
𝑇 𝑛 = 3𝑇 + 𝑛+1
2 4
What are the parameters?
a=3
b=2
c=1
Which condition?
𝑛
Θ 𝑛log𝑏 𝑎 𝑖𝑓 𝑎 > 𝑏𝑐 𝑇 𝑛 = 𝑎𝑇 + 𝑓(𝑛)
𝑇 𝑛 ∈ ൞Θ 𝑛𝑐 log 𝑛 𝑏
𝑖𝑓 𝑎 = 𝑏 𝑐
𝑓 𝑛 ∈ Θ 𝑛𝑐
Θ 𝑛𝑐 𝑖𝑓 𝑎 < 𝑏 𝑐
19
 Exercise 2
𝑛
𝑇 𝑛 = 2𝑇 + 𝑛+7
4
What are the parameters?
a=2
b=4
c = 1/2
Which condition?
𝑛
Θ 𝑛log𝑏 𝑎 𝑖𝑓 𝑎 > 𝑏𝑐 𝑇 𝑛 = 𝑎𝑇 + 𝑓(𝑛)
𝑇 𝑛 ∈ ൞Θ 𝑛𝑐 log 𝑛 𝑏
𝑖𝑓 𝑎 = 𝑏 𝑐
𝑓 𝑛 ∈ Θ 𝑛𝑐
Θ 𝑛𝑐 𝑖𝑓 𝑎 < 𝑏 𝑐
21
 Exercise 3
𝑛 1 2
𝑇 𝑛 =𝑇 + 𝑛 +𝑛
2 2
What are the parameters?
a=1
b=2
c=2
Which condition?
𝑛
Θ 𝑛log𝑏 𝑎 𝑖𝑓 𝑎 > 𝑏𝑐 𝑇 𝑛 = 𝑎𝑇 + 𝑓(𝑛)
𝑇 𝑛 ∈ ൞Θ 𝑛𝑐 log 𝑛 𝑏
𝑖𝑓 𝑎 = 𝑏 𝑐
𝑓 𝑛 ∈ Θ 𝑛𝑐
Θ 𝑛𝑐 𝑖𝑓 𝑎 < 𝑏 𝑐
23
 When Not to Use
• You cannot use the Master Theorem if:
– T(n) is not monotonic, such as T(n) = sin(n)
– f(n) is not a polynomial, i.e. f(n) = 2n
– b cannot be expressed as a constant. i.e.

𝑇 𝑛 =𝑇 sin 𝑛
• There is also a special fourth condition if
f(n) is not a polynomial; see later in slides
25
 When Not to Use
The recursion does not involve division:
𝑇 𝑛 =𝑇 𝑛−1 +𝑛

Master Theorem not applicable

𝑛
𝑇 𝑛 ≠ 𝑎𝑇 + 𝑓(𝑛)
𝑏

26
 Fourth Condition
• There is a 4th condition that allows
polylogarithmic functions

If f (n)  (n log b a

log n) for some k  0,
k

k 1
Then T (n)  (n log b a
log n)

• This condition is fairly limited,

• No need to memorize/write down

27
 Fourth Condition Example
• Given the following recurrence:
𝑛
𝑇 𝑛 = 2𝑇 + 𝑛 log 𝑛
2
• Clearly a=2, b=2, but f(n) is not polynomial
• However: 𝑓 𝑛 ∈ Θ 𝑛 log 𝑛 and 𝑘 = 1
𝑇 𝑛 = Θ 𝑛 log 2 𝑛

If 𝑓(𝑛) ∈ Θ 𝑛log𝑏 𝑎 log 𝑘 𝑛 for some 𝑘 ≥ 0,

Then 𝑇(𝑛) ∈ Θ 𝑛log𝑏 𝑎 log 𝑘+1 𝑛
28
 Bonus: Binary Max?
• A friend of yours claims to have discovered a (revolutionary!) new
algorithm for finding the maximum element in an unsorted array:
1. Split the array into two halves.
2. Recursively find the maximum in each half.
3. Whichever half-max is bigger is the overall max!

• Your friend says this algorithm leverages the power of "binary

partitioning" to achieve better than linear time.
– This sounds too good to be true. Give an intuitive argument why.
– Use the master theorem to formally prove this algorithm is Θ(n).

29
 Job Interview Question
Write an efficient algorithm that searches for
a value in an n x m table (two-dim array).
This table is sorted along
the rows and columns — that is,
1 4 7 11 15
table[i][j] ≤ table[i][j + 1],
table[i][j] ≤ table[i + 1][j] 2 5 8 12 19

• Obvious ideas: linear or 3 6 9 16 22

binary search in every row 10 13 14 17 24

– nm or n log m … too slow 18 21 23 26 30

30
 Potential Solution #1: Quad Partition

• Split the region into four quadrants, eliminate one.

• Then, recursively process the other 3 quadrants.
• Write a recurrence relation for 1 4 7 11 15
the time complexity in terms of
n, the length of one side: 2 5 8 12 19

3 6 9 16 22
T(n) = 3T(n/2) + c
10 13 14 17 24
Why n/2 and not n/4 if it's a quadrant?
Remember, n is the length of one side! 18 21 23 26 30

n 31
 Potential Solution #2: Binary Partition
How can we improve this?

• Split the region into four quadrants.

Use binary search!
• Scan down the middle column, until
you find "where the value should be" 1 4 7 11 15
if it were in that column.1
• This allows you to eliminate two 2 5 8 12 19
quadrants. (Why? Which ones?)
3 6 9 16 22
• Recursively process the other two.
• Write a recurrence relation, again 10 13 14 17 24
in terms of the side length n:
18 21 23 26 30

T(n) = 2T(n/2) + cn
n 32
1 Of course, you might get lucky and find the value here!
 Potential Solution #3: Improved Binary Partition

• Split the region into four quadrants.

• Scan down the middle column, until
you find "where the value should be" 1 4 7 11 15
if it were in that column.1
• This allows you to eliminate two 2 5 8 12 19
quadrants. (Why? Which ones?)
3 6 9 16 22
• Recursively process the other two.
• Write a recurrence relation, again 10 13 14 17 24
in terms of the side length n:
18 21 23 26 30

T(n) = 2T(n/2) + c log n

n 33
1 Of course, you might get lucky and find the value here!
 Exercise: Use the Master Theorem
• Use the master theorem to find the complexity of each approach:
• Quad Partition:

T(n) = 3T(n/2) + c
T(n) = (nlog23) ≈ (n1.58)
• Binary Partition:

T(n) = 2T(n/2) + cn
T(n) = (n log n)
• Improved Binary Partition:
T(n) = 2T(n/2) + c log n
T(n) = (n)
34
 Another Solution!
Stepwise Linear Search
1 bool stepWise(int mat[][n], int m, int target, int &row, int &col) {
2 if (target < mat[0][0] || target > mat[m - 1][n - 1])
3 return false;
4 row = 0; col = n - 1; 1 4 7 11 15
5 while (row <= m - 1 && col >= 0) {
6 if (mat[row][col] < target) 2 5 8 12 19
7 ++row;
8 else if (mat[row][col] > target) m 3 6 9 16 22
9 --col;
10 13 14 17 24
10 else
11 return true;
18 21 23 26 30
12 } // while
13 return false;
14 } // stepWise()
n 35
 Runtime Comparisons
• Source code and data (M = N = 100) available at
http://www.leetcode.com/2010/10/searching-2d-sorted-matrix.html
http://www.leetcode.com/2010/10/searching-2d-sorted-matrix-part-ii.html
http://www.leetcode.com/2010/10/searching-2d-sorted-matrix-part-iii.html
• Runtime for 1,000,000 searches
Algorithm Runtime
Binary search 31.62s
Diagonal Binary Search 32.46s
Step-wise Linear Search 10.71s
Quad Partition 17.33s
Binary Partition 10.93s
Improved Binary Partition 6.56s
36
 Questions for Self-Study
• Consider a recursive function that only calls itself.
Explain how one can replace recursion
by a loop and an additional stack.
• Go over the Master Theorem in the CLRS textbook
• Which cases of the Master Theorem were exercised for
different solutions in the 2D-sorted-matrix problem?
• Solve the same recurrences by
substitution w/o the Master Theorem
• Write (and test) programs for each
solution idea, time them on your data

37