1

Solving Recurrences
T(n) = k n=1
T(n) = 4-T(n/2) + k-n n>1
T(n) = k n=1
T(n) = 3-T(n/2) + k-n n>1
How?
In general, hard. Solutions not always known.
Will discuss techniques in a few minutes
Real goal: Find closed form of defined functions.
) )
2
n n T = ) )
3 log
2
n n T' =
2
Techniques for Solving Recurrences
Well use three techniques:
Substitution
Recursion Tree
Master method
3
Techniques: Substitution
Guess a solution & check it.
More details:
1. Guess a solution f(n) and use the recurrence to show that
T(n) e f(n). Sometimes we guess only the form of the solution,
using unknown constants.
2. Use induction to find the constants & verify the solution.
Completely dependent on making reasonable guesses.
4
Substitution Example 1
Inductive case, n>1:
T(n) = 4-T(n/2) + n Definition.
e 4-c-(n/2)
3
+ n Induction.
= c/2 - n
3
+ n Algebra.
Assume T(k) e c-k
3
, for Vk<n. Show T(n) e c-n
3
.
While this is O(n
3
), were not done.
Need to show c/2 - n
3
+ n e c-n
3
.
Fortunately, the constant factor is shrinking, not growing.
T(n) = 1 n=1
T(n) = 4-T(n/2) + n n>1
Guess:
T(n) e c-n
3
for Vn>n
0
5
Substitution Example 1
Inductive case, n>1:
T(n) e c/2 - n
3
+ n From before.
= c-n
3
- (c/2 - n
3
- n) Algebra.
e c-n
3
For n>0, if c>2.
T(n) = 1 n=1
T(n) = 4-T(n/2) + n n>1
Guess:
T(n) e c-n
3
for Vn>n
0
Assume T(k) e c-k
3
, for Vk<n. Show T(n) e c-n
3
.
6
Substitution Example 2
T(n) = 1 n=1
T(n) = 4-T(n/2) + n n>1
Guess: T(n) = O(n
2
).
Same recurrence, but now try tighter bound.
More specifically:
T(n) e c-n
2
for Vn>n
0
.
7
Substitution Example 2
T(n) = 4-T(n/2) + n
e 4-c-(n/2)
2
+ n
= c-n
2
+ n
T(n) = 1 n=1
T(n) = 4-T(n/2) + n n>1
Guess:
T(n) e c-n
2
for Vn>n
0
Assume T(k) e c-k
2
, for Vk<n. Show T(n) e c-n
2
.
Follow same steps, and we get...
Not e c-n
2
!
Problem is that the constant isnt shrinking.
8
Substitution Example 2
T(n) = 1 n=1
T(n) = 4-T(n/2) + n n>1
Guess:
T(n) e c-n
2
- d-n for Vn>n
0
Assume T(k) e c-k
2
- d-k, for Vk<n. Show T(n) e c-n
2
- d-n.
Solution: Use a tighter guess & inductive hypothesis.
Subtract a lower-order term a common technique.
9
Substitution Example 2
Base case, n=1:
T(n) = 1 Definition.
1 e c-d Choosing c, d appropriately.
T(n) = 1 n=1
T(n) = 4-T(n/2) + n n>1
Guess:
T(n) e c-n
2
- d-n for Vn>n
0
Assume T(k) e c-k
2
- d-n, for Vk<n. Show T(n) e c-n
2
- d-n.
10
Substitution Example 2
Inductive case, n>1:
T(n) = 4-T(n/2) + n Definition.
e 4-(c-(n/2)
2
- d-(n/2)) + n Induction.
= c-n
2
- 2-d-n + n Algebra.
= c-n
2
- d-n - (d-n - n) Algebra.
e c-n
2
- d-n Choosing d>1.
T(n) = 1 n=1
T(n) = 4-T(n/2) + n n>1
Guess:
T(n) e c-n
2
- d-n for Vn>n
0
Assume T(k) e c-k
2
- d-n, for Vk<n. Show T(n) e c-n
2
- d-n.
11
Substitution Example 2
T(n) = 1 n=1
T(n) = 4-T(n/2) + n n>1
Proved:
T(n) e 2-n
2
1-n for Vn>0
Thus, T(n) = O(n
2
).
12
Techniques: Substitution
Ability to guess effectively comes with experience.
Examples used O(), guessing T() e
Can use ;(), guessing T() >
13
Techniques: Recursion Tree
1. Unroll the recurrence to obtain a summation.
2. Solve or estimate summation.
3. Use solution as a guess in substitution.
Math can be tricky.
14
Recursion Tree Example 1
T(n) = 1 n=1
T(n) = 4-T(n/2) + n n>1
n/2 n/2 n/2 n/2 2n
n/4 4n n/4 n/4 n/4 n/4 n/4 n/4 n/4


1
1 4
#levels
How many levels?
?
?
Cost at
this level
T(?)
In this example, all terms on a level are the same.
Common, but not always true.
n n
log
2
n
Now, turn picture
into a summation
15
Recursion Tree Example 1
T(n) = 1 n=1
T(n) = 4-T(n/2) + n n>1
n
Cost at
this level
n
n/2
T(?)
n/2 n/2 n/2 2n
n/4 4n n/4 n/4 n/4 n/4 n/4 n/4 n/4


1
1 4
lg n
T(n) = n + 2n + 4n + + 2
lg n 1
n + 4
lg n
= n(1 + 2 + 4 + + 2
lg n 1
) + n
lg 4
)
2
1 n 0..lg i
i
n 2 n T(n) +

'
+

'

=

=
)
2
1 n lg
n
1 2
2
n +

'
+

'

)
2
n lg
n
2
2
n +

'
+

'

=
)
2
2 lg
n
2
n
n +

'
+

'

=
)
2
n
2
n
n +

'
+

'

=
= O(n
2
)
16
Recursion Tree Example 2
T(n) = 1 n=1
T(n) = T(n/3) + T(2n/3) + n n>1
Cost at
this level
T(?)
n n
n/3 2n/3 n
How many levels?
?
?
log
3/2
n
But, not all branches
have same depth!
Makes cost near the
leaves hard to
calculate.
Estimate!

n/9 2n/9 2n/9 4n/9
n

17
Recursion Tree Example 2
T(n) = 1 n=1
T(n) = T(n/3) + T(2n/3) + n n>1
Cost at
this level
T(?)
n n
n/3 2n/3 n

n/9 2n/9 2n/9 4n/9
n
#levels = log
3/2
n
Overestimate.
Consider all branches to
be of max depth.
T(n) e n (log
3/2
n) + n
T(n) = O(n log n)

n
1 1

18
Recursion Tree Example 2
T(n) = 1 n=1
T(n) = T(n/3) + T(2n/3) + n n>1
Cost at
this level
T(?)
n n
n/3 2n/3 n

n/9 2n/9 2n/9 4n/9
n
#levels = log
3/2
n
Underestimate.
Count the log
3
n complete
levels, & ignore the rest.
T(n) > n log
3
n
T(n) = ;(n log n)
Thus, T(n) = O(n log n)

19
Techniques: Master Method
Cookbook solution for some recurrences of the form
T(n) = a - T(n/b) + f(n)
where
a>1, b>1, f(n) asymptotically positive
First describe its cases, then outline proof.
20
Master Method Case 1
T(n) = a - T(n/b) + f(n)
f(n) = O(n
log
b
a - s
) for some s>0 T(n) = O(n
log
b
a
)
T(n) = O(n
lg 7
)
c-n
2
=
?
O(n
log
b
a - s
) = O(n
log
2
7 - s
) < O(n
2.8 - s
)
Yes, for any s e 0.8.
T(n) = 7-T(n/2) + c-n
2
a=7, b=2
E.g., Strassen matrix multiplication.
21
Master Method Case 2
T(n) = a - T(n/b) + f(n)
f(n) = O(n
log
b
a
) T(n) = O(n
log
b
a
lg n)
T(n) = 2-T(n/2) + c-n a=2, b=2
E.g., mergesort.
c-n =
?
O(n
log
b
a
) = O(n
log
2
2
) = O(n)
Yes.
T(n) = O(n lg n)
22
Master Method Case 3
T(n) = a - T(n/b) + f(n)
f(n) = ;(n
log
b
a + s
) for some s>0 and
a-f(n/b) e c-f(n) for some c<1 and all large enough n
T(n) = O(f(n))
T(n) = 4-T(n/2) + n
3
a=4, b=2
n
3
=
?
;(n
log
b
a + s
) = ;(n
log
2
4 + s
) = ;(n
2 + s
)
Yes, for any s e 1.
4-(n/2)
3
= -n
3
e
?
c-n
3
Yes, for any c > .
I.e., is the
constant factor
shrinking?
T(n) = O(n
3
)
23
Master Method Case 4
T(n) = a - T(n/b) + f(n)
None of previous apply. Master method doesnt help.
T(n) = 4-T(n/2) + n
2
/lg n a=4, b=2
Case 1?
n
2
/lg n =
?
O(n
log
b
a - s
) = O(n
log
2
4 - s
) = O(n
2 - s
) = O(n
2
/n
s
)
No, since lg n is asymptotically less than n
s
.
Thus, n
2
/lg n is asymptotically greater than n
2
/n
s
.
24
Master Method Case 4
T(n) = a - T(n/b) + f(n)
None of previous apply. Master method doesnt help.
T(n) = 4-T(n/2) + n
2
/lg n a=4, b=2
Case 2?
n
2
/lg n =
?
O(n
log
b
a
) = O(n
log
2
4
) = O(n
2
)
No.
25
Master Method Case 4
T(n) = a - T(n/b) + f(n)
None of previous apply. Master method doesnt help.
T(n) = 4-T(n/2) + n
2
/lg n a=4, b=2
Case 3?
n
2
/lg n =
?
;(n
log
b
a + s
) = ;(n
log
2
4 + s
) = ;(n
2 + s
)
No, since 1/lg n is asymptotically less than n
s
.