Data Structures and Algorithms
(ESO207A)
Lecture 18:
• Solving recurrences
that occur frequently in
… the analysis of algorithms.
1
Homework from the last class
• Suitably modify the Merge procedure to compute .
2
Relook
Merging A[] and A[]
Sorted Sorted
A x
𝒊 >x
𝒋 𝒌
3
Pesudo-code for Merging two sorted arrays
Merge(A,,C)
; ; 0;
While( ≤ and )
{ If(A[]< A[]) { C[] A[]; ++; ++ }
Else { C[] A[];++; ++ }
}
While( ≤ ) { C[] A[]; ++; ++ }
While( ≤ ) { C[] A[]; ++; ++ }
return C ;
We shall make just a slight change in the above
pseudo-code to achieve our main objective of
computing . If you understood the animation shown
in the previous slide, can you guess it now ? 4
Pesudo-code for
Merging and counting inversions
Merge_and_CountInversion(A,,C)
; ; 0;
0;
While( ≤ and )
{ If(A[]< A[]) { C[] A[]; ++; ++ }
Else { C[] A[];++; ++
? + ? ; ();
}
}
While( ≤ ) { C[] A[]; ++; ++ }
Nothing extra is
While( ≤ ) { C[] A[]; ++; ++ } needed here.
return ;
5
Counting Inversions
Final algorithm based on divide & conquer
Sort_and_CountInversion(A)
{ If () return 0;
else
{ ()/2;
Sort_and_CountInversion (A,);
T()
Sort_and_CountInversion (A,);
Create a temporary array C[ ]
Merge_and_CountInversion(A,,C);
Copy C[] to A[]; O()
return + + ;
}
}
6
Counting Inversions
Final algorithm based on divide & conquer
Time complexity analysis:
If = 1,
T() = c for some constant c
If > 1,
T() = c + 2 T(/2)
= O( log )
Theorem: There is a divide and conquer based algorithm
for computing the number of inversions in an array of size .
The running time of the algorithm is O( log ).
7
Commonly occurring recurrences
T()
T()
T()=T()
T()
=T() +=+=T(/2)
==T() +T()
+=24T()
+3T()
+T()
2T()
T()+ T()
8
Methods for solving Recurrences
commonly occuring in
algorithm analysis
9
Methods for solving common Recurrences
• Unfolding the recurrence.
• Guessing the solution and then proving by induction.
• A General solution for a large class of recurrences (Master theorem)
10
Solving a recurrence by unfolding
Let T(1) = 1,
T() = + 4 T(/2) for >1, where is some positive constant
Solving the recurrence for T() by unfolding (expanding)
T() = + T(/)
= + + T(/)
= + + T(/)
= +2 +
= +2 +
A geometric increasing series with log terms and common ratio 2
= O()
11
Solving a recurrence by unfolding
Let T(1) = 1,
T() = + 4 T(/2) for >1, where is some positive constant
Solving the recurrence for T() by unfolding (expanding)
T() = + T(/)
= + + T(/)
= + + T(/)
= +2 +
= +2 +
= O()
12
Solving a recurrence by guessing and
then proving by induction
T(1) = It looks similar/identical to the
T() = 2T(/2) + recurrence of merge sort. So
we guess
Guess: T() ≤ + for some constants and . T() = O( log )
Proof by induction:
Base case: holds true if ≥
Induction hypothesis: T() ≤ + for all <
To prove: T() ≤ +
These inequalities
Proof: T() = 2T(/2) + can be satisfied
≤ 2( + ) + // by induction hypothesis simultaneously
by selecting
= - +2 +
=
= + +(+ - ) =+
≤ + if ≥ +
Hence T() ≤ ( + for all value of .
13
So T() = )
Solving a recurrence by guessing and
then proving by induction
Key points:
• You have to make a right guess (past experience may help)
• What if your guess is too loose ?
• Be careful in the induction step.
14
A General Method for solving a large class of
Recurrences
15
Solving a large class of recurrences
T(1) = 1,
T() = f() + T(/)
Where
• and are constants and >1
• f() is a multiplicative function:
f() = f()f()
AIM : To solve T() for =
16
Warm-up
f() is a multiplicative function:
f() = f()f()
f() = ? 1
f() = ? f ( 𝑎) 𝒊
f() = ? 1/f (𝑛)
Example of a multiplicative function : f() =
Question: Can you express as power of ?
Answer:
17
Solving a slightly general class of recurrences
T() = f() + T(/)
= f() + f(/) + T(/)
= f() + f(/) + f(/) + T(/)
= …
= f() + f(/) + … + f(/) + … + f(/) +T(1)
=() +
… after rearranging …
= +
… continued to the next page …
18
T() = +
(since is multiplicative)
= +
= +
= +
= +
= + A geometric series
Case 1: = , T() = ??
𝑎𝑘 (𝑘+1) ¿ O( 𝒂 𝐥𝐨𝐠 𝒏 log𝑏 𝑛) ¿O( 𝑛𝐥𝐨𝐠 𝒂 log 𝑏 𝑛)
𝒃 𝒃
19
T() = +
(since is multiplicative)
= +
= +
= +
= + For < , the sum of this
series is bounded by
= ?O(1)
= +
Case 2: < , T() = ??
+O(1) = O) = O(f(𝑛))
20
T() = +
(since is multiplicative)
= +
= +
= + For > , the sum of this
= + series is equal to
= +
Case 3: > , T() = ??
+ =)
21
Three cases
T() = +
Case 1: = ,
T() = )
Case 2: < ,
T() =
Case 3: > ,
T() = )
22
Master theorem
T(1) = 1,
T() = f() + a T(/b) where f is multiplicative.
There are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )
23
Examples
24
Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )
Example 1: T()= + 4 T(/2)
This is case 3
Solution: T()= ??
O( 𝑛2 )
25
Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )
Example 2: T()= + 4 T(/2)
This is case 1
Solution: T()= ?? O (𝑛𝟐 log 𝑛)
2
26
Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )
Example 3: T()= + 4 T(/2)
This is case 2
Solution: T()= ??
O ( 𝑛3 )
27
Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )
Example 4: T()= 2 + 3 T(/2)
Solution: T()= ?? We can not apply master theorem directly
since f() = 2 is not multiplicative.
28
Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )
Example 4: T()= 2 + 3 T(/2)
Solution: G()= T()/2 This is case 3
G()= + 3 G(/2)
G(n) = 1.58
T(n) = .
= O (𝑛 )
29
Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )
Example 4: T()= 2 + 3 T(/2)
Solution: G()= T()/2
G()= + 3 G(/2)
G(n) = 1.58
T(n) = .
= O (𝑛 )
30
Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )
Example 6: T() = T() + c
Solution: T()= ?? We can not apply master theorem directly
since T() <> T(/) for any constant .
31
Solving T() = T() + c using the method of
unfolding
T() = c + T()
Can you guess the
= c + c+ T() number of terms in
= c + c+ c + T() this series ?
= c + c+ … c + … + T(1)
A series which is decreasing at a rate
⌈loglog𝑛⌉
faster than any geometric series
= O()
32
Master theorem
If T(1) = 1, and T() = f() + a T(/b) where f is multiplicative, then
there are the following solutions
Case 1: = , T() =
Case 2: < , T() =
Case 3: > , T() = )
Example 5: T()= + 2 T(/2)
Solution: T()= ?? We Using theapply
can not method of “unfolding”,
master theorem since
it can
f() be
= isshown that T() = .
not multiplicative.
33
Homework # 1
Solve the following recurrences systematically (if possible by various
methods). Assume that T() =1 for all these recurrences.
• T() = 1 + 2 T(/2)
• T() = + 2 T(/2)
• T() = + 7 T(/3)
• T() = + 2T(/2)
• T() = 1 + T(/5)
• T() = + 2 T(/4)
• T() = 1 + T()
• T() = + T(/10)
• T() = + T(/4)
34
Homework # 2
Find error in the following reasoning.
For the recurrence T(1) = , and T() = 2T(/2) + ,
one guesses T() = O()
Proposed (wrong)proof by induction:
Induction hypothesis: T() ≤ for all
T() = 2T(/2) +
≤ 2() + // by induction hypothesis
= +
= O()
35
Proof of correctness
Algorithm for majority element
36
The Algorithm at the end of th iteration
𝒊 −𝟏
𝒙
count
Question: What assertion holds at the end of th iteration ?
Answer:
P() : isisaamajority
majority
is a majority elementelement
element
of of {,…count times…,,A[],…,A[]}
of{A[],A[],…,A[]}
{A[],A[],…,A[]}
… ?…
Question: What is P() ?
Answer: is a majority element of {,…count times…,}
37
We need count > 0 at the end of the while loop for drawing this inference.
The Algorithm at the end of th iteration
𝒊 −𝟏
𝒙
count
Question: What assertion holds at the end of th iteration ?
Answer:
P() : After cancelling pairs of distinct element, {,…count times…,} remains.
So we have multiple assertions that hold.
38
The Algorithm at the end of th iteration
𝒊 −𝟏
𝒙
count
Question: What assertion holds at the end of th iteration ?
Answer:
1. If A[],…,A[] have a majority element, it must be .
2. is the majority element of {,…count times…,,A[],…,A[]}
3. After cancelling pairs of distinct elements from A[],…,A[],
the multiset {,…count times…,} remains.
39
Conclusion
Which of these assertions will suffice as proof of correctness ?
Do we need any other lemma ?
What about the following lemma ?
Lemma: If there is a majority element, say , after any sequence of
cancelling pairs of distinct elements from A, a nonzero copies of will always survive.
Do you use this lemma to establish validity of one of the assertions ?
[You surely do not require it at least for one assertion].
Can you use this lemma and one of the assertions to prove correctness of algorithm ?
Is there an alternate way to prove the correctness of the algorithm without P() ?
Ponder over these questions, without fea,r without aversion 40
Proof of correctness of an algorithm is a delicate and nontrivial concept.
Take your time to internalize it. Do not worry or feel anxious.
[No question will be asked in the mid sem exam on the proof of correctness.]
But you can not avoid it
It does not require any formula.
Just formalizing the right understanding of the algorithm.
Best wishes
41
