Design and Analysis of Algorithms

Design and Analysis of Algorithms

Unit-1

Dharmendra Kumar(Associate Professor(UCER))

 Definition of Algorithm

An algorithm is any well-defined computational

procedure that takes some value, or set of values, as input
and produces some value, or set of values, as output.

An algorithm is thus a sequence of computational

steps that transform the input into the output.

An algorithm is a set of steps to accomplish or

complete a task that is described precisely enough
that a computer can run it.
 Characteristics of an algorithm

 Input: Algorithm must contain 0 or more input values.

 Output: Algorithm must contain 1 or more input
values.
Finiteness: Algorithm must complete after finite
number of steps.
Definiteness: Each instruction of the algorithm must
be defined precisely or clearly.
Effectiveness: Every instruction must be basic i.e.
simple instruction.
 Design Strategy of Algorithm

 Divide and Conquer

 Dynamic Programming
 Branch and Bound
 Greedy Approach
 Backtracking
 Randomized Algorithm
 Analysis of algorithms

Analysis of algorithms is the determination of the

amount of time and space resources required to execute it.

Time complexity: The amount of time required by an

algorithm is called time complexity.

Space complexity: The amount of space/memory

required by an algorithm is called space complexity.
 Analysis of algorithms

The main concern of analysis of algorithms is the

required time or performance. Generally, we perform the
following types of analysis −

Worst case − The maximum number of steps taken on

any instance of size n.
Best case − The minimum number of steps taken on any
instance of size n.
Average case − An average number of steps taken on any
instance of size n.
 Running time of an algorithm

 Running time of an algorithm is the time taken by the

algorithm to execute it successfully.
 The running time of an algorithm on a particular input
is the number of primitive operations or "steps"
executed.
 It is convenient to define the notion of step so that it is
as machine independent as possible.
 It is denoted by T(n), where n is the input size.
 Pseudo code conventions

We use the following conventions in our pseudo code.

 Indentation is used to indicate a block.
 The looping constructs while, for, and repeat-until
and the if-else conditional construct have
interpretations similar to those in C.
 The symbol ―//‖ indicates that the remainder of the
line is a comment.
 A multiple assignment of the form ij  e assigns to
both variables i and j the value of expression e; it
should be treated as equivalent to the assignment j  e
followed by the assignment i  j .
 Pseudo code conventions

 We access array elements by specifying the array name

followed by the index in square brackets. For example,
A[i] indicates the ith element of the array A.
 The notation “..” is used to indicate a range of values
within an array. Thus, A[1..j] indicates the sub-array of
A consisting of the j elements A[1], A[2],…………,A[j].
 The Boolean operators “and” and “or” are used.
 A return statement immediately transfers control back
to the point of call in the calling procedure.
 Insertion Sort

Ex. Sort the following elements by insertion sort:- 4, 3, 2,

10, 12, 1, 5, 6.
 Insertion Sort Algorithm

Insertion_Sort(A)
1. n length[A]
2. for j2 to n
1. aA[j]
2. //Insert A[j] into the sorted sequence A[1 .. j-1].
3. i j-1
4. While i > 0 and a < A[i]
1. A[i+1]A[i]
2. i i-1
5. A[i+1]  a
 Insertion Sort Algorithm Analysis
Insertion_Sort(A)
Instruction cost times
n length[A] c1 1
for j2 to n c2 n
aA[j] c3 n-1
//Insert A[j] into the sorted 0 n-1
sequence A[1 .. j-1].
i j-1 c4 n-1
𝑛
While i > 0 and a < A[i] c5 𝑗<2 𝑡𝑗
𝑛
A[i+1]A[i] c6 𝑗<2(𝑡𝑗-1)
𝑛
i i-1 c7 𝑗<2(𝑡𝑗-1)
A[i+1]  a c8 n-1
 Time complexity of Insertion sort

T(n) = c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5. 𝑛𝑗<2 𝑡𝑗

+ c6. 𝑛𝑗<2(𝑡𝑗-1) + c7. 𝑛𝑗<2(𝑡𝑗-1) + c8.(n-1) ………..(1)

Here, there will be three case.

(1) Best case
(2) Worst case
(3) Average case
 Time complexity of Insertion sort

Best case: This case will be occurred when data is already

sorted.
In this case, value of tj will be 1. Put the value of tj in
equation (1) and find T(n). Therefore,
T(n) = c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5. 𝑛𝑗<2 1
+ c6. 𝑛𝑗<2(1-1) + c7. 𝑛𝑗<2(1-1) + c8.(n-1)
= (c2+c3+c4+c5+c8).n + (c1- c3 - c4 - c5 - c8)
= an + b
Clearly T(n) is in linear form, therefore,
T(n) = θ(n)
 Time complexity of Insertion sort
Worst case: This case will be occurred when data is in reverse sorted
order.
In this case, value of tj will be j. Put the value of tj in equation (1) and
find T(n). Therefore,
T(n) = c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5. 𝑛𝑗<2 𝑗
+ c6. 𝑛𝑗<2(j-1) + c7. 𝑛𝑗<2(𝑗-1) + c8.(n-1)
= c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5. (n+2).(n-1)/2
+ c6.n(n-1)/2+ c7. n(n-1)/2+ c8.(n-1)
= (c5+c6+c7).n2/2 + (c2+c3+c4+(c5 – c6- c7)/2+c8).n +
(c1- c3 - c4 - c5 - c8)
= an2 + bn + c
Clearly T(n) is in quadratic form, therefore, T(n) = θ(n2)
 Time complexity of Insertion sort
Average case: This case will be occurred when data is in any order
except best and worst case.
In this case, value of tj will be j/2. Put the value of tj in equation (1) and
find T(n). Therefore,
T(n) = c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5. 𝑛𝑗<2 (𝑗/2)
+ c6. 𝑛𝑗<2(j/2-1) + c7. 𝑛𝑗<2(𝑗/2-1) + c8.(n-1)
= c1.1 + c2.n + c3.(n-1) + c4.(n-1) + c5. (n+2).(n-1)/4
+ c6.(n-2)(n-1)/4+ c7. (n-2)(n-1)/4+ c8.(n-1)
= (c5+c6+c7).n2/4 + (c2+c3+c4+(c5 – 3c6- 3c7)/4+c8).n +
(c1- c3 - c4 - c5/2 + c6/2+ c7/2 - c8)
= an2 + bn + c
Clearly T(n) is in quadratic form, therefore, T(n) = θ(n2)
 Some other sorting algorithms
 Selection Sort
 Some other sorting algorithms
Selection_sort(A)
n ←length[A]
for i←1 to n-1
min ← i
for j←i+1 to n
if(A[j] < A[min])
min ←j
Interchange A[i] ↔ A[min]

Time complexity, T(n) = θ (n2)

Some other sorting algorithms
Bubble Sort
Some other sorting algorithms
Bubble _Sort(A)
 Divide and Conquer approach
problem.
The divide-and-conquer paradigm involves three steps at each level
of the recursion:

Divide the problem into a number of sub-problems that are smaller

instances of the same problem.

Conquer the sub-problems by solving them recursively. If the sub-

problem sizes are small enough, however, just solve the sub
problems in a straightforward manner.

Combine the solutions to the sub-problems into the solution for

the original problem.
Analysis of Divide and Conquer based algorithm
problem.

When an algorithm contains a recursive call to itself, we

can often describe its running time by a recurrence
equation or recurrence, which describes the overall
running time on a problem of size n in terms of the
running time on smaller inputs. We can then use
mathematical tools to solve the recurrence and provide
bounds on the performance of the algorithm.
Analysis of Divide and Conquer based algorithm
problem. equation for the running time of a divide-
A recurrence
and-conquer algorithm uses the three steps of the basic
paradigm.

The recurrence equation for the running time of a divide-

and-conquer algorithm is the following:-

T(n) = aT(n/b) + D(n) + C(n), if n > c

= θ(1) , otherwise
Where, n is the size of original problem. a is the number
of sub-problems in which the original problem divided at
an instant. Each sub-problems has size n/b. c is a small
integer.
Merge Sort
Merge Sort Algorithm
Merge Sort Algorithm
 Merge Sort Algorithm Analysis

The recurrence equation for the running time of

merge sort algorithm will be

T(n) = 2T(n/2) + θ(1)+ θ(n), if n > 1

= θ(1) , otherwise

It can be modified as:-

T(n) = 2T(n/2) + θ(n) , if n > 1
= θ(1) , otherwise
 Merge Sort Algorithm Analysis

When we solve recurrence equation, modify it

as:-
T(n) = 2T(n/2) + cn , if n > 1
=d , otherwise
Here, c and d are some constants.
 Merge Sort Algorithm Analysis
Iterative method:
T(n) = 2T(n/2) + cn
= 2(2T(n/4) + cn/2) + cn
= 22T(n/4) + 2cn
= 22(2T(n/8)+cn/4) + 2cn
= 23T(n/8)+ 3cn
= ………………….
= 2kT(n/2k) + kcn
= nT(1) + c nlog(n) (Let n = 2k )
= dn + cnlog(n) (since T(1) = d)
= θ(nlog(n))

Therefore, T(n) = θ(nlog(n))

 Asymptotic Notations
 The notations we use to describe the asymptotic
running time of an algorithm are defined in terms of
functions whose domains are the set of natural
numbers N={ 0, 1, 2, …………}.
 We will use asymptotic notations to describe the
running times of algorithms.
 Following notations are used to define the running
time of algorithms.
1. -notation
2. O-notation
3. Ω-notation
4. o-notation
5. ω-notation
 Asymptotic Notations
-notation ( Theta notation )
 For a given function g(n), it is denoted by (g(n)).
 It is defined as following:-
(g(n)) = { f(n) ! ∃positive constants c1, c2 and n0
such that
0 ≤ c1g(n) ≤ f(n) ≤ c2g(n), ∀ n ≥ n0 }

 This notation is said

to be tight bound.
 If f(n) ∈ (g(n)) then
f(n) = (g(n))
 Asymptotic Notations
O-notation ( Big-oh notation )
 For a given function g(n), it is denoted by O(g(n)).
 It is defined as following:-
O(g(n)) = { f(n) ! ∃ positive constants c and n0 such
that
0 ≤ f(n) ≤ cg(n), ∀ n ≥ n0 }

 This notation is said

to be upper bound.
 If f(n) ∈ O(g(n)) then
f(n) = O(g(n))
 If f(n) = (g(n)) then
f(n) = O(g(n))
 Asymptotic Notations
Ω-notation ( Big-omega notation )
 For a given function g(n), it is denoted by Ω(g(n)).
 It is defined as following:-
Ω(g(n)) = { f(n) ! ∃ positive constants c and n0 such
that
0 ≤ cg(n) ≤ f(n), ∀ n ≥ n0 }

 This notation is said

to be lower bound.
 If f(n) ∈ Ω(g(n)) then
f(n) = Ω(g(n))
 If f(n) = (g(n)) then
f(n) = Ω(g(n))
 f(n) = (g(n)) iff f(n) = Ω(g(n)) and f(n) = O(g(n))
 Asymptotic Notations
o-notation ( little-oh notation )
 The asymptotic upper bound provided by O-
notation may or may not be asymptotically tight.
 o-notation denotes an upper bound that is not
asymptotically tight.
 For a given function g(n), it is denoted by o(g(n)).
 It is defined as following:-
o(g(n)) = { f(n) ! for any positive constants c, there
exists a constant n0 such that
0 ≤ f(n) < cg(n), ∀ n ≥ n0 }
 Asymptotic Notations
𝛚-notation ( little-omega notation )
 The asymptotic lower bound provided by 𝛀-
notation may or may not be asymptotically tight.
 𝛚-notation denotes an upper bound that is not
asymptotically tight.
 For a given function g(n), it is denoted by 𝛚(g(n)).
 It is defined as following:-
𝛚(g(n)) = { f(n) ! for any positive constants c, there
exists a constant n0 such that
0 ≤ cg(n) < f(n), ∀ n ≥ n0 }
 Asymptotic Notations
Example: Show that (1/2)n2 - 3n = θ(n2).
Solution: Using definition of θ-notation,
c1g(n) ≤ f(n) ≤ c2g(n), ∀ n ≥ n0
In this question, f(n) = (1/2)n2 - 3n and g(n) = n2 , therefore
c1 n2 ≤ (1/2)n2 - 3n ≤ c2 n2, ∀ n ≥ n0
We divide above by n2, we get
c1 ≤ (1/2)- (3/n) ≤ c2 , ∀ n ≥ n0 ………………..(1)

Now, we have to find c1, c2 and n0, such that equation (1) is satisfied.
Consider, left part of (1), c1 ≤ (1/2)- (3/n) …………. (2)
The value of c1 will be positive value less than or equal to the minimum value
of (1/2)- (3/n). Minimum value of (1/2)- (3/n) = 1/14. Therefore, c1 =
1/14 . This value of c1 will satisfy equation (2) for n ≥ 7.
Here, c1 = 1/14 and n ≥ 7 which satisfy (2).
 Asymptotic Notations
Consider, right part of (1), (1/2)- (3/n) ≤ c2 , …………. (3)
The value of c2 will be positive value greater than or equal
to the maximum value of (1/2)- (3/n). Maximum value of
(1/2) - (3/n) = 1/2. Therefore, c2 = 1/2 . This value of c2
will satisfy equation (3) for n ≥ 1.
Here, c2 = 1/2 and n ≥ 1 which satisfy (3).
Therefore, for c1 = 1/14 , c2 = 1/2 and n0 = 7, equation (1)
is satisfied.
Hence by using definition of θ-notation ,
(1/2)n2 - 3n = θ(n2).
It is proved.
 Asymptotic Notations
Example: Show that 2n+5 = O(n2).
Solution: Using definition of O-notation,
f(n) ≤ cg(n) , ∀ n ≥ n0
In this question, f(n) = 2n+5 and g(n) = n2 , therefore
2n+5 ≤ c n2 ∀ n ≥ n0
We divide above by n2, we get
(2/n)+(5/n2) ≤ c , ∀ n ≥ n0 ………(1)
Now, we have to find c and n0, such that equation (1) is
satisfied.
 Asymptotic Notations
The value of c will be positive value greater than or equal to
the maximum value of (2/n)+(5/n2) .
Maximum value of (2/n)+(5/n2) = 7.
Therefore, c = 7.
Clearly equation (1) is satisfied for c = 7 and n ≥ 1.
Hence by using definition of O-notation ,
2n+5 = O(n2).
It is proved.
 Asymptotic Notations
Example: Show that 2n2+5n+6 = 𝛀(n).
Solution: Using definition of 𝛀 -notation,
cg(n) ≤ f(n) , ∀ n ≥ n0
In this question, f(n) = 2n2+5n+6 and g(n) = n , therefore
cn ≤ 2n2+5n+6 , ∀ n ≥ n0
We divide above by n, we get
c ≤ 2n + 5 + (6/n) , ∀ n ≥ n0 ………(1)
Now, we have to find c and n0, such that equation (1) is
always satisfied.
 Asymptotic Notations
The value of c will be positive value less than or equal to the
minimum value of 2n + 5 + (6/n) .
Minimum value of 2n + 5 + (6/n) = 12.
Therefore, c = 12.
Clearly equation (1) is satisfied for c = 12 and n ≥ 2.
Hence by using definition of 𝛀 -notation ,
2n2+5n+6 = 𝛀 (n).
It is proved.
 Asymptotic Notations
Example: Show that 2n2 = o(n3).
Solution: Using definition of o-notation,
f(n) < cg(n) , ∀ n ≥ n0
Here, f(n) = 2n2, and g(n) = n3. Therefore,
2n2 < cn3 , ∀ n ≥ n0
We divide above by n3, we get
(2/n) < c , ∀ n ≥ n0 …………(1)
for c = 1, there will be n0 = 3, which satisfy (1).
for c = 0.5, there will be n0 = 7, which satisfy (1).
Therefore, for every c, there exists n0 which satisfy (1).
Hence 2n2 = o(n3).
 Asymptotic Notations
Example: Show that 2n2 ≠ o(n2).
Solution: Using definition of o-notation,
f(n) < cg(n) , ∀ n ≥ n0
Here, f(n) = 2n2, and g(n) = n2. Therefore,
2n2 < cn2 , ∀ n ≥ n0
We divide above by n2, we get
2 <c, ∀ n ≥ n0 …………(1)
Clearly for c = 1, inequality (1) does not satisfy.
Therefore, for every c, there does not exist n0 which satisfy
(1). Hence 2n2 ≠ o(n2).
 Asymptotic Notations
Example: Show that 2n2 = 𝛚(n).
Solution: Using definition of 𝛚 -notation,
cg(n) < f(n) , ∀ n ≥ n0
Here, f(n) = 2n2, and g(n) = n. Therefore,
cn < 2n2 , ∀ n ≥ n0
We divide above by n, we get
c < 2n , ∀ n ≥ n0 …………(1)
for c = 1, there will be n0 = 1, which satisfy (1).
for c = 10, there will be n0 = 6, which satisfy (1).
Therefore, for every c, there exists n0 which satisfy (1).
Hence 2n2 = 𝛚(n).
 Asymptotic Notations
Example: Show that 2n2 ≠ 𝛚(n2).
Solution: Using definition of 𝛚 -notation,
cg(n) < f(n) , ∀ n ≥ n0
Here, f(n) = 2n2, and g(n) = n2. Therefore,
cn2 < 2n2 , ∀ n ≥ n0
We divide above by n2, we get
c <2, ∀ n ≥ n0 …………(1)
Clearly for c = 3, there does not exists n0, which satisfy (1).
Therefore, for every c, there does not exist n0 which satisfy
(1). Hence 2n2 ≠ 𝛚(n2).
 Asymptotic Notations
Example: Show that using definition of notations
(a) 3n3-10n+50 = θ(n3)
(b) 5n2-100n ≠ θ(n3)
(c) 3n3-10n+50 = O(n3)
(d) 5n2-100n ≠ O(n)
(e) 3n3-10n+50 = 𝛀(n3)
(f) 5n2-100n ≠ 𝛀(n3)
 Asymptotic Notations
Limit based method to compute notations for a function
𝑓 𝑛
First compute lim = c.
𝑛→∞ 𝑔 𝑛

(1) If c is a constant such that 0< c < ∞, then f(n) = θ(g(n)).

(2) If c is a constant such that 0 ≤ c < ∞, then f(n) = O(g(n)).

(3) If c is a constant such that 0< c ≤ ∞, then f(n) = 𝛀(g(n)).

(4) If c is a constant such that c = 0, then f(n) = o(g(n)).

(5) If c is a constant such that c = ∞, then f(n) = 𝛚(g(n)).

 Asymptotic Notations
Exercises
1. Let f(n) and g(n) be asymptotically non-negative
functions. Using the basic definition of ‚θ-notation, prove
that max(f(n), g(n)) = θ(f(n)+g(n)).
2. Show that for any real constants a and b, where b > 0,
(n+a)b = θ(nb)
3. Solve the followings:-
(a) Is 2n+1 = O(2n) ?
(b) Is 22n = O(2n) ?
4. Prove that o(g(n)) ⋂ 𝛚(g(n)) is the empty set.
5. Prove that n! = 𝛚(2n) and n! = o(nn) .
6. Which is asymptotically larger: lg(lg*n) or lg*(lg n)?
 Asymptotic Notations
Exercise(cont.)
7. Arrange the following in ascending order of growth
or rank the following functions by order of growth.
n3, (3/2)n, 2n, n2, log(n), 22n, loglog(n), n!, en.
8. Let f(n) and g(n) be two asymptotically positive
functions. Prove or disprove the following:-
(a) f(n) = O(g(n)) implies g(n) = O(f(n)).
(b) f(n) + g(n) = θ(min(f(n),g(n))).
(c) f(n) = O(g(n)) implies lg(f(n)) = O(lg(g(n))), where
lg(g(n)) ≥1 and f(n) ≥ 1 for all sufficiently large n.
(d) f(n) = O(g(n)) implies 2f(n) = O(2g(n)).
(e) f(n) = O(((f(n))2)
 Asymptotic Notations
Exercise(cont.)
Solution-(4):
Assume f(n) ϵ o(g(n)) ⋂ 𝛚(g(n)).
⇒ f(n) ϵ o(g(n)) and f(n) ϵ 𝛚(g(n))
⇒ f(n) < cg(n) and cg(n) < f(n) , for any c
⇒ Both inequality can not be true for any c.
Therefore, our assumption is incorrect.
Hence, f(n) ∉ o(g(n)) ⋂ 𝛚(g(n)). Therefore,
o(g(n)) ⋂ 𝛚(g(n)) is the empty set.
 Asymptotic Notations
Exercise(cont.)
Solution-(5): n! = 𝛚(2n) and n! = o(nn)
(i) c2n < n!
for c = 1 , n0 = 4
for c = 10, n0 = 6
(ii) n! < cnn
for c = 1 , n0 = 2
for c = 0.1, n0 = 3
 Asymptotic Notations
Exercise(cont.)
Solution-(6): lg(lg*n) or lg*(lg n)
lg(lg*n) = lg(lg n)*
lg*(lg n) = (lg lg n)*

Solution-(7):
n3, (3/2)n, 2n, n2, log(n), 22n, loglog(n), n!, en
Ascending order is
loglog(n), log(n), n2, n3, (3/2)n , 2n , en, n!, 22n
 Asymptotic Notations
AKTU questions
1. Take the following list of functions and arrange them in
ascending order of growth rate. That is, if function g(n)
immediately follows function f(n) in your list, then it
should be the case that f(n) is O(g(n)). f1(n) = n2.5, f2(n) =
√2n, f3(n) = n + 10, f4(n) = 10n, f5(n) = 100n, and
f6(n) = n2 log n

1. Rank the following by growth rate: n, 2lg √n, log n, log

(logn), log2n, (lgn)lgn, 4, (3/2)n, n!
 Recurrence relation
Recurrence equations will be of the following form:-

(1) T(n) = aT(n/b) + f(n)

(2) T(n) = T(n-1) + n
(3) T(n) = T(n/3) + T(2n/3) + n
(4) T(n) = T(n-1) + T(n-2)

Some approaches to sole recurrence relations

(1) Iterative method
(2) Substitution method
(3) Recurrence Tree
(4) Master theorem method
 Substitution method
The substitution method for solving recurrences comprises
two steps:
1. Guess the form of the solution.
2. Use mathematical induction to find the constants and show
that the solution works.
Example: Find the upper bound of following recurrence
relation T(n) = 2T(⌊n/2⌋) + n. ………..(1)
Solution: We will solve this using substitution method.
Guess the upper bound of this equation is T(n) = O(nlogn).
Now, we have to prove that this guessing solution is
correct.
By definition of upper bound,
T(n) ≤ c nlogn, ∀ n ≥ n0. ……………(2)
 Substitution method
We will prove inequality (2) using induction method.
Assume T(1) = 1.
Using equation (1),
T(2) = 2T(⌊2/2⌋) + 2 = 2T(1) + 2 = 4
T(3) = 2T(⌊3/2⌋) + 3 = 2T(1) + 3 = 5
For n=1.
T(1) ≤ c 1log1 ⇒ 1 ≤ c . 0 ⇒ 1 ≤ 0 (False)
Therefore, equation (2) is false for n = 1.
For n=2.
T(2) ≤ c 2log2 ⇒ 4 ≤ c . 2 ⇒ 4 ≤ 2c (True for c ≥ 2)
Therefore, equation (2) is true for n = 2.
For n=3.
T(3) ≤ c 3log3 ⇒ 5 ≤ 3c.log3 (True for c ≥ 2)
Therefore, equation (2) is true for n = 3.
 Substitution method
Assume equation (2) is true for n= n/2. We will prove for n.
Since equation (2) is true for n = n/2, therefore
T(n/2) ≤ c (n/2)log(n/2), ∀ n ≥ n0 ………(3)
Now for n,
T(n) = 2T(⌊n/2⌋) + n
≤ 2c (⌊n/2⌋)log(⌊n/2⌋) + n
≤ 2c (n/2)log(n/2) + n
= cn (log n – log 2) + n
= cn (log n – 1) + n
= cnlogn – cn + n
≤ cnlogn if c ≥ 1
Therefore, T(n) ≤ cnlogn if c ≥ 1.
Hence, equation (2) is also proved for n. Therefore, guessing solution is
correct.
Therefore the upper bound of given recurrence relation is
T(n) = O(nlogn).
 Substitution method
Example: Find the upper bound of following recurrence
relation T(n) = 2T(⌊n/2⌋ + 17) + n
Solution: When n is large, the difference between ⌊n/2⌋ and
⌊n/2⌋ + 17 is not that large. Therefore, given equation is
equivalent to the following equation
T(n) = 2T(⌊n/2⌋) + n
Similarly, since ⌊n/2⌋ and n/2 are approximately same,
therefore above equation is equivalent to the following
equation
T(n) = 2T(n/2) + n
Since this equation is equivalent to the previous question
therefore upper bound will be T(n) = O(nlogn).
 Substitution method
Example: Solve the following recurrence relation
T(n) = 2T(⌊√n⌋ ) + lg n
Solution: Here we change the variable n to m.
Let n = 2m
Put n = 2m in given equation, we get
T(2m) = 2T(⌊√2m⌋ ) + m
= 2T(⌊2m/2⌋ ) + m
Let T(2m) = S(m), therefore
S(m) = 2 S(⌊m/2⌋ ) + m
Since this equation is equialent to the first question,
therefore its solution will be S(m) = mlog(m).
Hence T(n) = T(2m) = S(m) = mlog(m) = log(n) loglog(n)
i.e. T(n) = log(n) loglog(n)
 Substitution method
Example: Solve the following recurrence relation
T(n) = T(⌊n/2⌋ ) + T(⌈ n/2 ⌉ ) + 1
Solution: Since ⌊n/2⌋ and ⌈ n/2 ⌉ are approximately equal to n/2,
therefore given equation is equivalent to following equation
T(n) = T(n/2) + T(n/2) + 1
T(n) = 2T(n/2) + 1
Now, we guess the solution is T(n) = O(n). Therefore, we have to prove
T(n) ≤ cn-d, ∀ n ≥ n0. ……………(2)
Assume this is true for n = n/2, therefore
T(n/2) ≤ cn/2 - d …………..(3)
Now for n,
T(n) = 2T(n/2) + 1
≤ 2 (cn/2 –d) + 1 (using equation (3))
= cn -2d + 1
≤ cn-d if d ≥ 1
Hence T(n) ≤ cn-d for d ≥ 1. Therefore T(n) = O(n).
 Recurrence tree method
In a recursion tree, each node represents the cost of a
single sub-problem somewhere in the set of recursive
function invocations. We sum the costs within each level of
the tree to obtain a set of per-level costs, and then we sum
all the per-level costs to determine the total cost of all
levels of the recursion.

A recursion tree is best used to generate a good guess,

which you can then verify by the substitution method.
 Recurrence tree method
Example: Solve the following recurrence equation
T(n) = 3T(⌊n/4⌋) + θ(n2)
Solution: This equation is equivalent to the following
equation T(n) = 3T(n/4) + cn2
Here, we create recurrence tree for above equation.
Recurrence tree method
 Recurrence tree method
Let h is the height of the tree. Then
n/4h = 1 ⇒ h = log4n
Now we add up the costs over all levels to determine the cost
for the entire tree:
T(n) = cn2 + (3/16)cn2 + (3/16)2 cn2 + …………….+
(3/16)log4n -1 cn2 + θ(nlog43)
𝑙𝑜𝑔4𝑛;1 3 𝑖
= cn2 𝑖<0 + θ(n log43)
16
∞ 3 𝑖
< cn2 𝑖<0 16 + θ(n log43)

= cn2 (1/(1-(3/16))) + θ(nlog43)

= (16/13) cn2 + θ(nlog43)
= O(n2)
Therefore, solution will be T(n) = O(n2).
 Recurrence tree method
Example: Solve the following recurrence relation using recurrence
tree method
T(n) = T(n/3) + T(2n/3) + θ(n)
Solution: This recurrence relation is equivalent to the following
relation
T(n) = T(n/3) + T(2n/3) + cn
Recurrence tree for this equation will be the following:-

T(n) cn cn

T(n/3) T(2n/3) cn/3 2cn/3

(a)
(b)
T(n/9) T(2n/9) T(2n/9) T(4n/9)
(c)
Recurrence tree method

(d)
 Recurrence tree method
Let the height of the tree is h. Therefore
𝑛
3 𝑕 = 1 ⇒ n = lg3/2n
2

Therefore, the total cost of the tree is

T(n) < cn + cn + …………………+ cn
= (h+1) cn
= (lg3/2n + 1)cn
= cnlg3/2n + cn
= O(nlg3/2n)
= O(nlgn)
Therefore T(n) = O(nlgn)
 Recurrence tree method
Exercise
(1) Draw the recursion tree for T(n) = 4 T(⌊n/2⌋) + cn, where
c is a constant and provide a tight asymptotic bound on its
solution. Verify your bound by the substitution method.
(2) Use a recursion tree to give an asymptotically tight
solution to the recurrence T(n) = T(n-a) + T(a) + cn, where a
≥ 1 and c > 0 are constants.
(3) Use a recursion tree to give an asymptotically tight
solution to the recurrence T(n) = T(αn) + T((1- α)n) + cn,
where α is a constant in the range 0 < α < 1 and c > 0 is also
a constant.
 Recurrence tree method
Exercise(Solution)
(3.) Assume ½ ≤ α < 1. Then 0 < (1-α) ≤ ½ .
Recurrence tree for this recurrence relation will be the
following: cn …………………………cn

cαn c(1-α)n ……………cn

cα2n cα(1- α)n cα(1- α)n c(1- α)2n ………cn

….. ……… …….. …….. ………cn
….. ……. ……. T(1)
....... …….. T(1)

T(1)
Let h is the height of the tree. Therefore,
αhn = 1 ⇒ h = log1/α(n)
Therefore, total cost of the tree
T(n) = cn + cn +cn + …………………+ cn
= (h+1) cn
= (log1/α(n) +1) cn
= cn log1/α(n) + cn
= O(n log1/α(n) )
= O(nlog n)
Therefore, T(n) = O(nlog n).
 Master Theorem
Let a ≥ 1 and b > 1 be constants, let f(n) be a function, and let
T(n) be defined on the nonnegative integers by the
recurrence
T(n) = aT(n/b) + f(n);
Where we interpret n/b to mean either ⌊n/b⌋ or ⌈n/b⌉. Then
T(n) has the following asymptotic bounds:
1. If f(n) = 𝑂(nlogba − ϵ) for some constant ϵ > 0, then
T(n) = θ(nlogba).
2. If f(n) =θ(nlogba) , then T(n) = θ(nlogba lg n).
3. If f(n) = 𝛺(nlogba + ϵ) for some constant ϵ > 0, and if
af(n/b) ≤ cf(n) for some constant c < 1 and all sufficiently
large n, then T(n) = θ(f(n).
 Master Theorem Method
Example: Solve the following recurrence relations using
master theorem method
(a) T(n) = 9T(n/3) + n
(b) T(n) = T(2n/3) + 1
(c) T(n) = 3T(n/4) + n log n
Solution:
(a) Consider T(n) = 9T(n/3) + n
In this recurrence relation, a = 9, b= 3 and f(n) = n.
Therefore, n𝑙𝑜𝑔𝑏𝑎 = nlog39 = n2
Clearly, n𝑙𝑜𝑔𝑏𝑎 > f(n) , therefore case1 can be applied.
Now determine ϵ such that f(n) = 𝑂(n2−ϵ) . Here ϵ = 1.
Therefore case 1 will be applied.
Hence solution will be T(n) = θ(n2).
 Master Theorem Method
Solution:
(b) Consider T(n) = T(2n/3) + 1
In this recurrence relation, a = 1, b= 3/2 and f(n) = 1.
Therefore, n𝑙𝑜𝑔𝑏𝑎 = nlog3/2 1 = 0
Clearly, f(n) =θ(nlogba) , therefore case 2 will be applied.
Hence solution will be T(n) = θ(log n).
 Master Theorem Method
Solution:
(c) Consider T(n) = 3T(n/4) + n log n
In this recurrence relation, a = 3, b= 4 and f(n) = n log n.
Therefore, n𝑙𝑜𝑔𝑏𝑎 = nlog43 = n0.793
Clearly, n𝑙𝑜𝑔𝑏𝑎 < f(n) , therefore case 3 can be applied.
Now determine ϵ such that f(n) =𝛺(n0.793+ϵ) . Here ϵ = 0.207.
Now, af(n/b) ≤ cf(n) imply that 3f(n/4) ≤ cf(n)
⇒ 3(n/4)log (n/4) ≤ c n log n
⇒ (¾)log(n/4) ≤ c log n
Clearly above inequality is satisfied for c = 3/4. Therefore
case 3 will be applied.
Hence solution will be T(n) = θ(n log n).
 Master theorem method
Example: Solve the following recurrence relation
T(n) = 2T(n/2) + n log n
Solution: Here, a = 2 , b=2 and f(n) = n log n.
nlogba = nlog22 = n
If we compare nlogba and f(n), we get f(n) is greater than nlogba
. Therefore, case 3 may be applied.
Now we have to determine ϵ >0 which satisfy f(n) = 𝛺
(nlogba+ϵ), i.e. n logn = 𝛺(n1+ϵ). Clearly there does not exist
any ϵ which satisfy this condition. Therefore case 3 can not
be applied. Other two cases are also not satisfied. Therefore
Master theorem can not be applied in this recurrence
relation.
 Generalized Master theorem
Theorem: If f(n) = θ(nlogba lgkn), where k ≥ 0, then the
solution of recurrence will be T(n) = θ(nlogba lgk+1n).
Now, consider the previous example:-
T(n) = 2T(n/2) + n log n
Solve it using aboe theorem,
Here a =2, b= 2, and k = 1. Therefore, the solution of this
recurrence will be
T(n) = θ(nlog22 lg1+1n)
= θ(n lg2n)
Hence, T(n) = θ(n lg2n)
 Recurrence relation
Exercise
1. Use the master method to give tight asymptotic bounds for
the following recurrences:-
(a) T(n) = 8T(n/2) + θ(n2)
(b) T(n) = 7T(n/2) + θ(n2)
(c) T(n) = 2T(n/4) + 1
(d) T(n) = 2T(n/4) + √n
2. Can the master method be applied to the recurrence
4T(n/2) + n2 log n ? Why or why not? Give an asymptotic
upper bound for this recurrence.
 Recurrence relation
3. Give asymptotic upper and lower bounds for T(n) in each
of the following recurrences. Assume that T(n) is constant
for n ≤ 2. Make your bounds as tight as possible, and justify
your answers.
(a) T(n) = 2T(n/2) + n4
(b) T(n) = T(7n/10) + n
(c) T(n) = 16T(n/4) + n2
(d) T(n) = 2T(n/4) + √n
(e) T(n) = T(n-2) + n2
(f) T(n) = 7T(n/3) + n2
(g) T(n) = 3T(n/3 -2) + n/2
 Recurrence relation
4. Give asymptotic upper and lower bounds for T(n) in each of the
following recurrences. Assume that T(n) is constant for sufficiently small
n. Make your bounds as tight as possible, and justify your answers.
(a) T(n) = 4T(n/3) + n lgn
(b) T(n) = 3T(n/3) + n/lgn
(c) T(n) = 2T(n/2) + n/lgn
(d) T(n) = T(n/2) + T(n/4) + T(n/8) + n
(e) T(n) = T(n-1) + 1/n
(f) T(n) = T(n-1) + lg n
(g) T(n) = T(n-2) + 1/lg n
(h) T(n) = √n T(√n) + n
 Recurrence relation
Some exercise solution
(3-e) Recurrence relation is T(n) = T(n-2) + n2 .
We will solve it using iteration method.
T(n) = T(n-2) + n2
= T(n-4) + (n-2)2 + n2
= T(n-6) + (n-4)2 + (n-2)2 + n2
……………………………………………….
………………………………………………

= T(0) + 22 + 42 + 62 +……………..+(n-2)2 + n2
= d + 22 + 42 + 62 +………+(n-2)2 + n2 (Let T(0) =d)
𝑛 𝑛:1 𝑛:2
=d+ = θ(n3)
6
 Recurrence relation
Some exercise solution
(3-g) Recurrence relation is T(n) = 3T(n/3 -2) + n/2.
This recurrence relation is equivalent to the following
equation T(n) = 3T(n/3) + n/2
Apply master theorem, here a = 3, b = 3 and f(n) = n/2.
nlogba = nlog33 = n
Clearly, f(n) = n/2 = θ(n) = θ(nlogba)
Therefore case 2 will be applied. Hence the solution is
T(n) = θ(n lg n)
 Recurrence relation
Some exercise solution
(4-a) Recurrence relation is T(n) = 4T(n/3) + n lgn.
Apply master theorem, here a = 4, b = 3 and f(n) = n lgn.
nlogba = nlog34 = n1.26
Clearly, f(n) = n lgn = O(n1.26-ϵ) = O(nlogba - ϵ)
Therefore case 1 will be applied. Hence the solution is
T(n) = θ(n1.26)
 Recurrence relation
Some exercise solution
(4-b) Recurrence relation is T(n) = 3T(n/3) + n/lgn.
We will use recurrence tree method to solve it.
Recurrence tree will be n/lgn
…………………… n/lgn

𝑛 𝑛
………… n/lg 𝑛
3
𝑛 𝑛 𝑛 𝑛 /lg
/lg /lg 3 3
3 3 3 3

𝑛 𝑛 𝑛
𝑛 𝑛 ………………………………….. /lg …. … n/lg
/lg 9 9 32
9 9

………………………………………………..
T(1)=d T(1) = d…………………………T(1) = d……. 3hd
Here d -> constant , h-> height of tree
 Recurrence relation
Now, we calculate height of tree.
𝑛
Clearly, = 1 , ⇒ h = 𝑙𝑜𝑔3𝑛.
3𝑕
Now, total cost of this tree is
𝑛 𝑛 𝑛 𝑛
T(n) = + 𝑛 + 𝑛 + …………+ 𝑛 + 3hd
𝑙𝑔𝑛 lg( 3 ) lg(32) lg(3ℎ−1)

1 1 1 1
=n( + 𝑛 + 𝑛 + …………+ 𝑛 ) + 3hd
𝑙𝑔𝑛 lg( 3 ) lg(32) lg(3ℎ−1)

Substituting n = 3h, we get

1 1 1 1
=n( + + + …………+ ) + dn
𝑙𝑔3𝑕 lg(3𝑕;1) lg(3𝑕;2) lg(3)
1 1 1 1
=n( + + + ……+ ) + dn
𝑕 𝑙𝑔3 (h;1)lg 3 (h;2)lg 3 lg 3
1 1 1 1 1
= (n/lg3) ( + + + ……+ + ) + dn
𝑕 (h;1) (h;2) 2 1
 Recurrence relation
𝑛 1 1 1 1 1
T(n) = ( 1+ + + + + ⋯ … … … + ) + 𝑑𝑛
𝑙𝑔3 2 3 4 5 𝑕

Suppose h = 2m
𝑛 1 1 1 1 1
= ( 1+ + + + + ⋯………+ 𝑚 ) + 𝑑𝑛
𝑙𝑔3 2 3 4 5 2
𝑛 1 1 1 1 1 1 1 1
< ( 1+( + ) + + + + + ⋯………+ ( + +
𝑙𝑔3 2 2 4 4 4 4 2𝑚 2𝑚
1
……..+ )) + 𝑑𝑛
2𝑚
𝑛
= ( 1+1+1+………….+1) + dn
𝑙𝑔3
𝑛 𝑛
= (m+1) + dn = (lg h +1) +dn
𝑙𝑔3 𝑙𝑔3
𝑛
= (lg log3n +1) +dn
𝑙𝑔3

= O(n lg lg n)
 Some exercise solution
(4-d) Recurrence relation is
T(n) = T(n/2) + T(n/4) + T(n/8) + n
We will use recurrence tree method to solve it.
Recurrence tree will be n n

𝑛 𝑛 𝑛
7𝑛
2 4 8
8
7
𝑛 𝑛 𝑛 𝑛 𝑛 𝑛
( 8 )2 𝑛
𝑛 𝑛 𝑛
4 8 16 8 16 32 16 32 64
7
( 8 )3 𝑛

𝑛
Let h is the height of the tree. Therefore, = 1 ⇒ h = lg n.
2𝑕
 Some exercise solution
Therefore total cost
7𝑛 7 2 7 3 7 ℎ
T(n) =n+ + 𝑛+ 𝑛 + ………….+ 𝑛
8 8 8 8
7 7 2 7 3 7 ℎ
= n( 1+ + + + ………….+ )
8 8 8 8
7 7 2 7 3
< n ( 1+ + + + … … … … … … .)
8 8 8
Here, we consider up to infinite term because common ratio
7
of this series is that is less than 1.
8
1
= n( 7 )
(1; )
8

= 8n = O(n)
Therefore, T(n) = O(n).
 AKTU Examination Questions
1. Solve the recurrence T (n) = 2T(n/2) + n2+ 2n+ 1
2. Solve the recurrence using recursion tree method:
T (n) = T (n/2) + T (n/4) + T (n/8) + n
3. Use a recursion tree to give an asymptotically tight solution
to the recurrence T(n) = T(αn) + T((1 - α)n) + cn, where α is a
constant in the range 0 <α< 1 and c > 0 is also a constant.
4. The recurrence T (n) = 7T (n/3) + n2 describes the running
time of an algorithm A. Another competing algorithm B has a
running time of S (n) = a S (n/ 9) + n2. What is the smallest
value of ‗a‘ such that A is asymptotically faster than B?
5. Solve the recurrence relation by substitution method
T(n)= 2T(n/2) + n
 AKTU Examination Questions
6. Show that the solution to T (n) = 2T (⌊n/2⌋ + 17) + n is
O(nlgn).
7. Solve the recurrence: T (n) = 50 T (n/49) + log n!
8. Solve the following recurrence using Master method:
T (n) = 4T (n/3) + n2
9. Find the time complexity of the recurrence relation
T(n) = n +T(n/10)+T(7n/5)
10. Solve the following By Recursion Tree Method
T(n) = n + T(n/5)+T(4n/5)
11. The recurrence T (n) =7T (n/2) +n2 describe the running
time of an algorithm A. A competing algorithm A has a running
time of T‘ (n) =aT‘ (n/4) +n2. What is the largest integer value
for a A‘ is asymptotically faster than A?
 Heapsort
Heap

 The (binary) heap data structure is an array object

that we can view as a nearly complete binary tree.

 Each node of the tree corresponds to an element

of the array. The tree is completely filled on all
levels except possibly the lowest, which is filled
from the left up to a point.
 Heapsort

Max-heap Array
 Heapsort
Index: If i the index of a node, then the index of parent and its
child are the following:-
Parent(i) = ⌊i/2⌋
Left(i) = 2i
Right(i) = 2i+1
Note: Root node has always index 1 i.e. A[1] is root element.
Heap-size: Heap-size is equal to the number of elements in the heap.
Height of a node: The height of a node in a heap is the
number of edges on the longest simple downward path from the
node to a leaf.
Height of heap: The height of the heap is equal to the height of its
root.
 Heapsort
Types of heap
There are two kinds of binary heaps:
(1) max-heaps (2) min-heaps
Max-heap: The heap is said to be max-heap if it satisfy the
max-heap property.
The max-heap property is that the value at the parent node
is always greater than or equal to value at its children.
Min-heap: The heap is said to be min-heap if it satisfy the
min-heap property.
The min-heap property is that the value at the parent node
is always less than or equal to value at its children.
 Heapsort
Heap sort algorithm consists of the following two sub-
algorithms.
(1)Max-Heapify: It is used to maintain the max-heap
property.
(2)Build-Max-Heap: It is used to construct a max-heap for
the given set of elements.
 Heapsort
Max-Heapify Algorithm
Action done by max-heapify algorithm is shown in the following
figures:-
 Heapsort
Max-Heapify Algorithm
 Heapsort
Time complexity of Max-Heapify Algorithm
The running time of max-heapify is determined by the
following recurrence relation:-
T(n) ≤ T(2n/3) + θ(1)
Here n is the size of the sub-tree rooted at node i.
Using master theorem, the solution of this recurrence
relation is
T(n) = θ(lg n)
 Heapsort
Build-Max-Heap Algorithm
Example: Construct max-heap corresponding to the
following elements
4, 1, 3, 2, 16, 9, 10, 14, 8, 7.
Solution:
 Heapsort
Build-Max-Heap Algorithm (cont.)
 Heapsort
Build-Max-Heap Algorithm (cont.)

The running time of this algorithm is

T(n) = O(n lg n)
But this upper bound is not asymptotically tight.
Now, we shall calculate tight upper bound.
 Heapsort
Time complexity of Build-Max-Heap Algorithm
We can derive a tighter bound by observing that the time for MAX-
HEAPIFY to run at a node varies with the height of the node in the tree,
and the heights of most nodes are small.

Our tighter analysis relies on the properties that an n-element heap has
height ⌊lg n⌋ and at most ⌈n/2h+1⌉ nodes of any height h.

If h is the height of the sub-tree then running time of max-

heapify is O(h).
Therefore, total cost of build-max-heap is
T(n) =
 Heapsort
Time complexity of Build-Max-Heap Algorithm
Now, T(n) =

=
Since

Therefore,
T(n) = O( 2n)
= O(n)
 Heapsort Algorithm
Example: Sort the following elements using heapsort
5, 13, 2, 25, 7, 17, 20, 8, 4.
Solution: The operation of HEAPSORT is shown as
following:-
Heapsort Algorithm
Heapsort Algorithm
 Heapsort Algorithm

Time complexity: The HEAPSORT procedure takes time

O(nlgn), since the call to BUILD-MAX-HEAP takes time
O(n) and each of the n-1 calls to MAX-HEAPIFY takes time
O(lgn).
 Heapsort Algorithm
Exercise
1. What are the minimum and maximum numbers of
elements in a heap of height h?
2. Show that an n-element heap has height ⌊lgn⌋.
3. Is the array with values 23, 17, 14, 6, 13, 10, 1, 5, 7, 12 a
max-heap?
4. Show that there are at most ⌈n/2h+1⌉nodes of height h in
any n-element heap.
5. What is the running time of HEAPSORT on an array A of
length n that is already sorted in increasing order? What
about decreasing order?
 Heapsort Algorithm
AKTU Examination Questions

1. Sort the following array using Heap-Sort techniques—

5,8,3,9,2,10,1,35,22
2. How will you sort following array A of elements using
heap sort: A = (23, 9, 18, 45, 5, 9, 1, 17, 6).
 Quicksort
Quicksort, like merge sort, applies the divide-and-conquer
paradigm. The three-step divide-and-conquer process for
sorting a typical subarray A[p .. r] is the following:-

Divide: Partition (rearrange) the array A[p .. r] into two

(possibly empty) subarrays A[p .. q-1] and A[q+1 .. R] such that
each element of A[p .. q-1] is less than or equal to A[q], which is,
in turn, less than or equal to each element of A[q+1 .. r].
Compute the index q as part of this partitioning procedure.
Conquer: Sort the two subarrays A[p .. q-1] and A[q+1 .. r] by
recursive calls to quicksort.
Combine: Because the subarrays are already sorted, no work is
needed to combine them: the entire array A[p .. r] is now
sorted.
 Quicksort
Algorithm
It divides the large array into smaller sub-arrays. And then quicksort
recursively sort the sub-arrays.
Pivot
1. Picks an element called the "pivot".
Partition
2. Rearrange the array elements in such a way that the all values lesser
than the pivot should come before the pivot and all the values greater
than the pivot should come after it.
This method is called partitioning the array. At the end of the partition
function, the pivot element will be placed at its sorted position.
Recursive
3. Do the above process recursively to all the sub-arrays and sort the
elements.
 Quicksort
Algorithm
Base Case
If the array has zero or one element, there is no need to call the
partition method.
So we need to stop the recursive call when the array size is less than or
equal to 1.

Pivot
There are many ways we can choose the pivot element.
i) The first element in the array
ii) The last element in the array
iii) The middle element in the array
iv) We can also pick the element randomly.
Note: In our algorithm, we are going to pick the last element as the
pivot element.
 Quicksort
Example: Sort the following elements using quicksort
2, 8, 7, 1, 3, 5, 6, 4
Solution: Here, we pick the last element in the list as a pivot
1 2 3 4 5 6 7 8
82
8 2 8 7 1 3 5 6 4
7 2 8 7 1 3 5 6 4

2 8 7 1 3 5 6 4

2 8 7 1 3 5 6 4

2 1 7 8 3 5 6 4

2 1 3 8 4 5 6 4
 Quicksort
82 1 2 3 4 5 6 7 8

8 2 1 3 8 7 5 6 4
7
2 1 3 4 7 5 6 8
Partition completed in first pass

2 1 3 4 7 5 6 8

2 1 3 4 7 5 6 8

2 1 3 4 7 5 6 8

2 1 3 4 7 5 6 8

2 1 3 4 7 5 6 8
Partition completed in second pass
 Quicksort
82 1 2 3 4 5 6 7 8

8 2 1 3 4 7 5 6 8
7
2 1 3 4 7 5 6 8

1 2 3 4 5 7 6 8

1 2 3 4 5 6 7 8
Partition completed in third pass
1 2 3 4 5 6 7 8

1 2 3 4 5 6 7 8
Process
1 2 3 4 5 6 7 8 completed
 Quicksort Algorithm
8
Quicksort(A, p, r)
1 if p < r
2 q = PARTITION(A, p, r)
3 QUICKSORT(A, p, q-1)
4 QUICKSORT(A, q+1, r)

To sort an entire array A, the initial call is QUICKSORT(A, 1,

length[A]).
 Quicksort Algorithm
8
 Quicksort Algorithm Analysis
The running time of quicksort algorithm is
T(n) = T(n1) + T(n2) + θ(n) ……………….(1)
Here n1 and n2 are the size of sub-problems.

The running time of quicksort depends on whether the

partitioning is balanced or unbalanced, which in turn depends
on which elements are used for partitioning. If the
partitioning is balanced, the algorithm runs asymptotically as
fast as merge sort. If the partitioning is unbalanced, however,
it can run asymptotically as slowly as insertion sort.
 Quicksort Algorithm Analysis
Worst-case partitioning
The worst-case behavior for quicksort occurs when the
partitioning routine produces one sub-problem with n-1
elements and one with 0 elements. Let us assume that this
unbalanced partitioning arises in each recursive call. Therefore,
T(n) = T(0) + T(n-1) + θ(n)
T(n) = T(n-1) + θ(n) (since T(0) = θ(1) )
After solving this recurrence relation, we get
T(n) = θ(n2)

Therefore the worst-case running time of quicksort is no better

than that of insertion sort. Moreover, the θ(n2) running time
occurs when the input array is already completely sorted—a
common situation in which insertion sort runs in O(n) time.
 Quicksort Algorithm Analysis
Best-case partitioning
If PARTITION produces two sub-problems, each of size no
more than n/2, since one is of size ⌊n/2⌋ and one of size
⌈n/2⌉-1 . In this
case, quicksort runs much faster. The recurrence for the
running time is then
T(n) = 2T(n/2) + θ(n)
After solving this recurrence relation, we get
T(n) = θ(n lgn)

Note: By equally balancing the two sides of the partition at

every level of the recursion, we get an asymptotically faster
algorithm.
 Quicksort Algorithm Analysis
Balanced partitioning
The average-case running time of quicksort is much closer to
the best case than to the worst case.
Suppose that the partitioning algorithm always produces a 9-
to-1 proportional split. In this case, running time will be
T(n) = T(9n/10) + T(n/10) + θ(n)
This relation is equivalent to the following
T(n) = T(9n/10) + T(n/10) + cn
We can solve this recurrence using recurrence tree method.
 Quicksort Algorithm Analysis
Recurrence tree will be
 Quicksort Algorithm Analysis
Therefore, the solution of recurrence relation will be
T(n) ≤ cn + cn + cn+……………..+cn
= cn(1+lg10/9n)
= cn + cnlg10/9n

Therefore, T(n) = O(nlgn)

 Quicksort Algorithm Analysis
Exercise
1. Sort the following elements using quicksort
13, 19, 9, 5, 12, 8, 7, 4, 21, 2, 6, 11.
2. What is the running time of QUICKSORT when all
elements of array A have the same value?
3. Show that the running time of QUICKSORT is‚ θ(n2) when
the array A contains distinct elements and is sorted in
decreasing order.
4. Suppose that the splits at every level of quicksort are in the
proportion 1-α to α, where 0 < α ≤ 1/2 is a constant. Show that
the minimum depth of a leaf in the recursion tree is
approximately lgn/ lgα and the maximum depth is
approximately lgn/ lg(1-α) . (Don‘t worry about integer round-
off.)
A randomized version of quicksort
 A randomized version of quicksort

The running time of randomized quick sort will be

T(n) = O(nlgn)
 Sorting in Linear Time
Comparison Sort
In a comparison based sort, we use only comparisons between
elements to gain order information about an input sequence
<a1, a2,……………., an>. That is, given two elements
ai and aj , we perform one of the tests ai < aj, ai ≤ aj , ai=aj , ai
≥ aj, or ai > aj to determine their relative order.

The decision-tree model

We can view comparison sorts abstractly in terms of decision
trees.
A decision tree is a full binary tree that represents the
comparisons between elements that are performed by a
particular sorting algorithm operating on an input of a given
size.
 Sorting in Linear Time
In a decision tree, we annotate each internal node by i:j for
some i and j in the range 1≤i, j ≤ n, where n is the number of
elements in the input sequence. We also annotate each leaf by
a permutation < 𝞹(1), 𝞹(2), 𝞹(3), …………, 𝞹(n) >.

 The execution of the sorting algorithm corresponds to

tracing a simple path from the root of the decision tree
down to a leaf.
 Each internal node indicates a comparison ai ≤ aj . The left
sub-tree then dictates subsequent comparisons once we
know that ai ≤ aj , and the right sub-tree dictates subsequent
comparisons knowing that ai > aj .
 When we come to a leaf, the sorting algorithm has
established the ordering <a𝞹(1), a𝞹(2),…………., a𝞹(n)>.
 Sorting in Linear Time
The decision tree operating on three elements is the
following:-
 Sorting in Linear Time
Theorem:
Any comparison sort algorithm requires 𝛺(nlg n) comparisons
in the worst case.
Proof: The worst-case number of comparisons for a given
comparison sort algorithm equals the height of its decision
tree.
Consider a decision tree of height h with l reachable leaves
corresponding to a comparison sort on n elements. Because
each of the n! permutations of the input appears as some
leaf, we have n! ≤ l . Since a binary tree of height h has no
more than 2h leaves, therefore
n! ≤ l ≤ 2h ⇒ h ≥ lg(n!)
Therefore h = 𝛺(nlgn)
 Counting sort
 Counting sort assumes that each of the n input elements
is an integer in the range 0 to k, for some integer k.

 When k = O(n), the sorting algo. runs in θ(n) time.

 Counting sort determines, for each input element x, the

number of elements less than x. It uses this information to
place element x directly into its position in the output
array.
 Counting sort
Example: Sort the following elements using counting sort
2, 5, 3, 0, 2, 3, 0, 3
Solution:
Counting sort
 Counting sort

 Time complexity of counting sort is

T(n) = θ(n+k)
 If k ≤ n, then T(n) = θ(n)
 Stable sorting
A sorting algorithm is said to be stable if two objects with
equal keys appear in the same order in sorted output as they
appear in the input array to be sorted.

Some stable sorting algorithms

1. Counting sort
2. Insertion sort
3. Merge Sort
4. Bubble Sort

Some unstable sorting algorithms

1. Selection sort
2. Quicksort
3. Heap sort
 Radix sorting
 This sorting algorithm assumes all the numbers must be of
equal number of digits.

 This algorithm sort all the elements on the basis of digits

used in the number.

 First sort the numbers on the basis of least significant digit.

Next time on the basis of 2nd least significant digit. After it
on the basis of 3rd least significant digit. And so on. At the
last, it sort on the basis of most significant digit.
 Radix sorting
Example: Sort the following elements using radix sort
326, 453, 608, 835, 751, 435, 704, 690.
Solution: In these elements, number of digits is 3. Therefore, we
have to used 3 iterations.
Radix sorting
 Radix sorting
Time complexity of radix sort is
T(n) = θ(d(n+k))

Note: When d is constant and k=O(n), we can make radix

sort run in linear time i.e T(n) = θ(n).
 Bucket sorting
 Bucket sort assumes that the input is generated by a
random process that distributes elements uniformly and
independently over the interval [0,1).

 Bucket sort divides the interval [0,1) into n equal-sized

subintervals, or buckets, and then distributes the n input
numbers into the buckets. Since the inputs are uniformly
and independently distributed over [0,1), we do not expect
many numbers to fall into each bucket. To produce the
output, we simply sort the numbers in each bucket and
then go through the buckets in order, listing the elements
in each.
 Bucket sorting
Example: Sort the following elements using bucket sort
0.78, 0.17, 0.39, 0.26, 0.72, 0.94, 0.21, 0.12, 0.23, 0.68
 Bucket sorting

Time complexity T(n) = θ(n)

 Linear search

 It is a simple search algorithm that search a target value

in an array or list.
 It sequentially scan the array, comparing each array item
with the search element.
 If search element is found in an array, then the index of
an element is returned.
 The time complexity of linear search is O(n).
 It can be applied to both sorted and unsorted array.
 Binary search

 Binary search is the most popular Search algorithm. It is

efficient and also one of the most commonly used
techniques that is used to solve problems.

 Binary search works only on a sorted set of elements. To

use binary search on a collection, the collection must first
be sorted.

 When binary search is used to perform operations on a

sorted set, the number of iterations can always be reduced
on the basis of the value that is being searched.
Binary search process
 Binary search
Binary-search(A, n, x)
l=1
r=n
while l ≤ r
do
m = ⌊(l + r) / 2⌋
if A[m] < x then
l=m+1
else if A[m] > x then
r=m−1
else
return m
return unsuccessful

Time complexity T(n) = O(lgn)

 Unit-I

Thank you.