CSE-243: Algorithm Design and Analysis

 Lecture 10-12: Sorting Algorithms and Into. to Divide & Conquer

Instructor Teaching Assistants
Md. Sabir Hossain Rakibul Islam
Lecturer Id: 1604060
Dept. of CSE, CUET Email: [email protected]
Mobile: 01737143868/01882826575 Saiful Islam Rimon
Email: [email protected] Id: 1604061
Email : [email protected]

Acknowledgment: Most of the content of this slide is adopted from the book: Fundamentals of Computer Algorithms by Ellis
Horowitz, Sanguthevar Rajasekaran, Sartaj Sahni, Introduction to Algorithms by Thomas H. Cormen, Charles E. Leiserson,
Ronald L. Rivest, Clifford Stein.
 Recap
• Definitions
– Internal, External, In-Place Sort, Stability
• Classifications
– Comparisons Sorts
• Insertion sort
• Selection sort
• Bubble sort
• Merge sort
• Quicksort
• Heapsort
• Shellsort

– Non-Comparisons Sorts
• Counting sort (indexes using key values)
• Radix sort (examines individual bits of keys)
• Bucket sort (examines bits of keys)

2
 Today’s Target
• Divide and Conquer Algorithm Basic
• Merge Sort Algorithm
– Demo
– Algorithm
– Complexity Analysis
• Substitution Method
• Master Method
• Recurrence Tree Method
• Recurrence Tree Method More Examples
– Relationship between Master and Rec. Tree Method

3
 Divide-and-Conquer
• Divide the problem into a number of sub-problems
– Similar sub-problems of smaller size

• Conquer the sub-problems

– Solve the sub-problems recursively

– Sub-problem size small enough  solve the problems in

straightforward manner

• Combine the solutions of the sub-problems

– Obtain the solution for the original problem

4
 Merge Sort Approach
• To sort an array A[p . . r]:
• Divide
– Divide the n-element sequence to be sorted into two
subsequences of n/2 elements each
• Conquer
– Sort the subsequences recursively using merge sort
– When the size of the sequences is 1 there is nothing
more to do
• Combine
– Merge the two sorted subsequences

5
Merge Sort Demo

6
Merge sort

MERGE-SORT A[1 . . n]
1.If n = 1, done.
2.Recursively sort A[ 1 . . ⎡ n/2 ⎤ ]
and A[ ⎡ n/2 ⎤ +1 . . n ] .
3.“Merge” the 2 sorted lists

Key subroutine: MERGE

 Merging two sorted arrays
20 12
13 11
7 9
2 1
 Merging two sorted arrays
20 12
13 11
7 9
2 1

1
 Merging two sorted arrays
20 12 20 12
13 11 13 11
7 9 7 9
2 1 2

1
 Merging two sorted arrays
20 12 20 12
13 11 13 11
7 9 7 9
2 1 2

1 2
 Merging two sorted arrays
20 12 20 12 20 12
13 11 13 11 13 11
7 9 7 9 7 9
2 1 2

1 2
 Merging two sorted arrays
20 12 20 12 20 12
13 11 13 11 13 11
7 9 7 9 7 9
2 1 2

1 2 7
 Merging two sorted arrays
20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2

1 2 7
 Merging two sorted arrays
20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2

1 2 7 9
 Merging two sorted arrays
20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2

1 2 7 9
 Merging two sorted arrays
20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2

1 2 7 9 11
 Merging two sorted arrays
20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2

1 2 7 9 11
 Merging two sorted arrays
20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2

1 2 7 9 11 12
 Merging two sorted arrays
20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2

1 2 7 9 11 12

Time = (n) to merge a total

of n elements (linear time).
 Implementation: merge_sort

https://www.hackerearth.com/practice/algorithms/sorting/merge-sort/tutorial/
21
Implementation: merge

22
23
Analyzing merge sort

T(n) MERGE-SORT A[1 . . n]

(1) 1.If n = 1, done.
2T(n/2) 2.Recursively sort A[ 1 . . ⎡ n/2 ⎤ ]
and A[ ⎡ n/2 ⎤ +1 . . n ] .
(n) 3.“Merge” the 2 sorted lists
 Recurrence for merge sort
(1) if n = 1;
T(n) =
2T(n/2) + (n) if n > 1.
• We shall usually omit stating the base
case when T(n) = (1) for sufficiently
small n, but only when it has no effect on
the asymptotic solution to the recurrence.
 Substitution Method
Any difference best
T(1) = b / worse case?
T(n) = 2T(n/2) + cn for n>1

26
 Substitution Method
Any difference best
T(1) = b / worse case?
T(n) = 2T(n/2) + cn for n>1

T(n) = 2T(n/2)+cn
T(n) = 4T(n/4) +cn +cn substitute
T(n) = 8T(n/8)+cn+cn+cn substitute
T(n) = 2kT(n/2k)+kcn inductive leap
T(n) = nT(1) + cn log n where k = log n select value for k
T(n) = (n log n) simplify
27
 Master Method
(1) if n = 1;
T(n) =
2T(n/2) + (n) if n > 1.
 Master Method

(1) if n = 1;
T(n) =
2T(n/2) + (n) if n > 1.

•T(n) = 2T(n/2) + Θ(n).

•It falls in case 2 as c is 1 and Logba is also 1.
•So the solution is Θ(n Logn)
 Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
 Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
T(n)
 Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
T(n/2) T(n/2)
 Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/2 cn/2

T(n/4) T(n/4) T(n/4) T(n/4)

 Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

(1)
 Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/2 cn/2
h = lg n cn/4 cn/4 cn/4 cn/4

(1)
 Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn
cn/2 cn/2
h = lg n cn/4 cn/4 cn/4 cn/4

(1)
 Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn
cn/2 cn/2 cn
h = lg n cn/4 cn/4 cn/4 cn/4

(1)
 Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn
cn/2 cn/2 cn
h = lg n cn/4 cn/4 cn/4 cn/4 cn

…
(1)
 Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn
cn/2 cn/2 cn
h = lg n cn/4 cn/4 cn/4 cn/4 cn

…
(1) #leaves = n (n)
 Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn cn
cn/2 cn/2 cn
h = lg n cn/4 cn/4 cn/4 cn/4 cn

…
(1) #leaves = n (n)
Total  (n lg n)
)
Equal amount of work done at each level
 Tree for different recurrence
Solve T(n) = 2T(n/2) + c, where c > 0 is constant.
c c
c c 2c 4c
h = 1 + lg n
c c c c

…
n/2 c
(1) #leaves
#leaves == nn (n)
Note that 1 + ½ + ¼ + … < 2 Total  (n)
All the work done at the leaves
 Tree for yet another recurrence
Solve T(n) = 2T(n/2) + cn2, c > 0 is constant.
cn2 cn2

h = 1 + lg n cn2/4 cn2/4 cn2/2

cn2/16 cn2/16 cn2/16 cn2/16 cn2/4

…
(1) #leaves
#leaves ==nn (n)
Note that 1 + ½ + ¼ + … < 2 Total  (n2)
All the work done at the root
 References
• Introduction to Algorithms, Third Edition
(International Edition), Thomas H. Cormen,
Charles E. Leiserson, Ronald L. Rivest, Clifford
Stein, ISBN-13: 978-0262033848. Page: 41-50
• Fundamentals of Computer Algorithms,
Second Edition, Ellis Horowitz, Sanguthevar
Rajasekaran, Sartaj Sahni, ISBN 10:
8173716129 / ISBN 13: 9788173716126,
Published by Universities Press/Orient
BlackSwan, 2008. Page: 29-49
 References
• http://www.geeksforgeeks.org/sorting-algorithms/
• https://visualgo.net/en/sorting
• http://sorting.at/
• https://www.toptal.com/developers/sorting-
algorithms
• http://www.geeksforgeeks.org/quick-sort/

45
• Allah Hafez