DATA STRUCTURES AND

ALGORITHMS

Week 15: Sorting Algorithms

Sohail Muhammad
Department of CS
Bahria University, Islamabad

1
Introduction To Searching
• Linear Search
• Locate an item by a sequence of comparisons
• For example:
• Compare each element of array with key value
• Useful for small and unsorted arrays
• Binary Search
• Item being sought – repeatedly compared with items in the list
• For example:
• Can only be used on sorted arrays
• Compares middle element with key
• If equal, match found
• If key < middle, repeat search through the first half of the array
• If key > middle, repeat search through the last half of the array
• Very fast; at most n steps, where 2^n > # of elements
• 30 element array takes at most 5 steps
• 2^5 > 30

• Fast Searching
• Location of an item is determined directly as a function of the item itself
• No hit and trial comparisons
• Ideally – Time required to locate an item is constant and does not depend on the
number of items stored
• Hash tables and hash functions
2
Searching Time Complexities

3
 Sorting Algorithms
• Sorting refers to arranging data in a particular format
• Sorting algorithm specifies the way to arrange data in a particular
order
• Most common orders are numerical or lexicographical order.
• Data searching can be optimized to a very high level if data is stored in
a sorted manner
• Examples
• Telephone Directory
• Dictionary
• ….

4
Common Sorting Algorithms

• Selection Sort
• Exchange Sorts
• Bubble Sort
• Quick Sort
• Insertion Sorts
• Simple Insertion Sort
• Shell Sort
• Merge sort
• Heap sort
• Radix sort
5
Selection Sort

6
SELECTION SORT

• Key Idea:

• Find the minimum value in the list

• Swap it with the value in the first position
• Repeat the steps above for remainder of the list (starting at
the next position)

8
Example: Selection Sort
26 33 43 100 46 88 52 17 53 77

17 | 33 43 100 46 88 52 26 53 77

17 26 | 43 100 46 88 52 33 53 77

17 26 33 | 100 46 88 52 43 53 77

17 26 33 43 | 46 88 52 100 53 77

17 26 33 43 46 | 88 52 100 53 77

17 26 33 43 46 52 | 88 100 53 77

17 26 33 43 46 52 53 | 100 88 77

17 26 33 43 46 52 53 77 | 88 100

17 26 33 43 46 52 53 77 88 | 100

17 26 33 43 46 52 53 77 88 100 | 11
SELECTION SORT

13
Exchange Sort

14
 Exchange sorts

• Unlike selection sorts in which some element is selected and then moved
to its correct position in the list, exchange sorts systematically
interchange pairs of elements that are out of order until eventually no
such pairs remain and the list is therefore sorted.

• One common example of an exchange sort is Bubble sort.

15
Bubble sort

• Bubble sort is similar to selection sort in the sense that it repeatedly

finds the largest/smallest value in the unprocessed portion of the
array and puts it back

• However, finding the largest value is not done by selection

• We "bubble" up the largest value instead.

16
BUBBLE SORT

• Compares adjacent items and exchanges them if they

are out of order

• Comprises of several passes

• In one pass, the largest value has been “bubbled” to its

proper position

• In second pass, the last value does not need to be

compared
17
Bubble Sort : example
1 2 3 4 5 6

77 42 35 12 101 5

1 2 3 4 5 6

42 Swap 42
77 77 35 12 101 5

1 2 3 4 5 6

42 7735 Swap 35
77 12 101 5

22
Bubble Sort : example
1 2 3 4 5 6

42 35 12 Swap 12
77 77 101 5

1 2 3 4 5 6

42 35 12 77 101 5

No need to swap
1 2 3 4 5 6

42 35 12 77 5 Swap 101
101 5

23
Bubble Sort : example 2

1 2 3 4 5 6

42 35 12 77 5 101

Largest value correctly placed

• Notice that only the largest value is correctly

placed
• All other values are still out of order
• So we need to repeat this process

24
Repeat “Bubble Up” How Many
Times?
• If we have N elements…

• And if each time we bubble an element, we place it

in its correct location…

• Then we repeat the “bubble up” process N – 1 times.

• This guarantees we’ll correctly place all N elements.

25
“Bubbling” All the Elements
1 2 3 4 5 6
42 35 12 77 5 101

1 2 3 4 5 6
35 12 42 5 77 101

1 2 3 4 5 6
N-1

12 35 5 42 77 101

1 2 3 4 5 6
12 5 35 42 77 101

1 2 3 4 5 6
5 12 35 42 77 101

26
Bubble sort - implementation

for( int i=0; i<n-1; i++)

{
for(int j=0; j<n-1; j++)
if(x[j] > x[j+1])
{
temp = x[j];
x[j] = x[j+1];
x[j+1] = temp;
}
}

27
Reducing the Number of
Comparisons
1 2 3 4 5 6
77 42 35 12 101 5
1 2 3 4 5 6
42 35 12 77 5 101

1 2 3 4 5 6
35 12 42 5 77 101

1 2 3 4 5 6
12 35 5 42 77 101

1 2 3 4 5 6
12 5 35 42 77 101

28
Bubble sort - implementation

for( int i=0; i<n-1; i++)

{
for(int j=0; j < n-i-1; j++)
if(x[j] > x[j+1])
{
temp = x[j];
x[j] = x[j+1];
x[j+1] = temp;
}
}

29
Already Sorted Collections?

• What if the collection was already sorted?

• What if only a few elements were out of place and after a couple of
“bubble ups,” the collection was sorted?

• We want to be able to detect this and “stop early”!

1 2 3 4 5 6
5 12 35 42 77 101

30
Using a Boolean “Flag”

• We can use a boolean variable to determine if any

swapping occurred during the “bubble up.”

• If no swapping occurred, then we know that the

collection is already sorted!

• This boolean “flag” needs to be reset after each

“bubble up.”

31
 Bubble sort - implementation
void bubbleSort (int S[ ], int length)
{
int hold, j, pass;
bool swapped = TRUE;
for(pass=0; pass<length-1 && swapped==TRUE; pass++)
{
swapped = FALSE;
for(j = 0; j<length-pass-1; j++) {
if(S[j] > S[j+1]){
swapped=TRUE;
hold = S[j];
S[j] = S[j+1];
S[j+1] = hold;
}
}
}
}

32
Improved BubbleSort

1 2 3 4 5 6
77 12 42 33 101 5

33
Insertion Sorts

34
Simple insertion sort

• Insert a new element into a list of already sorted elements so that the
resulting list is still sorted

• We divide our array in a sorted and an unsorted array

• Initially the sorted portion contains only one element – the first
element in the array.

• We take the second element in the array, and put it into its correct
place

35
 Simple insertion sort

67 33 21 84 49 50 75

36
Simple insertion sort

36 36 24 10 6 6
24 24 36 24 10 10
10 10 10 36 24 12
6 6 6 6 36 24
12 12 12 12 12 36

37
 Simple insertion sort

67 33 21 84 49 50 75

67 33 21 84 49 50 75 Initial sorted sublist of one element

33 67 21 84 49 50 75 Insert 33 to get 2-element sorted sublist

21 33 67 84 49 50 75 Insert 21 to get 3-element sorted sublist

21 33 67 84 49 50 75 Insert 84 to get 4-element sorted sublist

21 33 49 67 84 50 75 Insert 49 to get 5-element sorted sublist

21 33 49 50 67 84 75 Insert 50 to get 6-element sorted sublist

21 33 49 50 67 75 84 Insert 75 to get 7-element sorted sublist

38
 Simple insertion sort

for( k = 1; k < n; k++)

{
temp = x[k];
for( i = k-1; i >= 0 && temp < x[i]; i--)
x[i+1] = x[i];
x[i+1] = temp;
}

40
Execution times Insertion Sort

• Time as function of list size

• More to come – Algorithmic
complexity analysis
• Insertion sort a good choice for lists of
few thousand elements

Bubble Sort Selection Sort

41
 Insertion sort – shell sort

• Shell Sort (Diminishing Increment Sort)

• Sorts separate subfiles of the original file
• Subfiles contain every kth element of the original file

K = 5 , 5 subfiles:

Subfile1 x[0] x[5] x[10] .

Subfile2 x[1] x[6] x[11] . .
Subfile3 x[2] x[7] x[12] . .
Subfile4 x[3] x[8] x[13] . .
Subfile5 x[4] x[9] x[14] . .

42
Shell (Insertion) sort

• Sort the k subfiles

• Chose a smaller value of k and again partition the file into a new set
of subfiles
• Sort each of the larger files
• Repeat the process, each time with a smaller value of k, eventually
selecting k = 1, and thus sorting the entire file
• A decreasing sequence of increments is fixed at the start of the entire
process and the last value in this sequence must be 1

43
 Shell sort

Example:
25 57 48 37 12 92 86 33

Let the sequence be k = (5,3,1).

First Iteration ( k = 5) Second Iteration (k = 3) Third Iteration (k = 1)

(x[0], x[5]) (x[0],x[3],x[6])
(x[1], x[6]) (x[0],x[1],x[2],x[3],x[4],x[5],x[6],x[
(x[1],x[4],x[7])
(x[2], x[7]) 7])
(x[2],x[5])
(x[3])
(x[4])

44
 Shell sort
Let the sequence be k = (5,3,1).

Example:
25 57 48 37 12 92 86 33
Pass 1:
(x[0], x[5]) (25, 92) (25, 92)
(x[1], x[6]) (57, 86) (57, 86)
(x[2], x[7]) (48, 33) (33, 48) 25, 57, 33, 37, 12, 92, 86, 48
(x[3]) (37) (37)
(x[4]) (12) (12)

Pass 2:
(x[0],x[3],x[6]) (25, 37, 86) (25, 37, 86)
(x[1],x[4],x[7]) (57, 12, 48) (12, 48, 57) 25, 12, 33, 37, 48, 92, 86, 57
(x[2],x[5]) (33, 92) (33, 92)
Pass 3:
(x[0],x[1],x[2],x[3],x[4],x[5],x[6],x[7]) 25, 12, 33, 37, 48, 92, 86, 57
45
Sorted Array: 12, 25, 33, 37, 48, 57, 86, 92
 Shell sort
void shellsort(int x[], int n, int incrmnts[], int numinc)
{
int span, y;

for(int incr = 0; incr < numincr; incr++)

{
/*Span is the size of increment*/
span = incrmnts[incr];

for(int j = span, j < n; j++)

{
/*Insert x[j] at its proper location within its subfile
using simple insert sort*/
y = x[j];
for(int k = j – span;k>=0 && y<x[k]; k-=span)
x[k+span] = x[k];
x[k+span] = y;
}
}
} 48
Merge Sort

49
Divide and Conquer

• Divide and Conquer cuts the problem in half each

time, but uses the result of both halves:
• Divide the problem in half until the problem is trivial
• Solve for both halves
• Combine the solutions

50
Merge sort

51
 MERGE SORT
Merging Lists:

Combining two lists that have previously been sorted.

File 1:
15 20 25 35 45 60 65 70
File 2:
10 30 40 50 55

52
 MERGE SORT
Merging Lists:

Combining two lists that have previously been sorted.

File 1:
15 20 25 35 45 60 65 70
File 2:
10 30 40 50 55

Objective:
To merge File1 and File2 to produce sorted File3.

File 3:
10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70
53
 MERGE SORT

File 1: 15 20 25 35 45 60 65 70
File 2: 10 30 40 50 55

Read one element from each file, say x from File1 and y from File2.

File 1: 15 20 25 35 45 60 65 70
x
File 2: 10 30 40 50 55
y
Compare the two and write the smaller, in this case y, to File3.

File 3: 10
Read another value y from File2. 54
 File 1: 15 20 25 35 45 60 65 70
x
File 2: 10 30 40 50 55
y

Now x is smaller than y, so it is written to File3, and a new value of x

is read from File1:

File 1: 15 20 25 35 45 60 65 70
x
File 2: 10 30 40 50 55
y

File 3: 10 15
Again x is less than y, so it is written to File3 and a new x is read from55File1.
 File 1: 15 20 25 35 45 60 65 70
x
File 2: 10 30 40 50 55
y

File 3: 10 15 20

Continuing in this manner….

File 1: 15 20 25 35 45 60 65 70
x
File 2: 10 30 40 50 55 Since y < x, write y to File3.
y

File 3: 10 15 20 25 30 35 40 45 50
55 56
 MERGE SORT

Because the end of File2 has been reached, simply copy the remaining
items in File1 to File3.

File 3: 10 15 20 25 30 35 40 45 50 55 60 65 70

57
MERGE SORT

Algorithm:

/* Merge sorted files File1 and File2 ; giving File3. */

1. Open File1 and File2 for input, and File3 for output.
2. Read the first element x from File1 and first element y from File2.
3. Repeat the following until the end of either File1 or File2 is reached:
If x < y, then
a. Write x to File3.
b. Read a new x value from File1.
Otherwise:
a. Write y to File3.
b. Read a new y value from File2.
4. If the end of File1 was encountered, copy any remaining items
from File2 into File3. If the end of File2 was encountered, copy the
rest of File1 into File3.

58
 MERGE SORT

Sorting a single File:

Divide the file into n subfiles of size 1.
Merge adjacent pairs of files – giving n/2 files of size 2.
Repeat this process until there is only one file remaining of size n.

59
98 23 45 14 6 67 33 42

6 14 23 33 42 45 67 98

100
 98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

98 23 45 14 6 67 33 42

23 98 14 45 6 67 33 42

14 23 45 98 6 33 42 67

6 14 23 33 42 45 67 98

101
 Algorithm
Mergesort(Passed an array)
if array size > 1
Divide array in half
Call Mergesort on first half.
Call Mergesort on second half.
Merge two halves.

Merge(Passed two arrays)

Compare leading element in each array
Select lower and place in new array.
(If one input array is empty then place
remainder of other array in output array)

102
Merge Sort
void merge(int*,int*,int,int,int);
void mergesort(int *a, int*b, int low, int high) {
int pivot;
if(low<high) {
pivot=(low+high)/2;
mergesort(a,b,low,pivot);
mergesort(a,b,pivot+1,high);
merge(a,b,low,pivot,high);
}
}
int main() {
const int num = 8
int a[] = {12,10,43,23,78,45,56,98};
int b[num];
mergesort(a,b,0,num-1);
for(int i=0; i<num; i++)
cout<<a[i]<<" "; cout<<endl;
103
}
void merge(int *a, int *b, int low, int pivot, int high)
{ 14 23 45 98 6 33 42 67
int h,i,j,k;
h=low;
i=low; low =0 pivot=3 high = 7
j=pivot+1;

while((h<=pivot)&&(j<=high))
{
14 23 45 98 6 33 42 67
if(a[h]<=a[j])
{
b[i]=a[h];
h=0 j=4 high = 7
h++;
} i=0
else
{
b[i]=a[j];
j++;
} ‘h’ should not go beyond
i++; ‘pivot’ and ‘j’ should not go
} beyond ‘high’

109
QUICK SORT

114
Quick sort
• An exchange sort scheme – efficient than bubble sort
• Uses divide-n-conquer strategy
• A recursive approach of problem solving in which the
original problem is partitioned into simpler sub-problems,
each of which can be considered independently

• With quick sort the divide-n-conquer approach chooses

some element called a pivot and then performs a
sequence of exchanges so that all elements that are less
than this pivot are to its left and all elements that are
greater than the pivot are to its right

• This correctly positions the pivot and divides the list into
two smaller sublists, each of which may then be sorted in
the same way.

• Repeating this procedure will eventually produce smaller

lists that can easily be sorted. 115
 QUICK SORT
Algorithm:

1. If the list has 0 or 1 elements, return.

Else do the following:
2. Pick an element in the list to use as pivot.
3. Split the remaining elements into two disjoint groups:
Smaller Than Pivot = { all elements < pivot }
Larger Than Pivot = { all elements > pivot }
4. Return the list rearranged as:
QuickSort( Smaller Than Pivot), pivot, QuickSort( Larger Than
Pivot).

116
 QUICK SORT

75 70 65 84 98 78 100 93 55 61 81 68

117
 QUICK SORT

75 70 65 84 98 78 100 93 55 61 81 68

For simplicity let the first number be the pivot;

pivot = 75.

Smaller than pivot consists of:

70 65 55 61 and 68

Larger than pivot consists of:

84 98 78 100 93 and 81
Rearrange the list such that:
Smaller than pivot 75 Larger than pivot

118
 QUICK SORT
The Split Operation:

Carry out two searches:

One from the right end of the list for elements less than or
equal to the pivot 75.
The other from the left end for elements greater than 75.

right
pivot

75 70 65 84 98 78 100 93 55 61 81 68

left

120
75 70 65 84 98 78 100 93 55 61 81
68

121
 QUICK SORT
void Split (int x[], int first, int last, int & pos)
{
int pivot = x[first];
int left = first;
int right = last;
while (left < right)
{
while( x[right] > pivot)
right --;
while (x[left] <= pivot && left < right)
left++;
If (left < right)
Interchnage (x[left], x[right];
}
125
 QUICK SORT

x[first] = x[right];

x[right] = pivot;
}
pos = right;

126
 QUICK SORT
void QuickSort (int x[], int first, int last)
{
int pos;
if ( first < last)//List has more than one element.
{
Split (x, first, last, pos); // Split into two sublists.
QuickSort (x, first, pos – 1); // Sort the left sublist.
QuickSort (x, pos + 1, last); // Sort the right sublist.
}
// else list has 0 or 1 element and requires no sorting

127
 Useful Links
http://www.csanimated.com/animation.php?t=Quicksort
http://www.hakansozer.com/category/c/
http://www.nczonline.net/blog/2012/11/27/computer-science-in-
javascript-quicksort/
http://www.sorting-algorithms.com/shell-sort

129