Linear Search, Binary Search, Bubble Sort
Problem: Search
We are given a list of records.
Each record has an associated key.
Give efficient algorithm for searching for a
record containing a particular key.
Efficiency is quantified in terms of average
time analysis (number of comparisons) to
retrieve an item.
Search
[0] [1] [2] [3] [4] [ 700 ]
…
Number 281942902 Number 233667136 Number 580625685
Number 701466868 Number 506643548 Number 155778322
Each record in list has an associated key. Number 580625685
In this example, the keys are ID numbers.
Given a particular key, how can we efficiently
retrieve the record from the list?
Serial Search (Linear Search)
Step through array of records, one at a time.
Look for record with matching key.
Search stops when
record with matching key is found
or when search has examined all records without
success.
Worst Case Time for Serial Search
For an array of n elements, the worst
case time for serial search requires n
array accesses: O(n).
Consider cases where we must loop
over all n records:
desired record appears in the last position of
the array
desired record does not appear in the array at
all
Average Case for Serial Search
Assumptions:
1. All keys are equally likely in a search
2. We always search for a key that is in the array
Example:
We have an array of 10 records.
If search for the first record, then it
requires 1 array access; if the second,
then 2 array accesses. etc.
The average of all these searches is:
(1+2+3+4+5+6+7+8+9+10)/10 = 5.5
Average Case Time for Serial Search
Generalize for array size n.
Expression for average-case running time:
(1+2+…+n)/n = n(n+1)/2n = (n+1)/2
Therefore, average case time complexity for
serial search is O(n).
Implementation
Linear Search ( Array A, Value x)
Step 1: Set i to 1
Step 2: if i > n then go to step 7
Step 3: if A[i] = x then go to step 6
Step 4: Set i to i + 1
Step 5: Go to Step 2
Step 6: Print Element x Found at index i and go
to step 8
Step 7: Print element not found
Step 8: Exit
8
Binary Search
Perhaps we can do better than O(n) in the
average case?
Assume that we are give an array of records
that is sorted. For instance:
an array of records with integer keys sorted from
smallest to largest (e.g., ID numbers), or
an array of records with string keys sorted in
alphabetical order (e.g., names).
Binary Search
Example: sorted array of integer keys. Target=7.
[0] [1] [2] [3] [4] [5] [6]
3 6 7 11 32 33 53
Binary Search
Example: sorted array of integer keys. Target=7.
[0] [1] [2] [3] [4] [5] [6]
3 6 7 11 32 33 53
Find approximate midpoint
Binary Search
Example: sorted array of integer keys. Target=7.
[0] [1] [2] [3] [4] [5] [6]
3 6 7 11 32 33 53
Is 7 = midpoint key? NO.
Binary Search
Example: sorted array of integer keys. Target=7.
[0] [1] [2] [3] [4] [5] [6]
3 6 7 11 32 33 53
Is 7 < midpoint key? YES.
Binary Search
Example: sorted array of integer keys. Target=7.
[0] [1] [2] [3] [4] [5] [6]
3 6 7 11 32 33 53
Search for the target in the area before midpoint.
Binary Search
Example: sorted array of integer keys. Target=7.
[0] [1] [2] [3] [4] [5] [6]
3 6 7 11 32 33 53
Find approximate midpoint
Binary Search
Example: sorted array of integer keys. Target=7.
[0] [1] [2] [3] [4] [5] [6]
3 6 7 11 32 33 53
Target = key of midpoint? NO.
Binary Search
Example: sorted array of integer keys. Target=7.
[0] [1] [2] [3] [4] [5] [6]
3 6 7 11 32 33 53
Target < key of midpoint? NO.
Binary Search
Example: sorted array of integer keys. Target=7.
[0] [1] [2] [3] [4] [5] [6]
3 6 7 11 32 33 53
Target > key of midpoint? YES.
Binary Search
Example: sorted array of integer keys. Target=7.
[0] [1] [2] [3] [4] [5] [6]
3 6 7 11 32 33 53
Search for the target in the area after midpoint.
Binary Search
Example: sorted array of integer keys. Target=7.
[0] [1] [2] [3] [4] [5] [6]
3 6 7 11 32 33 53
Find approximate midpoint.
Is target = midpoint key? YES.
Binary Search Implementation
Step 1: Start
Step 2: Input Sorted array in "a[]" and element to be searched in "x" and size
of array in "size"
Step 3: Initialize low=0, high=size-1
Step 4: Repeat until low>=high
Step 4.1: mid=(low+high)/2
Step 4.2: If a[mid] is equal to x,
then, print index value of mid and Goto step 6
Else
If a[mid]<x
low=mid+1
else
high=mid-1
Step 5: Print x not found in the list
Stop 6: Stop
Binary Search: Analysis
Worst case complexity?
Each level in the recursion, we split the array
in half (divide by two).
Therefore maximum recursion depth is
floor(log2n) and worst case = O(log2n).
Average case is also = O(log2n).
Analysis of input size at each iteration of Binary Search:
At Iteration 1:
Length of array = n
At Iteration 2:
Length of array = n/2
At Iteration 3:
Length of array = (n/2)/2 = n/22
Therefore, after Iteration k:
Length of array = n/2k
Also, we know that after After k iterations, the length of the array becomes 1 Therefore, the
Length of the array
n/2k = 1
=> n = 2k
Applying log function on both sides:
=> log n = log 2k
2 2
=> log2n = k * log22
As (loga (a) = 1) Therefore, k = log2(n)
23
Why Sorting?
Practical application
People by last name
Countries by population
Search engine results by relevance
Fundamental to other algorithms
24
Problem statement
There are n comparable elements in an array and we
want to rearrange them to be in increasing order
Pre:
An array A of data records
A value in each data record
A comparison function
<, =, >, compareTo
Post:
For each distinct position i and j of A, if i<j then A[i]
A[j]
A has all the same data it started with
25
Bubble sort
bubble sort: orders a list of values by
repetitively comparing neighboring elements
and swapping their positions if necessary
more specifically:
scan the list, exchanging adjacent elements if
they are not in relative order; this bubbles the
highest value to the top
scan the list again, bubbling up the second highest
value
repeat until all elements have been placed in their
proper order
26
"Bubbling" largest element
Traverse a collection of elements
Move from the front to the end
"Bubble" the largest value to the end using pair-
wise comparisons and swapping
0 1 2 3 4 5
42Swap77 12 101
77 42 35 5
27
"Bubbling" largest element
Traverse a collection of elements
Move from the front to the end
"Bubble" the largest value to the end using pair-
wise comparisons and swapping
0 1 2 3 4 5
42 35Swap35
77 77 12 101 5
28
"Bubbling" largest element
Traverse a collection of elements
Move from the front to the end
"Bubble" the largest value to the end using pair-
wise comparisons and swapping
0 1 2 3 4 5
42 35 12Swap12
77 77 101 5
29
"Bubbling" largest element
Traverse a collection of elements
Move from the front to the end
"Bubble" the largest value to the end using pair-
wise comparisons and swapping
0 1 2 3 4 5
42 35 12 77 101 5
No need to swap
30
"Bubbling" largest element
Traverse a collection of elements
Move from the front to the end
"Bubble" the largest value to the end using pair-
wise comparisons and swapping
0 1 2 3 4 5
42 35 12 77 5 Swap101
101 5
31
"Bubbling" largest element
Traverse a collection of elements
Move from the front to the end
"Bubble" the largest value to the end using pair-
wise comparisons and swapping
0 1 2 3 4 5
42 35 12 77 5 101
Largest value correctly placed
32
Bubble Sort
1. algorithm Bubble_Sort(list)
2. Pre: list != fi
3. Post: list is sorted in ascending order for
all values
4. for i <- 0 to list:Count - 1
5. for j <- 0 to list:Count - 1
6. if list[i] < list[j]
7. Swap(list[i]; list[j])
8. end if
9. end for
10. end for
11. return list
12. end Bubble_Sort
Another Example
Implementation
void bubbleSort(int[] a) {
for (int i = 0; i < [Link]; i++) {
for (int j = 1; j < [Link] - i; j++) {
// swap adjacent out-of-order elements
if (a[j-1] > a[j]) {
swap(a, j-1, j);
}
}
}
}
35
Bubble sort runtime
Running time (# comparisons) for input size n:
n 1 n 1 i n1
1 (n 1 i)
i0 j 1 i0
n1 n1 n1
n 1 1 i
i0 i0 i0
2 (n 1)n
n n
2
(n 2 )
number of actual swaps performed depends on the
data; out-of-order data performs many swaps
36
Selection sort
selection sort: orders a list of values by
repetitively putting a particular value into its
final position
more specifically:
find the smallest value in the list
switch it with the value in the first position
find the next smallest value in the list
switch it with the value in the second position
repeat until all values are in their proper places
37
Selection sort example
38
Selection sort example 2
Index
0 1 2 3 4 5 6 7
Value
27 63 1 72 64 58 14 9
1st pass
1 63 27 72 64 58 14 9
2nd pass
1 9 27 72 64 58 14 63
3rd pass
1 9 14 72 64 58 27 63
39
Selection sort code
void selectionSort(int[] a) {
for (int i = 0; i < [Link]; i++) {
// find index of smallest element
int minIndex = i;
for (int j = i + 1; j < [Link]; j++) {
if (a[j] < a[minIndex]) {
minIndex = j;
}
}
// swap smallest element with a[i]
swap(a, i, minIndex);
}
}
40
Selection sort runtime
Running time for input size n:
n1 n1 n1
1 (n 1 (i 1) 1)
i0 j i1 i0
n1
(n i 1)
i0
n1 n1 n1
n 1 i 1
i0 i0 i0
2 (n 1)n
n n
2
(n 2 )
41
Insertion sort
insertion sort: orders a list of values by
repetitively inserting a particular value into a
sorted subset of the list
more specifically:
consider the first item to be a sorted sublist of
length 1
insert the second item into the sorted sublist,
shifting the first item if needed
insert the third item into the sorted sublist, shifting
the other items as needed
repeat until all values have been inserted into their
proper positions
42
Insertion Sort
Idea: like sorting a hand of playing cards
Start with an empty left hand and the cards facing
down on the table.
Remove one card at a time from the table, and
insert it into the correct position in the left hand
compare it with each of the cards already in the hand,
from right to left
The cards held in the left hand are sorted
these cards were originally the top cards of the pile on
the table
43
Insertion Sort
To insert 12, we need to
make room for it by moving
first 36 and then 24.
6 10 24 36
12
44
Insertion Sort
6 10 24 36
12
45
Insertion Sort
6 10 24 3
6
12
46
Insertion Sort example 1
input array
5 2 4 6 1
3
at each iteration, the array is divided in two sub-arrays:
left sub-array right sub-array
sorted unsorted
47
Insertion Sort example 1 (Cont.)
48
INSERTION-SORT
Alg.: INSERTION-SORT(A) 1 2 3 4 5 6 7 8
for j ← 2 to n a1 a2 a3 a4 a5 a6 a7 a8
do key ← A[ j ]
key
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
Insertion sort – sorts the elements in place
49
Insertion sort example 2
Simple sorting algorithm.
n-1 passes over the array
At the end of pass i, the elements that occupied
A[0]…A[i] originally are still in those spots and in
sorted order.
2 15 8 1 17 10 12 5
0 1 2 3 4 5 6 7
after
2 8 15 1 17 10 12 5
pass 2
0 1 2 3 4 5 6 7
after
1 2 8 15 17 10 12 5
pass 3
0 1 2 3 4 5 6 7
50
Insertion sort example 3
51
Insertion sort code
void insertionSort(int[] a) {
for (int i = 1; i < [Link]; i++) {
int temp = a[i];
// slide elements down to make room for a[i]
int j = i;
while (j > 0 && a[j - 1] > temp) {
a[j] = a[j - 1];
j--;
}
a[j] = temp;
}
}
52
Insertion sort runtime
worst case: reverse-ordered elements in array.
n1
(n 1)n
i 1 2 3 ... (n 1) 2
i1
(n 2 )
best case: array is in sorted ascending order.
n1
1 n 1 (n)
i1
average case: each element is about halfway in
order.
n1
i 1 (n 1)n
2 2 (1 2 3... (n 1)) 4
i1
(n 2 )
53
Comparing sorts
We've seen "simple" sorting algorithms so far,
such as selection sort and insertion sort.
They all use nested loops and perform
approximately n2 comparisons
They are relatively inefficient
54
Insertion Sort - Summary
Advantages
Good running time for “almost sorted” arrays (n)
Disadvantages
(n2) running time in worst and average case
n2/2 comparisons and exchanges
55
