Insertion Sort

Sorting in Place (array)

1 2 3 i n

0 2 4 6 8 10 15 5 3 20 7

• Insertion Sort with an array sorted

x
• Selection Sort with an array 1 2 3 i n

• Bubblesort with an array 0 2 4 6 8 10 15 5 3 20 7

sorted

5 x
1 2 3 i+1 n
0 2 4 5 6 8 10 15 3 20 7

sorted

Complexity
Insertion Sort
for (i=2; i=n, i++)
x ← A[i] - For n elements, we need to index through n-1
A[0] ← x entries
j ← i-1
while x.key < A[j].key - For each entry, we need to examine and shift up
A[j+1] ← A[j] to n-1 other entries
j ← j-1
A[j+1] ← x - Complexity => O(n2)
- Best case complexity is O(n) when elements are
already in order
1 2 3 i n

0 2 4 6 8 10 15 5 3 20 7 - Worst case complexity is O(n2) when elements

are in reverse order
sorted
x

1
 Selection Sort Selection Sort
(array) (array)
1 i n

2 3 5 6 15 8 12 13 20

Sorted
and ≤ the x
Finding the
others smallest
element
for (i=1; i=n-1, i++)
k ← i
x ← A[i]
for (j=i+1; j=n, j++)
if A[j].key < x.key
k ← j
x ← A[j]
A[k] ← A[i]
A[i] ← x

Complexity of Selection Sort

for (i=1; i=n-1, i++)
k ← i -For each i from 1 to n-1, there is one exchange
x ← A[i]
for (j=i+1; j=n, j++)
and (n-i) comparisons.
if A[j].key < x.key - In total, there are (n-1) exchanges and
k ← j
x ← A[j]
(n-1)+(n-2)+ … +2+1 = n(n-1)/2 comparisons.
A[k] ← A[i] - Complexity => O(n2).
A[i] ← x
- Worst case occurs if all the elements are
1 i n
reverse order.
2 3 5 6 15 8 12 13 20

2
 "Bubbling Up" the Largest
Element
Bubble Sort • Traverse a collection of elements
– Move from the front to the end
– “Bubble” the largest value to the end using pair-
wise comparisons and swapping

1 2 3 4 5 6

77 42 35 12 101 5

"Bubbling Up" the Largest "Bubbling Up" the Largest

Element Element
• Traverse a collection of elements • Traverse a collection of elements
– Move from the front to the end – Move from the front to the end
– “Bubble” the largest value to the end using – “Bubble” the largest value to the end using
pair-wise comparisons and swapping pair-wise comparisons and swapping
1 2 3 4 5 6 1 2 3 4 5 6
42Swap
77 77
42 35 12 101 5 42 35Swap35
77 77 12 101 5

3
 "Bubbling Up" the Largest "Bubbling Up" the Largest
Element Element
• Traverse a collection of elements • Traverse a collection of elements
– Move from the front to the end – Move from the front to the end
– “Bubble” the largest value to the end using – “Bubble” the largest value to the end using
pair-wise comparisons and swapping pair-wise comparisons and swapping
1 2 3 4 5 6 1 2 3 4 5 6

42 35 12Swap12
77 77 101 5 42 35 12 77 101 5

No need to swap

"Bubbling Up" the Largest "Bubbling Up" the Largest

Element Element
• Traverse a collection of elements • Traverse a collection of elements
– Move from the front to the end – Move from the front to the end
– “Bubble” the largest value to the end using – “Bubble” the largest value to the end using
pair-wise comparisons and swapping pair-wise comparisons and swapping
1 2 3 4 5 6 1 2 3 4 5 6

42 35 12 77 5 Swap101
101 5 42 35 12 77 5 101

Largest value correctly placed

4
 Repeat “Bubble Up” How Many Times?
Items of Interest
• If we have N elements…
• Notice that only the largest value is correctly
placed
• And if each time we bubble an element, we place
• All other values are still out of order it in its correct location…
• So we need to repeat this process
• Then we repeat the “bubble up” process N – 1
1 2 3 4 5 6 times.
42 35 12 77 5 101
• This guarantees we’ll correctly
Largest value correctly placed place all N elements.

Complexity of Bubblesort
“Bubbling” All the Elements (arrays)
1 2 3 4 5 6
42 35 12 77 5 101
Comparisons
1 2 3 4 5 6
n -1
35 12 42 5 77 101
C= (n-i) = n (n-1)/2 = O(n2)
1 2 3 4 5 6 i=1
N-1

12 35 5 42 77 101
1 2 3 4 5 6 Movements
12 5 35 42 77 101 Dmin = 0 (already in order)
1 2 3 4 5 6 Dmax = 3•C = O(n2) (in reverse order)
5 12 35 42 77 101

5
 Already Sorted Collections? Using a Boolean “Flag”
• What if the collection was already sorted?
• We can use a boolean variable to determine if
• What if only a few elements were out of place any swapping occurred during the “bubble
and after a couple of “bubble ups,” the up.”
collection was sorted?
• We want to be able to detect this • If no swapping occurred, then we know that
and “stop early”! the collection is already sorted!

1 2 3 4 5 6 • This boolean “flag” needs to be reset after each

5 12 35 42 77 101 “bubble up.”

An Animated Example An Animated Example

N 8 did_swap true N 8 did_swap false
to_do 7 to_do 7
index index 1

98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

6
 An Animated Example An Animated Example
N 8 did_swap false N 8 did_swap true
to_do 7 to_do 7
index 1 index 1

Swap Swap

98 23 45 14 6 67 33 42 23 98 45 14 6 67 33 42
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

An Animated Example An Animated Example

N 8 did_swap true N 8 did_swap true
to_do 7 to_do 7
index 2 index 2

Swap

23 98 45 14 6 67 33 42 23 98 45 14 6 67 33 42
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

7
 An Animated Example An Animated Example
N 8 did_swap true N 8 did_swap true
to_do 7 to_do 7
index 2 index 3

Swap

23 45 98 14 6 67 33 42 23 45 98 14 6 67 33 42
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

An Animated Example An Animated Example

N 8 did_swap true N 8 did_swap true
to_do 7 to_do 7
index 3 index 3

Swap Swap

23 45 98 14 6 67 33 42 23 45 14 98 6 67 33 42
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

8
 An Animated Example An Animated Example
N 8 did_swap true N 8 did_swap true
to_do 7 to_do 7
index 4 index 4

Swap

23 45 14 98 6 67 33 42 23 45 14 98 6 67 33 42
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

An Animated Example An Animated Example

N 8 did_swap true N 8 did_swap true
to_do 7 to_do 7
index 4 index 5

Swap

23 45 14 6 98 67 33 42 23 45 14 6 98 67 33 42
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

9
 An Animated Example An Animated Example
N 8 did_swap true N 8 did_swap true
to_do 7 to_do 7
index 5 index 5

Swap Swap

23 45 14 6 98 67 33 42 23 45 14 6 67 98 33 42
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

An Animated Example An Animated Example

N 8 did_swap true N 8 did_swap true
to_do 7 to_do 7
index 6 index 6

Swap

23 45 14 6 67 98 33 42 23 45 14 6 67 98 33 42
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

10
 An Animated Example An Animated Example
N 8 did_swap true N 8 did_swap true
to_do 7 to_do 7
index 6 index 7

Swap

23 45 14 6 67 33 98 42 23 45 14 6 67 33 98 42
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

An Animated Example An Animated Example

N 8 did_swap true N 8 did_swap true
to_do 7 to_do 7
index 7 index 7

Swap Swap

23 45 14 6 67 33 98 42 23 45 14 6 67 33 42 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

11
 After First Pass of Outer Loop The Second “Bubble Up”
N 8 did_swap true N 8 did_swap false
to_do 7 to_do 6
index 8 Finished first “Bubble Up” index 1

23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

The Second “Bubble Up” The Second “Bubble Up”

N 8 did_swap false N 8 did_swap false
to_do 6 to_do 6
index 1 index 2

No Swap

23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

12
 The Second “Bubble Up” The Second “Bubble Up”
N 8 did_swap false N 8 did_swap true
to_do 6 to_do 6
index 2 index 2

Swap Swap

23 45 14 6 67 33 42 98 23 14 45 6 67 33 42 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

The Second “Bubble Up” The Second “Bubble Up”

N 8 did_swap true N 8 did_swap true
to_do 6 to_do 6
index 3 index 3

Swap

23 14 45 6 67 33 42 98 23 14 45 6 67 33 42 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

13
 The Second “Bubble Up” The Second “Bubble Up”
N 8 did_swap true N 8 did_swap true
to_do 6 to_do 6
index 3 index 4

Swap

23 14 6 45 67 33 42 98 23 14 6 45 67 33 42 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

The Second “Bubble Up” The Second “Bubble Up”

N 8 did_swap true N 8 did_swap true
to_do 6 to_do 6
index 4 index 5

No Swap

23 14 6 45 67 33 42 98 23 14 6 45 67 33 42 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

14
 The Second “Bubble Up” The Second “Bubble Up”
N 8 did_swap true N 8 did_swap true
to_do 6 to_do 6
index 5 index 5

Swap Swap

23 14 6 45 67 33 42 98 23 14 6 45 33 67 42 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

The Second “Bubble Up” The Second “Bubble Up”

N 8 did_swap true N 8 did_swap true
to_do 6 to_do 6
index 6 index 6

Swap

23 14 6 45 33 67 42 98 23 14 6 45 33 67 42 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

15
 The Second “Bubble Up” After Second Pass of Outer Loop
N 8 did_swap true N 8 did_swap true
to_do 6 to_do 6
index 6 index 7 Finished second “Bubble Up”

Swap

23 14 6 45 33 42 67 98 23 14 6 45 33 42 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

The Third “Bubble Up” The Third “Bubble Up”

N 8 did_swap false N 8 did_swap false
to_do 5 to_do 5
index 1 index 1

Swap

23 14 6 45 33 42 67 98 23 14 6 45 33 42 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

16
 The Third “Bubble Up” The Third “Bubble Up”
N 8 did_swap true N 8 did_swap true
to_do 5 to_do 5
index 1 index 2

Swap

14 23 6 45 33 42 67 98 14 23 6 45 33 42 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

The Third “Bubble Up” The Third “Bubble Up”

N 8 did_swap true N 8 did_swap true
to_do 5 to_do 5
index 2 index 2

Swap Swap

14 23 6 45 33 42 67 98 14 6 23 45 33 42 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

17
 The Third “Bubble Up” The Third “Bubble Up”
N 8 did_swap true N 8 did_swap true
to_do 5 to_do 5
index 3 index 3

No Swap

14 6 23 45 33 42 67 98 14 6 23 45 33 42 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

The Third “Bubble Up” The Third “Bubble Up”

N 8 did_swap true N 8 did_swap true
to_do 5 to_do 5
index 4 index 4

Swap

14 6 23 45 33 42 67 98 14 6 23 45 33 42 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

18
 The Third “Bubble Up” The Third “Bubble Up”
N 8 did_swap true N 8 did_swap true
to_do 5 to_do 5
index 4 index 5

Swap

14 6 23 33 45 42 67 98 14 6 23 33 45 42 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

The Third “Bubble Up” The Third “Bubble Up”

N 8 did_swap true N 8 did_swap true
to_do 5 to_do 5
index 5 index 5

Swap Swap

14 6 23 33 45 42 67 98 14 6 23 33 42 45 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

19
 After Third Pass of Outer Loop The Fourth “Bubble Up”
N 8 did_swap true N 8 did_swap false
to_do 5 to_do 4
index 6 Finished third “Bubble Up” index 1

14 6 23 33 42 45 67 98 14 6 23 33 42 45 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

The Fourth “Bubble Up” The Fourth “Bubble Up”

N 8 did_swap false N 8 did_swap true
to_do 4 to_do 4
index 1 index 1

Swap Swap

14 6 23 33 42 45 67 98 6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

20
 The Fourth “Bubble Up” The Fourth “Bubble Up”
N 8 did_swap true N 8 did_swap true
to_do 4 to_do 4
index 2 index 2

No Swap

6 14 23 33 42 45 67 98 6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

The Fourth “Bubble Up” The Fourth “Bubble Up”

N 8 did_swap true N 8 did_swap true
to_do 4 to_do 4
index 3 index 3

No Swap

6 14 23 33 42 45 67 98 6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

21
 The Fourth “Bubble Up” The Fourth “Bubble Up”
N 8 did_swap true N 8 did_swap true
to_do 4 to_do 4
index 4 index 4

No Swap

6 14 23 33 42 45 67 98 6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

After Fourth Pass of Outer Loop The Fifth “Bubble Up”

N 8 did_swap true N 8 did_swap false
to_do 4 to_do 3
index 5 Finished fourth “Bubble Up” index 1

6 14 23 33 42 45 67 98 6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

22
 The Fifth “Bubble Up” The Fifth “Bubble Up”
N 8 did_swap false N 8 did_swap false
to_do 3 to_do 3
index 1 index 2

No Swap

6 14 23 33 42 45 67 98 6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

The Fifth “Bubble Up” The Fifth “Bubble Up”

N 8 did_swap false N 8 did_swap false
to_do 3 to_do 3
index 2 index 3

No Swap

6 14 23 33 42 45 67 98 6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

23
 The Fifth “Bubble Up” After Fifth Pass of Outer Loop
N 8 did_swap false N 8 did_swap false
to_do 3 to_do 3
index 3 index 4 Finished fifth “Bubble Up”

No Swap

6 14 23 33 42 45 67 98 6 14 23 33 42 45 67 98
1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

Finished “Early” Summary of bubblesort

• “Bubble Up” algorithm will move largest value to its
N 8 did_swap false correct location (to the right)
to_do 3 • Repeat “Bubble Up” until all elements are correctly
We didn’t do any swapping, placed:
index 4 so all of the other elements
must be correctly placed. – Maximum of N-1 times
– Can finish early if no swapping occurs
We can “skip” the last two • We reduce the number of elements we compare each
passes of the outer loop. time one is correctly placed
6 14 23 33 42 45 67 98 O(n2)
1 2 3 4 5 6 7 8

24