Priority Queues

CS 250 Data Structures and Algorithms

Dr. Faisal Shafait

1 / 70
 Priority Queues

Priority Queues: background

Many algorithms process items in a specific order.

e.g. scheduling jobs according to their importance relative to other
jobs.
it requires sorting the jobs by importance, and then evaluating
them in this sorted order.
Many naturally occurring processes are accurately modeled by priority
queues.
everyone keeps a priority queue of tasks to do.

2 / 70
 Priority Queues

Priority Queues: overview

Priority queues are data structures that provide more flexibility than
simple sorting,
they allow new elements to enter a system at arbitrary intervals.
it is much more cost-effective to insert a new job into a priority
queue than to re-sort everything on each such arrival.

3 / 70
 Priority Queues

Priority Queues: operations

Operations.
INSERT(x). given an item x with key k, insert it into the priority
queue.
FIND-MINIMUM() or FIND-MAXIMUM(). return item with smallest
or largest key value.
DELETE-MINIMUM() or DELETE-MAXIMUM(). remove item with
smallest or largest key value.
We can simply call the operations as PUSH, TOP and POP.

4 / 70
 Priority Queues

Priority Queues: implementation

Goal: make run time of each operation as close to O(1) as possible.

Two implementations:
multiple queues
an AVL tree

5 / 70
 Priority Queues

Priority Queues: implementation

Assume that there are a fixed number of priorities, say M

create an array of M queues
Push a new object onto the queue corresponding to the priority
Top and pop find the first empty queue with highest priority

6 / 70
 Priority Queues

Priority Queues: implementation

record Multiqueue
{
queues[M] // M queues for M priorities
count // total number of items
}

Operations:
EMPTY()
TOP()
PUSH(n)
POP()

7 / 70
 Priority Queues

Priority Queues: implementation

Check if the multiqueue is empty,

EMPTY()
return count == 0

Insert item x with priority k,

PUSH(x, k)
if k < 0 and k >= M:
illegal argument exception
queues[k].ENQUEUE(x)
count += 1

8 / 70
 Priority Queues

Priority Queues: implementation

Return an item from multiqueue,
TOP()
if EMPTY():
underflow exception
for i = 0 to M-1:
if queues[i].IS-NOT-EMPTY():
return queues[i].FRONT():

Delete an item from multiqueue,

POP()
if EMPTY():
underflow exception
for i = 0 to M-1:
if queues[i].IS-NOT-EMPTY():
queues[i].DEQUEUE()
count -= 1
return

9 / 70
 Priority Queues

Priority Queues: implementation

The run times are reasonable:

PUSH is Θ(1)
TOP and POP are both O (M )
Unfortunately, it restricts the range of priorities
The memory requirement is Ω(M + n)

10 / 70
 Priority Queues

Priority Queues: implementation

We could simply insert the objects into an AVL tree where the order is
given by the stated priority:
PUSH is Θ(ln(n))
TOP is Θ(ln(n))
POP is Θ(ln(n))
There is significant overhead for maintaining both the tree and the
corresponding balance.

11 / 70
 Priority Queues

Priority Queues: summary

overview of priority queues

defined its basic operations.
analyzed running times of different implementations.

12 / 70
 Binary Heaps

Binary Heaps: background

A non-empty binary tree is a heap if,

all keys in either sub-tree are greater than the root key

7 12

9 31 41

33 19 32 58 42 53

36 88 55 39 89

Not: by definition a single node tree is a heap.

13 / 70
 Binary Heaps

Binary Heaps: operations

Operations.
Top
Pop
Push

14 / 70
 Binary Heaps

Binary Heaps: operations

Top. return the root: key 3

Θ(1) time.

7 12

9 31 41

33 19 32 58 42 53

36 88 55 39 89

15 / 70
 Binary Heaps

Binary Heaps: operations

Pop removes the minimum key.

promote the node of the sub-tree with the least value.
recurse for the subtree.

7 12

9 31 15

33 19 32 58 41

36 88 55 39 89 42 53

16 / 70
 Binary Heaps

Binary Heaps: operations

Pop removes the minimum key.

promote the node of the sub-tree with the least value.
recurse for the subtree.

12

9 31 15

33 19 32 58 41

36 88 55 39 89 42 53

17 / 70
 Binary Heaps

Binary Heaps: operations

Pop removes the minimum key.

promote the node of the sub-tree with the least value.
recurse for the subtree.

9 12

31 15

33 19 32 58 41

36 88 55 39 89 42 53

18 / 70
 Binary Heaps

Binary Heaps: operations

Pop removes the minimum key.

promote the node of the sub-tree with the least value.
recurse for the subtree.

9 12

19 31 15

33 32 58 41

36 88 55 39 89 42 53

19 / 70
 Binary Heaps

Binary Heaps: operations

Pop removes the minimum key.

promote the node of the sub-tree with the least value.
recurse for the subtree.

9 12

19 31 15

33 55 32 58 41

36 88 55 39 89 42 53

20 / 70
 Binary Heaps

Binary Heaps: quiz

Pop two times.

9 12

19 31 15

33 55 32 58 41

36 88 39 89 42 53

21 / 70
 Binary Heaps

Binary Heaps: quiz

Pop two times.

19 12

33 31 15

36 55 32 58 41

88 39 89 42 53

22 / 70
 Binary Heaps

Binary Heaps: quiz

Pop two times.

12

19 15

33 31 41

36 55 32 58 42

88 39 89 53

23 / 70
 Binary Heaps

Binary Heaps: operations

Push. into a heap,

either at a leaf.
or, at the root.
Consider the first approach,
insert key as a leaf.
move it up if it is smaller than the parent.

24 / 70
 Binary Heaps

Binary Heaps: operations

Push 17 into the tree

select an arbitrary leaf node.

12

19 15

33 31 41

36 55 32 58 42

88 39 89 53

25 / 70
 Binary Heaps

Binary Heaps: operations

Push 17 into the tree

select an arbitrary leaf node.

12

19 15

33 31 41

36 55 32 58 42

88 17 39 89 53

26 / 70
 Binary Heaps

Binary Heaps: operations

Push 17 into the tree

select an arbitrary leaf node.

12

19 15

33 31 41

36 55 17 58 42

88 32 39 89 53

27 / 70
 Binary Heaps

Binary Heaps: operations

Push 17 into the tree

select an arbitrary leaf node.

12

19 15

33 17 41

36 55 31 58 42

88 32 39 89 53

28 / 70
 Binary Heaps

Binary Heaps: operations

Push 17 into the tree

select an arbitrary leaf node.

12

17 15

33 19 41

36 55 31 58 42

88 32 39 89 53

29 / 70
 Binary Heaps

Binary Heaps: operations

Push 17 into the tree

select an arbitrary leaf node.

12

17 15

33 19 41

36 55 31 58 42

88 32 39 89 53

This process is called percolation.

lighter (smaller) objects move up from the bottom of the heap.
30 / 70
 Binary Heaps

Binary Heaps: operations

Push key 25 into the heap.

12

17 15

33 19 41 32

36 55 31 58 42 89 39 53

88

31 / 70
 Binary Heaps

Binary Heaps: operations

Percolate 25 to its appropriate location.

12

17 15

33 19 41 32

36 55 31 58 42 89 39 53

88 25

32 / 70
 Binary Heaps

Binary Heaps: operations

Percolate 25 to its appropriate location.

12

17 15

33 19 41 32

25 55 31 58 42 89 39 53

88 36

33 / 70
 Binary Heaps

Binary Heaps: operations

Percolate 25 to its appropriate location.

12

17 15

25 19 41 32

33 55 31 58 42 89 39 53

88 36

34 / 70
 Binary Heaps

Binary Heaps: operations

Pop key 12 from the heap.

percolation creates a hole.
results in a non-complete tree.

12

17 15

25 19 41 32

33 55 31 58 42 89 39 53

88 36

35 / 70
 Binary Heaps

Binary Heaps: operations

Pop key 12 from the heap.

percolation creates a hole.
results in a non-complete tree.

15

17 32

25 19 41 39

33 55 31 58 42 89 φ 53

88 36

36 / 70
 Binary Heaps

Binary Heaps: operations

Pop key 12 from the heap.

copy the last entry in the heap.
results in a complete tree.

12

17 15

25 19 41 32

33 55 31 58 42 89 39 53

88 36

37 / 70
 Binary Heaps

Binary Heaps: operations

Pop key 12 from the heap.

copy the last entry in the heap.
results in a complete tree.

36

17 15

25 19 41 32

33 55 31 58 42 89 39 53

88 36

38 / 70
 Binary Heaps

Binary Heaps: operations

Pop key 12 from the heap.

copy the last entry in the heap.
results in a complete tree.

15

17 36

25 19 41 32

33 55 31 58 42 89 39 53

88 36

39 / 70
 Binary Heaps

Binary Heaps: operations

Pop key 12 from the heap.

copy the last entry in the heap.
results in a complete tree.

15

17 32

25 19 41 36

33 55 31 58 42 89 39 53

88 36

40 / 70
 Binary Heaps

Binary Heaps: quiz

Pop key 15 from the heap.

15

17 32

25 19 41 36

33 55 31 58 42 89 39 53

88

41 / 70
 Binary Heaps

Binary Heaps: quiz

Pop key 15 from the heap.

88

17 32

25 19 41 36

33 55 31 58 42 89 39 53

88

42 / 70
 Binary Heaps

Binary Heaps: quiz

Pop key 15 from the heap.

17

88 32

25 19 41 36

33 55 31 58 42 89 39 53

88

43 / 70
 Binary Heaps

Binary Heaps: quiz

Pop key 15 from the heap.

17

19 32

25 88 41 36

33 55 31 58 42 89 39 53

88

44 / 70
 Binary Heaps

Binary Heaps: quiz

Pop key 15 from the heap.

17

19 32

25 31 41 36

33 55 88 58 42 89 39 53

88

45 / 70
 Binary Heaps

Binary Heaps: quiz

Pop key 17 from the heap.

17

19 32

25 31 41 36

33 55 88 58 42 89 39 53

46 / 70
 Binary Heaps

Binary Heaps: quiz

Pop key 17 from the heap.

53

19 32

25 31 41 36

33 55 88 58 42 89 39 53

47 / 70
 Binary Heaps

Binary Heaps: quiz

Pop key 17 from the heap.

19

53 32

25 31 41 36

33 55 88 58 42 89 39 53

48 / 70
 Binary Heaps

Binary Heaps: quiz

Pop key 17 from the heap.

19

25 32

53 31 41 36

33 55 88 58 42 89 39 53

49 / 70
 Binary Heaps

Binary Heaps: quiz

Pop key 17 from the heap.

19

25 32

33 31 41 36

53 55 88 58 42 89 39 53

50 / 70
 Binary Heaps

Binary Heaps: implementation

An array can be used to represent a complete tree.
filled in breadth-first traversal order.
15

17 32

25 19 41 36

33 55 31 58 42 89 39 53

88

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 15 | 17 | 32 | 25 | 19 | 41 | 36 | 33 | 55 | 31 | 58 | 42 | 89 | 39 | 53 | 88 |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

51 / 70
 Binary Heaps

Binary Heaps: implementation

Recall the associations.

the children of node k are 2k , 2k + 1.
the parent of node k (k , 0) is k /2

52 / 70
 Binary Heaps

Binary Heaps: implementation

The children of 15 are 17 and 32.

15

17 32

25 19 41 36

33 55 31 58 42 89 39 53

88

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 15 | 17 | 32 | 25 | 19 | 41 | 36 | 33 | 55 | 31 | 58 | 42 | 89 | 39 | 53 | 88 |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

53 / 70
 Binary Heaps

Binary Heaps: implementation

The children of 17 are 25 and 19.

15

17 32

25 19 41 36

33 55 31 58 42 89 39 53

88

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 15 | 17 | 32 | 25 | 19 | 41 | 36 | 33 | 55 | 31 | 58 | 42 | 89 | 39 | 53 | 88 |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

54 / 70
 Binary Heaps

Binary Heaps: implementation

An array can be used to represent a complete tree.

filled in breadth-first traversal order.

6 12

9 14 23 29

10 25 19 15

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 3 | 6 | 12 | 9 | 14 | 23 | 29 | 10 | 25 | 19 | 15 | | | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

55 / 70
 Binary Heaps

Binary Heaps: implementation

Push key 26.
use count + 1 to get the new position.
requires no percolations.

6 12

9 14 23 29

10 25 19 15 26

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 3 | 6 | 12 | 9 | 14 | 23 | 29 | 10 | 25 | 19 | 15 | 26 | | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

56 / 70
 Binary Heaps

Binary Heaps: implementation

Push key 8.
it requires percolations.

6 12

9 14 23 29

10 25 19 15 26 8

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 3 | 6 | 12 | 9 | 14 | 23 | 29 | 10 | 25 | 19 | 15 | 26 | 8 | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

57 / 70
 Binary Heaps

Binary Heaps: implementation

Swap 8 and 23.

6 12

9 14 8 29

10 25 19 15 26 23

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 3 | 6 | 12 | 9 | 14 | 8 | 29 | 10 | 25 | 19 | 15 | 26 | 23 | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

58 / 70
 Binary Heaps

Binary Heaps: implementation

Swap 8 and 12.

6 8

9 14 12 29

10 25 19 15 26 23

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 3 | 6 | 8 | 9 | 14 | 12 | 29 | 10 | 25 | 19 | 15 | 26 | 23 | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

59 / 70
 Binary Heaps

Binary Heaps: implementation

Pop the top.
copy the last item in the array.
it requires percolations.

6 8

9 14 12 29

10 25 19 15 26 23

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 3 | 6 | 8 | 9 | 14 | 12 | 29 | 10 | 25 | 19 | 15 | 26 | 23 | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

60 / 70
 Binary Heaps

Binary Heaps: implementation

Pop the top.
copy the last item in the array.
it requires percolations.

23

6 8

9 14 12 29

10 25 19 15 26 23

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 23 | 6 | 8 | 9 | 14 | 12 | 29 | 10 | 25 | 19 | 15 | 26 | | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

61 / 70
 Binary Heaps

Binary Heaps: implementation

Swap 23 and 6.

23 8

9 14 12 29

10 25 19 15 26 23

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 6 | 23 | 8 | 9 | 14 | 12 | 29 | 10 | 25 | 19 | 15 | 26 | | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

62 / 70
 Binary Heaps

Binary Heaps: implementation

Swap 23 and 9.

9 8

23 14 12 29

10 25 19 15 26 23

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 6 | 9 | 8 | 23 | 14 | 12 | 29 | 10 | 25 | 19 | 15 | 26 | | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

63 / 70
 Binary Heaps

Binary Heaps: implementation

Swap 23 and 10.

9 8

10 14 12 29

23 25 19 15 26 23

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 6 | 9 | 8 | 10 | 14 | 12 | 29 | 23 | 25 | 19 | 15 | 26 | | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

64 / 70
 Binary Heaps

Binary Heaps: implementation

Pop the top again.
copy the last item in the array.
it requires percolations.

9 8

10 14 12 29

23 25 19 15 26

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 6 | 9 | 8 | 10 | 14 | 12 | 29 | 23 | 25 | 19 | 15 | 26 | | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

65 / 70
 Binary Heaps

Binary Heaps: implementation

Pop the top again.
copy the last item in the array.
it requires percolations.

26

9 8

10 14 12 29

23 25 19 15 26

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 26 | 9 | 8 | 10 | 14 | 12 | 29 | 23 | 25 | 19 | 15 | | | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

66 / 70
 Binary Heaps

Binary Heaps: implementation

Swap 26 and 8.

9 26

10 14 12 29

23 25 19 15 26

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 8 | 9 | 26 | 10 | 14 | 12 | 29 | 23 | 25 | 19 | 15 | | | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

67 / 70
 Binary Heaps

Binary Heaps: implementation

Swap 23 and 12.

9 12

10 14 26 29

23 25 19 15 26

____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____ ____
| | | | | | | | | | | | | | | | | |
| | 8 | 9 | 12 | 10 | 14 | 26 | 29 | 23 | 25 | 19 | 15 | | | | | |
|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|____|
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16]

68 / 70
 Binary Heaps

Binary heaps: runtime

Running times.
TOP is Θ(1).
POP is O (ln(n)).
PUSH is,
O (ln(n)) if key inserted is less than the root
Θ(1) if greater than its parent.

69 / 70
 Binary Heaps

Heapsort

Sort an array A using a binary heap

Insert all elements into the heap.
Inserting each element is at most O (ln(n))
There are n elements in total
Constructing the heap will be O (nln(n))
POP the smallest element (root node) until the heap is empty,
resulting in a sorted array.
Popping each element is at most O (ln(n))
There are n elements in total
Emptying the heap will be O (nln(n))
Hence, the worst-case complexity of Heapsort is O (nln(n))

70 / 70