COMMONWEALTH OF AUSTRALIA

Copyright Regulations 1969

WARNING

This material has been reproduced and communicated to you by or on behalf

of the University of Sydney pursuant to Part VB of the Copyright Act 1968
(the Act). The material in this communication may be subject to copyright under
the Act. Any further copying or communication of this material by you may be
the subject of copyright protection under the Act.

Do not remove this notice.

The University of Sydney Page 1

Data structures and Algorithms

Lecture 5: Priority Queues

[GT 5]

A/Prof Julian Mestre

School of Computer Science

Some content is taken from material

provided by the textbook publisher Wiley.

The University of Sydney Page 2

 Priority Queue ADT

Special type of ADT map to store a collection of key-value

items where we can only remove smallest key:
– insert(k, v): insert item with key k and value v
– remove_min(): remove and return the item with smallest key
– min(): return item with smallest key
– size(): return how many items are stored
– is_empty(): test if queue is empty

We can also have a max version of this min version, but we

cannot use both versions at once.

The University of Sydney Page 3

 Example

A sequence of priority queue methods:

Method Return value Priority queue

insert(5,A) {(5,A)}
insert(9,C) {(5,A),(9,C)}
insert(3,B) {(3,B),(5,A),(9,C)}
min() (3,B) {(3,B),(5,A),(9,C)}
remove_min() (3,B) {(5,A),(9,C)}
insert(7,D) {(5,A),(7,D),(9,C)}
remove_min() (5,A) {(7,D),(9,C)}
remove_min() (7,D) {(9,C)}
remove_min() (9,C) {}
is_empty() true {}

The University of Sydney Page 4

 Application: Stock Matching Engines

At the heart of modern stock trading systems are highly reliable

systems known as matching engines, which match the stock trades
of buyers and sellers.

Buyers post bids to buy a number

of shares of a given stock
at or below a specified price

Sellers post offers (asks) to sell a

number of shares of a given stock
at or above a specified price.

The University of Sydney Page 5

 Application: Stock Matching Engines
Buy and sell orders are organized according to a price-time priority,
where price has highest priority and time is used to break ties

When a new order is entered, the matching engine determines if a trade

can be immediately executed and if so, then it performs the appropriate
matches according to price-time priority.

Sell orders

Matching
Engine

Buy orders

The University of Sydney Page 6

 Application: Stock Matching Engines
A matching engine can be implemented with two priority queues,
one for buy orders and one for sell orders.

This data structure performs element removals based on priorities

assigned to elements when they are inserted.

while True:
bid ← buy_orders.remove_max()
ask ← sell_orders.remove_min()
if bid.price ≥ ask.price then
carry out trade (bid, ask)
else
buy_orders.insert(bid)
sell_orders.insert(ask)

The University of Sydney Page 7

 Sequence-based Priority Queue

Unsorted list implementation Sorted list implementation

4 5 2 3 1 1 2 3 4 5

– insert in O(1) time since we can – insert in O(n) time since we have to find
insert the item at the beginning the place where to insert the item
or end of the sequence – remove_min and min in O(1) time since
– remove_min and min in O(n) time the smallest key is at the beginning
since we have to traverse the
entire list to find the smallest key

The University of Sydney Page 8

 Priority Queue Sorting
We can use a priority queue to sort a list of keys:
1. iteratively insert keys into an empty priority queue
2. iteratively remove_min to get the keys in sorted order

Complexity analysis:
def priority_queue_sorting(A):
– n insert operations
pq ← new priority queue
– n remove_min operations n ← size(A)
for i in [0:n] do
pq.insert(A[i])
Either sequence-based for i in [0:n] do
implementation take O(n2) A[i] = pq.remove_min()

The University of Sydney Page 9

 Selection-Sort

Variant of pq-sort using unsorted sequence implementation:

1. inserting elements with n insert operations takes O(n) time
2. removing elements with n remove_min operations takes O(n2)

Can be done in place def selection_sort(A):

(no need for extra space) n ← size(A)
for i in [0:n] do
# find s ⩾ i minimizing A[s]
Top level loop invariant: s ← i
– A[0, i) is sorted for j in [i:n] do
if A[j] < A[s] then
– A[i, n) is the priority queue s ← j
and all ⩾ A[i-1] # swap A[i] and A[s]
A[i], A[s] ← A[s], A[i]

The University of Sydney Page 10

Selection-Sort Example

i A s def selection_sort(A):
0 7, 4, 8, 2, 5, 3, 9 3 n ← size(A)
for i in [0:n] do
1 2, 4, 8, 7, 5, 3, 9 5 # find s ⩾ i minimizing A[s]
2 2, 3, 8, 7, 5, 4, 9 5 s ← i
3 2, 3, 4, 7, 5, 8, 9 4 for j in [i:n] do
if A[j] < A[s] then
4 2, 3, 4, 5, 7, 8, 9 4 s ← j
5 2, 3, 4, 5, 7, 8, 9 5 # swap A[i] and A[s]
A[i], A[s] ← A[s], A[i]
6 2, 3, 4, 5, 7, 8, 9 6

The University of Sydney Page 11

Insertion-Sort

Variant of pq-sort using sorted sequence implementation:

1. inserting elements with n insert operations takes O(n2) time
2. removing elements with n remove_min operations takes O(n)

Can be done in place def insertion_sort(A):

(no need for extra space) n ← size(A)
for i in [1:n] do
x ← A[i]
Top level loop invariant: # move forward entries > x
– A[0, i) is the priority queue j ← i
while j > 0 and x < A[j-1] do
(and thus sorted) A[j] ← A[j-1]
– A[i, n) is yet-to-be-inserted j ← j - 1
# if j>0 ⇒ x ≥ A[j-1]
# if j<i ⇒ x < A[j+1]
The University of Sydney A[j] ← x Page 12
Insertion-Sort Example

i A j def insertion_sort(A):
1 7, 4, 8, 2, 5, 3, 9 0 n ← size(A)
for i in [1:n] do
2 4, 7, 8, 2, 5, 3, 9 2 x ← A[i]
3 4, 7, 8, 2, 5, 3, 9 0 # move forward entries > x
4 2, 4, 7, 8, 5, 3, 9 2 j ← i
while j > 0 and x < A[j-1] do
5 2, 4, 5, 7, 8, 3, 9 1 A[j] ← A[j-1]
6 2, 3, 4, 5, 7, 8, 9 6 j ← j - 1
# if j>0 ⇒ x ≥ A[j-1]
# if j<i ⇒ x < A[j+1]
A[j] ← x

The University of Sydney Page 13

 Heap data structure (min-heap)

A heap is a binary tree storing (key, value) items at its nodes,

satisfying the following properties:

2
1. Heap-Order: for every node m ≠ root, 5 6
key(m) ³ key(parent(m))
9 7

2. Complete Binary Tree: let h be the height The last node is the
– every level i < h is full (i.e., there are 2i nodes) rightmost node of
maximum depth
– remaining nodes take leftmost positions of level h

The University of Sydney Page 14

 Example

(4,C)

(5,A) (6,Z)

(15,K) (9,F) (7,Q) (20,B)

(16,X) (25,J) (14,E) (12,H) (11,S) (13,W)

The University of Sydney Page 15

 Minimum of a Heap
Fact: The root always holds the smallest key in the heap
Proof:
– Suppose the minimum key is at some internal node x
– Because of the heap property, as we move up the tree, the keys
can only get smaller (assuming repeats, otherwise contradiction)
– If x is not the root, then its parent must also hold a smallest key
– Keep going until we reach the root

The University of Sydney Page 16

 Height of a Heap
Fact: A heap storing n keys has height log n
Proof:
– Let h be the height of a heap storing n keys
– Since there are 2i keys at depth i = 0, … , h - 1 and at least one
key at depth h, we have n ³ 1 + 2 + 4 + … + 2h-1 + 1
– Thus, n ³ 2h , applying log2 on both sides, log2 n ³ h

depth keys
0 1

1 2
h-1 2h-1

h ≥1

The University of Sydney Page 17

 Insertion into a Heap

– Create a new node with given key

– Find location for new node
– Restore the heap-order property

insert(1)

2 2

5 6 5 6

9 7 9 7 1

The University of Sydney Page 18

 Upheap
2
Restore heap-order property by
swapping keys along upward path 5 6
from insertion point z
9 7 1
def up_heap(z):
while z ≠ root and
key(parent(z)) > key(z) do 2
swap key of z and parent(z) z
z ← parent(z) 5 1

9 7 6

Correctness: after swapping the z

subtree rooted at z has the property 1

Complexity: O(log n) time because 5 2

the height of the heap is log n 9 7 6
The University of Sydney Page 19
 Finding the position for insertion

– start from the last node

– go up until a left child or the root is reached
– if we reach the root then need to open a new level
– otherwise, go to the sibling (right child of parent)
– go down left until a leaf is reached

Complexity of this search is O(log n) because the height is log n.

Thus, overall complexity of insertion is O(log n) time

The University of Sydney Page 20

 Example insertion
(4,C) (4,C)
(4,C)

(5,A) (6,Z) (5,A) (6,Z)

(5,A) (6,Z)

(15,K) (9,F) (7,Q) (20,B) (15,K) (9,F) (7,Q)

(15,K) (9,F) (7,Q) (20,B)
(20,B) (2,T)

(16,X) (25,J) (14,E) (12,H) (11,S) (13,W) (2,T) (16,X) (25,J) (14,E) (12,H) (11,S) (13,W)
(16,X) (25,J) (14,E) (12,H) (11,S) (13,W)

(4,C) (4,C) (4,C)

(2,T)
(5,A) (6,Z) (5,A) (5,A) (2,T)

(15,K) (9,F) (7,Q) (2,T) (15,K) (9,F) (7,Q) (15,K) (9,F) (7,Q) (6,Z)
(6,Z)

(16,X) (25,J) (14,E) (12,H) (11,S) (13,W) (20,B) (16,X) (25,J) (14,E) (12,H) (11,S) (13,W) (20,B) (16,X) (25,J) (14,E) (12,H) (11,S) (13,W) (20,B)

(2,T)
(2,T)

(4,C)
(5,A) (5,A) (4,C)

(15,K) (9,F) (7,Q) (6,Z) (15,K) (9,F) (7,Q) (6,Z)

(16,X) (25,J) (14,E) (12,H) (11,S) (13,W) (20,B) (16,X) (25,J) (14,E) (12,H) (11,S) (13,W) (20,B)

The University of Sydney Page 21

 Removal from a Heap

– Replace the root key with the key of the last node w
– Delete w
– Restore the heap-order property

remove_min()

2 7

5 9 5 9
w
6 7 6

last node new last node

The University of Sydney Page 22
 Downheap
z
7
Restore heap-order property by
swapping keys along downward 5 9
path from the root
6
def down_heap(z):
while z has child with
key(child) < key(z) do 5
x ← child of z with smallest key
swap keys of x and z 7 9
z ← x
z
6
Correctness: after swap z heap-
order property is restored up to
level of z 5

6 9
Complexity: O(log n) time because
the height of the heap is log n 7 z
The University of Sydney Page 23
 Example removal

(4,C) (13,W)
(13,W)
(13,W)
(5,A) (6,Z)
(5,A) (6,Z)
(5,A) (6,Z)
(15,K) (9,F) (7,Q) (20,B)
(15,K) (9,F) (7,Q) (20,B)
(15,K) (9,F) (7,Q) (20,B)
(16,X) (25,J) (14,E) (12,H) (11,S)
(16,X) (25,J) (14,E) (12,H) (11,S)
(16,X) (25,J) (14,E) (12,H) (11,S)

(5,A) (5,A)
(5,A)

(9,F) (6,Z) (9,F) (6,Z)

(13,W) (6,Z)

(15,K) (13,W) (7,Q) (20,B) (15,K) (13,W) (7,Q) (20,B)

(15,K) (9,F) (7,Q) (20,B)

(16,X) (25,J) (14,E) (12,H) (11,S) (16,X) (25,J) (14,E) (12,H) (11,S)
(16,X) (25,J) (14,E) (12,H) (11,S)

(5,A) (5,A)

(9,F) (6,Z) (9,F) (6,Z)

(15,K) (7,Q) (20,B) (15,K) (12,H) (7,Q) (20,B)

(12,H)
(13,W)

(16,X) (25,J) (14,E) (11,S) (16,X) (25,J) (14,E) (13,W) (11,S)

The University of Sydney Page 24

 Finding next last node after deletion

– start from the (old) last node

– go up until a right child or the root is reached
– if we reach the root then need to close a level
– otherwise, go to the sibling (left child of parent)
– go down right until a leaf is reached

Complexity of this search is O(log n) because the height is log n.

Thus, overall complexity of deletion is O(log n) time

The University of Sydney Page 25

 Heap-based implementation of a priority queue

The University of Sydney Page 26

 Heap-Sort

Consider a priority queue Recall that priority-queue

with n items implemented sorting uses:
with a heap:
– n insert ops
– the space used is O(n)
– n remove_min ops
– methods insert and
remove_min take O(log n)

Heap-sort is the version of priority-queue sorting that implements

the priority queue with a heap. It runs in O(n log n) time.

The University of Sydney Page 27

 Heap-in-array implementation
We can represent a heap with n keys
by means of an array of length n

Special nodes: 2
– root is at 0
5 6
– last node is at n-1
9 7
For the node at index i:
– the left child is at index 2i+1
– the right child is at index 2i+2
– Parent is at index ⌊(i-1)/2⌋
2 5 6 9 7
0 1 2 3 4

The University of Sydney Page 28

 Refinements and Generalization

Heap-sort can be arranged to work in place using part of the array for
the output and part for the priority queue

A heap on n keys can be constructed in O(n) time. But the n remove_min

still take O(n log n) time

Sometimes it is useful to support a few more operations (all are given a

pointer to e):
– remove(e): Remove item e from the priority queue
– replace_key(e, k): update key of item e with k
– replace_value(e, v): update value of item e with v

The University of Sydney Page 29

 Summary: Priority queue implementations

The University of Sydney Page 30

 Implementing a Priority Queue

Entries: An object that keeps track of the associations

between keys and values

Comparators: A function or an interface to compare entry

objects

compare(a, b): returns an integer i such that

• i < 0 if a ≺ b,
• i = 0 if a = b Warning: do not assume
• i > 0 if a ≻ b that compare(a,b) is always
-1, 0, 1

The University of Sydney Page 31

 Stock Application Revisited

Online trading system where orders are stored in two priority queues
(one for sell orders and one for buy orders) as (p, t, s) entries:
– The key is (p, t), the price of the order p and the time t
such that we first sort by p and break ties with t
– The value is s, the number of shares the order is for

How do we implement the following:

– What should we do when a new order is placed?
– What if someone wishes to cancel their order before it executes?
– What if someone wishes to update the price or number of shares
for their order?

The University of Sydney Page 32