CSE 317: Design and Analysis of Algorithms

Shahid Hussain
Week 10: October 21 – October 26, 2024: Fall 2024

1
Greedy Algorithms
Greedy Algorithms

• Greedy algorithms use the following meta-heuristic:

• to make the locally optimal choice at each stage with the hope of finding the
global optimum
• Often greedy algorithms produce sub-optimal result however the algorithms
considered here are all optimal
• Running time for algorithms usually depend on the choice of underlying data
structure and for two of the greedy algorithms we will discuss choosing a right
data structure plays an important role
• We will discuss binary heaps for implementation of two such algorithms:
Prim’s algorithm and Dijkstra’s algorithm.

2
Binary Heaps

• In a binary heap elements are stored in a complete binary tree, namely, a

binary tree in which each level is filled from the left to the right, and must be
full before the next level is started
• In addition, the key value of any node of the tree is less than or equal to key
values of its children. In particular, the root of the tree always contains an
element with the smallest key

3
Binary Heaps (cont.)

• The binary tree (for binary heaps) can be easily represented using an array
• The nodes of tree have a natural ordering: row by row, starting at the root
and moving from the left to the right within each level. The array has
positions 1, . . . , n corresponding to the considered nodes
• One can show that the node at location j in the array has the parent at
location ⌊j/2⌋ has two of its child nodes at locations 2j and 2j + 1

4
Binary Heaps (cont.)

2 3

4 5 6 7

8 9 10 11 12 13 14 15

5
Binary Heaps (cont.): Insert Key

• To insert a new element, we place it at the bottom of the tree (in the first
available position)
• If the key of new element is less than the key of its parent, then we should
swap these elements, etc
• The number of swaps is at most the depth of the tree, which is ⌊log2 n⌋ where
n is the number of elements

2 1

2 3 2 2

4 5 4 Insert 4 5 4 3
−−−−−→1

6
Binary Heaps (cont.) Delete-Min

• To delete the min element (which is always the root of the heap) we return
the element attached to the root
• Then we remove this element from the heap, take the element attached to the
last node in the tree (in the rightmost position in the bottom level) and place
it at the root
• If the key of this element is bigger than any of the root’s child nodes, we swap
these elements, etc. The number of swaps is at most ⌊log2 n⌋

2 4 2

2 3 2 3 4 3

4 5 4 Delete-min 4 5 −→ 4 5
−−−−−−−→
7
Binary Heaps (cont.) Build-Heap

• To build a heap with n elements we can insert n elements in an empty heap.

Each operation taking O(log n) operations so building a heap takes O(n log n)
operations
• We can use binary heaps to come up with a sorting algorithm called heap sort
• The idea is to build a heap with the elements to be sorted, then delete the
min element n times
• Each delete-min operation takes O(log n) operations, so the total running time
is O(n log n)

8
Single Source Shortest Paths:
Dijkstra’s Algorithm
Single Source Shortest Paths: Dijkstra’s Algorithm

• Let G = (V, E) be a directed graph s.t. each edge e ∈ E has a length le ≥ 0

• For a path P , the length of P (l(P )) is the sum of lengths of all edges in P
• Let s be a node of G (source node)
• We assume that s has a path to every other node in G
• The algorithm constructs a set S of nodes u for which we have determined the
length of the shortest path d(u) from s
• Initially, S = {s} and d(s) = 0
• Now for each node v ∈ V \ S, we determine the length d′ (v) of a shortest path
of the following kind: the considered path passes through nodes of S until
some node u ∈ S and then passes through an edge (u, v) ∈ E
• It is clear that
d′ (v) = min d(u) + le .
e=(u,v)∈E,u∈S

• If there is no (u, v) ∈ E such that u ∈ S, then d′ (v) = +∞ 9

Single Source Shortest Paths: Dijkstra’s Algorithm (cont.)

• We choose v ∈ V \ S for which d′ (v) = min{d′ (v ′ ) : v ′ ∈ V \ S} and add v to S.

We set d(v) = d′ (v) and predecessor (v) = u, where u ∈ S and there exists an
edge e = (u, v) ∈ E such that d′ (v) = d(u) + le
• The value predecessor (v) for each v ∈ V , v ̸= s, will allow us to restore the
shortest paths from s to all other nodes in G
• The algorithm stops when S = V

10
Single Source Shortest Paths: Dijkstra’s Algorithm (cont.)

3
1
1
s 1 1
3
2

11
Single Source Shortest Paths: Dijkstra’s Algorithm (cont.)

• During each step the algorithm adds new node v to the set S, computes the
value d(v) for this node and forms the path Pv from s to v with the help of
values predecessor (·)
• Let us show by induction on |S| that Pv is the shortest path from s to v
• From here it follows that d(v) is the minimum length of a path from s to v
• Let |S| = 1, in this case S = {s} and d(s) = 0, the statement holds in this case
• Assume that the statement holds when |S| = k for some value of k ≥ 1

12
Single Source Shortest Paths: Dijkstra’s Algorithm (cont.)

• So during the step number k + 1 the algorithm chooses the node v, and
e = (u, v) be the final edge in the path Pv
• Let us consider an arbitrary path P from s to v
• Since v ∈
/ S, there is an edge e′ = (x, y) in this path such that x ∈ S and y ∈
/S
• By the inductive hypothesis, the length of any path from s to x is at least d(x)
• Therefore l(P ) ≥ d(x) + le′ ≥ d(u) + le = l(Pv ). The inequality
d(x) + le′ ≥ d(u) + le follows from the description of the algorithm
• Therefore l(P ) ≥ l(Pv ), and Pv is the shortest path from s to v

13
Single Source Shortest Paths: Dijkstra’s Algorithm (cont.)

• To optimize, we use binary heap to store values d′ (v) for nodes v ∈ V \ S

• Initially, we have d′ (s) = 0 and d′ (v) = +∞ for each v ∈ V \ {s}
• We use make-queue operation to build a binary heap with values d′ (v) as keys
• We modify the heap to have current set of nodes V \ S and current values d′ (v)
• After the k-th step the heap has all node v from V \ S and their current values
• During the step number k + 1, by operation delete-min, we choose v with the
minimum value d′ (v) and delete v from the heap, and set set d(v) = d′ (v)
• Now we modify the values of d′ (w) for each node w ∈ V \ (S ∪ {v})
• If (v, w) is not an edge then we keep d′ (w) unchanged
• Let e′ = (v, w) be an edge, then the new value of the key for w is
min{d′ (w), d(v) + le′ }
• If d′ (w) > d(v) + le′ then it is necessary to set predecessor (w) = v and to use
the decrease-key operation to decrease the key of node w up to d(v) + le′ 14
Single Source Shortest Paths: Dijkstra’s Algorithm (cont.)

• To decrease the key for w we should find w in the heap (in the corresponding
array)
• To this end, we will assume that all n nodes in G are numbered by numbers
1, . . . , n and we have an array in which in the t-th position we have the
current value of the position of t-th node in the heap
• We can store in this array values d(u) and predecessor (u)
• The decrease-key operation can be used at most once per edge, when the
initial node of the edge is added to S
• Let |V | = n and |E| = m
• The algorithm makes one operation of make-queue in time O(n log n), at most
n operations of delete-min in time O(n log n), and at most m operations of
decrease-key in time O(m log n)
• Thus, the overall time for the implementation is O((m + n) log n)
15
Minimum Spanning Trees
Minimum Spanning Trees: Prim’s Algorithm

• Let G = (V, E) be a connected undirected graph such that each edge e in E is

labeled with a cost ce ≥ 0
• We call a subset T ⊆ E a spanning tree of G if (V, T ) is a tree
• A spanning tree for which the total cost e∈T ce is as small as possible is
P

called minimum spanning tree

• We need to construct a minimum spanning tree for G
• We use Prim’s algorithm to construct such an MST

16
Minimum Spanning Trees: Prim’s Algorithm

• We start with a root node s, we maintain a set S ⊆ V on which a spanning

tree is constructed
• Initially, S = {s}. During each iteration, we add to S one node v ∈ V \ S
which minimizes the value a(v) = mine=(u,v)∈E,u∈S ce and include the edge
e = (u, v) that achieves this minimum in the constructed spanning tree T .
Algorithms stops when S = V
• It is clear that the constructed graph is connected and has |V | − 1 edges. So
this graph is a spanning tree.

3
1 7
2
s 1 5
2 1
4
17
Minimum Spanning Trees: Prim’s Algorithm

• Let us prove by induction on |S| that partial spanning tree constructed during
the step number |S| is a part of minimum spanning tree
• If |S| = 1 then the partial spanning tree is empty, and the considered
statement holds. Let for some k ≥ 1 this statement hold: |S| = k and the
constructed partial spanning tree Tk is a part of some minimum spanning
tree T

18
Minimum Spanning Trees: Prim’s Algorithm

• During the step number k + 1 we add to S an edge e = (u, v) such that

u ∈ S, v ∈ V \ S and e has the minimum cost ce among all edges that join S
and V \ S
• If e ∈ T then the statement holds.
• Now suppose e ∈/ T . In this case we add e to T , as a result we obtain a graph
K with a cycle
• This cycle must contain another edge e′ which joins S and V \ S

19
Minimum Spanning Trees: Prim’s Algorithm

• If we remove e′ from K, we obtain a connected graph with |V | nodes and

|V | − 1 edges (since T is a spanning tree, |T | = |V | − 1)
• Therefore, we obtain a tree. So T ′ = (T \ {e′ }) ∪ {e} is a spanning tree.
According to the choice of e, the total cost of T ′ is at most the total cost of T .
Thus, T ′ is a minimum spanning tree. It is clear, that Tk ⊆ T ′ and e ∈ T ′
• Therefore the considered statement holds. From this statement it follows that
Prim’s algorithm constructs a minimum spanning tree

20
Minimum Spanning Trees: Prim’s Algorithm

• e can implement Prim’s algorithm almost in the same way as Dijkstra’s

algorithm
• By analogy with Dijkstra’s algorithm we should be able do decide which node
v ∈ V S will be added to S. To this end we will consider the attachment cost
a(v) = mine=(u,v)∈E,u∈S ce for each node v ∈ V \ S.
• As before, we keep the nodes in a priority queue with a(v) as the key
• We build a binary heap with a make-queue operation
• We select a node with a delete-min operation, and update the attachment cost
using decrease-key operation

21
Minimum Spanning Trees: Prim’s Algorithm

• We perform make-queue operation one time

• The are n − 1 iterations in which we perform delete-min operation, and we
perform decrease-key operation at most once for each edge
• So the overall running time is O((n + m) log n) where n = |V | and m = |E|
• At the beginning of the algorithm we have S = {s}. We form values of a(v)
for v ∈ V \ {s} in the following way
• If (s, v) ∈ E then a(v) = c(s, v). Otherwise, a(v) = +∞
• Let during some step we add to S a new node v. Then for each node
w ∈ V \ (S ∪ {v}) for which (v, w) ∈ E, instead of old value a(w) we should
use the new value which is equal to min{a(w), c(v, w))}

22