CS 316: ALGORITHMS

GRAPH: MINIMUM SPANNING

TREES (MST)
PROBLEM: LAYING TELEPHONE WIRE

Central office
WIRING: NAÏVE APPROACH

Central office

Expensive!
WIRING: BETTER APPROACH

Central office

Minimize the total length of wire connecting the customers

PROBLEM 7
8
b c d
4 9
2
a 11 i 4 14 e

8 7 6
10
h g f
1 2
A town has a set of houses and a set of roads
A road connects 2 and only 2 houses
A road connecting houses u and v has a repair cost w(u, v)
PROBLEM 7
8
b c d
4 9
2
a 11 i 4 14 e

8 7 6
10
h g f
1 2
Goal: Repair enough (and no more) roads such that:
1. Everyone stays connected
i.e., can reach every house from all other houses
2. Total repair cost is minimum
MINIMUM SPANNING TREES
A connected, undirected graph:

• Vertices = houses, Edges = roads

A weight w(u, v) on each edge (u, v)  E
8 7
8 7 b c d
b c d 4 9
4 9 2
2 a 11 i 14 e
4
a 11 i 14 e 7 6
46 8 10
7
8 10 h g f
1 2
g g 2
f
1

Find T  E such that:

1. T connects all vertices
2. w(T) = Σ(u,v)T w(u, v) is minimized
MINIMUM SPANNING TREES
Spanning Tree
• A tree (i.e., connected, acyclic graph) which contains all
the vertices of the graph
Minimum Spanning Tree
• Spanning tree with the minimum sum of weights
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
Spanning forest g g f
1 2
• If a graph is not connected, then there is a spanning
tree for each connected component of the graph
MINIMUM SPANNING TREE (MST)

Black edges and nodes are in T

Is T a minimum cost spanning tree?
1
a b
5
2 4

c 3
d

Not minimum cost  can swap edges 4 and 2.

MINIMUM SPANNING TREE (MST)

Which edges form a MST?

1 1
a b a b
3 3
4 2 4 2

c 3
d c 3
d
Minimum spanning tree is not unique
Number of edges in a MST: |V| - 1
 APPLICATIONS OF MST
Any time you want to visit all vertices in a graph at
minimum cost

• In the design of electronic circuitry, it is often necessary

to make a set of pins electrically equivalent by wiring
them together

• Find the least expensive way to connect a set of cities,

terminals, computers, etc.

• Provides a heuristic for traveling salesman problems.

The optimum traveling salesman tour is at most twice
the length of the minimum spanning tree (why??)
GROWING A MST(GENERIC
ALGORITHM)
GENERIC_MST(G, w)
1 A:={ } -- A is a subset of MST
2 while A does not form a spanning tree do
3 find an edge (u,v) that is safe for A
4 A:=A∪{(u,v)}
5 return A
HOW TO FIND A SAFE EDGE
7
b c d 9
4
2
S↑ a 11 i e ↑S
4
7 6
8 10
V-S↓ h g f ↓ V-S
1 2

We need some definitions and a theorem:

• A cut (S,V-S) of an undirected graph G=(V,E) is a
partition of V.
HOW TO FIND A SAFE EDGE
7
b c d 9
4
2
S↑ a 11 i e ↑S
4
7 6
8 10
V-S↓ h g f ↓ V-S
1 2

We need some definitions and a theorem:

• An edge crosses the cut (S,V-S) if one of its endpoints
is in S and the other is in V-S.
• An edge is a light edge crossing a cut if its weight is
the minimum of any edge crossing the cut.
• The edge (d,c) is the light edge crossing the cut.
GREEDY AND DYNAMIC SEARCH

• Greedy method involves finding the best option out of multiple

present values. In this method, we consider the first stage and
decide the output without considering the future outputs. In other
words, the Greedy algorithm solves the problem by considering
the best option at that specific moment.
• Dynamic programming involves dividing the main problem into
small subproblems. The method stores the results of subproblems
and applies them to similar subproblems. Here, storing the
answers of subproblems is called memorization. It checks the
answers of subproblems and finally comes to a conclusion to find
the optimal or best solution. As dynamic programming checks the
previous answers and avoids computing the same answer multiple
times, it is more efficient.
GREEDY AND DYNAMIC SEARCH

• Greedy method is an algorithm that follows the problem-

solving heuristic of making the locally optimal choice at each
store with the intent of finding a global optimum.
• Dynamic Programming, on the other hand, is an algorithm that
helps to efficiently solve a class of problems that have
overlapping subproblems and optimal substructure property.
These definitions explain the main difference between Greedy
Method and Dynamic Programming.
THE ALGORITHMS OF MST
Kruskal and Prim algorithms are elaborations of
the generic algorithm.

Each of them uses a specific rule to determine a

safe edge of GENERIC_MST.
 PRIM ALGORITHM

 starts with one node.

 one by one, adds a node

 each time selecting the node whose connecting edge

is safe edge
 PRIM’S ALGORITHM
The edges in set A always form a single tree

Starts from an arbitrary “root”: VA = {a}

At each step: 8 7
b c d
4
• Find a light edge crossing (VA, V - VA) 2
9

a 11 i 4 14 e
• Add this edge to A 7 6
8 10
• Repeat until the tree spans all vertices h
1
g 2
f
 HOW TO FIND LIGHT EDGES QUICKLY?
Use a priority queue Q contains vertices not yet included in the
tree, i.e., (V – VA)
• EX: VA = {a, b}, Q = {c, d, e, f, g, h, i}
We associate a key with each vertex v:
key[v] = minimum weight of any edge (u, v) connecting v to VA

8 7
b c d
4 9
2
a i 4 14 e
11
7 6
8 10
Key[a]=min(8, 11) h g f
1 2
 HOW TO FIND LIGHT EDGES QUICKLY?
Initially  Key of v is  if v is not adjacent to any vertices
in VA

After adding a new node to VA  We update the weights of

all the nodes adjacent to it
EX: after adding a to the tree, k[b]=4 and k[h]=8

8 7
b c d
4 9
2
a i 4 14 e
11
7 6
8 10
h g f
1 2
EXAMPLE
  
b
8
c
7
d 0  
4 9
 2  Q = {a, b, c, d, e, f, g, h, i}
a 11 i 4 14 e
7 6 VA = 
8 10
h
1
g 2
f Extract-MIN(Q)  a
  

4
   key [b] = 4  [b] = a
8 7
b c d key [h] = 8  [h] = a
4 9
 2 
a 11 i 14 e
4 4 8
7 6
8 10 Q = {b, c, d, e, f, g, h, i} VA = {a}
h g 2
f
1
8    = {a, #, #, #, #,#, a,#}
Extract-MIN(Q)  b
 EXAMPLE
4 8
  key [c] = 8  [c] = b
8 7
b c d
9
key [h] = 8  [h] = a 
4
 2  unchanged
a 11 i 4 14 e 8 8
7 6
8 10 Q = {c, d, e, f, g, h, i} VA = {a, b}
h g f
8
1

2
  = {b, #, #, #,#, a,#}
4 8 
7 Extract-MIN(Q)  c
8 7 key [d] = 7  [d] = c
b c d
4 2 2 9 key [f] = 4  [f] = c
 
a 11 i 4 14 e key [i] = 2  [i] = c
8
7 6
10 7 4 8 2
h
1
g 2
f Q = {d, e, f, g, h, i} VA = {a, b, c}
8  
4  = { c, #, c ,#, a, c}
Extract-MIN(Q)  i
 EXAMPLE key [h] = 7  [h] = i
8
4
8
7 key [g] = 6  [g] = i
7
b c d 7 46 8
4 9

2 2 Q = {d, e, f, g, h} VA = {a, b, c, i}
a 11 i 4 14 e
8
7 6  = { c, #, c ,i, i}
10
h g 2
f Extract-MIN(Q)  f
1
87 
6 4
4 8 7
key [g] = 2  [g] = f
b
8
c
7
d
key [d] = 7  [d] = c
4 9 10 unchanged
2 2 
key [e] = 10  [e] = f
a 11 i 4 14 e
8
7 6 7 10 2 8
10
h g 2
f Q = {d, e, g, h} VA = {a, b, c, i, f}
1
7 6
2 4  = { c, f ,f, i}
Extract-MIN(Q)  g
 EXAMPLE
4 8 7 key [h] = 1  [h] = g
8 7
b c d 7 10 1
4 9
2 2 10 Q = {d, e, h} VA = {a, b, c, i, f, g}
a 11 i 4 14 e
7 6  = { c, f ,g}
8 10
h g f
Extract-MIN(Q)  h
1 2
7
1 2 4
4 8 7 7 10
8 7
b c d Q = {d, e} VA = {a, b, c, i, f, g, h}
4 9
2 2 10  = { c, f}
a 11 i 14 e
7 6
4
Extract-MIN(Q)  d
8 10
h g 2
f
1
1 2 4
 8 7
EXAMPLE 4
b c d
9
2
a 11 i 4 14 e
7 6
8 10
4 8 7 h g 2
f
1
8 7
b c d
4 9 9
2 10
2
key [e] = 9  [e] = d
a 11 i 4 14 e
7 6 9
8 10
h g f Q = {e} VA = {a, b, c, i, f, g, h, d}
1 2
1 2 4  = { d}
Extract-MIN(Q)  e
Q =  VA = {a, b, c, i, f, g, h, d, e}
 PRIM(V, E, W, R) In Q change the
1. Q←  key of r to be 0
2. for each u  V
3. do key[u] ← ∞
4. π[u] ← NIL
5. INSERT(Q, u)
6. key[r] ← 0 ► DECREASE-KEY(Q, r, 0)
7. while Q  
8. do u ← EXTRACT-MIN(Q)
9. for each v  Adj[u]
10. do if v  Q and w(u, v)<key[v]
11. then π[v] ← u
12. DECREASE-KEY(Q, v, w(u, v))
RUNNING TIME ANALYSIS

• Depending on the heap implementation, running time is :

Extract_Min Decrease_Key
Binary heap O(lg V) O(lg V)
 PRIM(V, E, W, R)
1. Q← 
2. for each u  V Total time: O(VlgV + ElgV) = O(ElgV)

3. do key[u] ← ∞ O(V) if Q is implemented

4. π[u] ← NIL as a min-heap

5. INSERT(Q, u)
6. key[r] ← 0 ► DECREASE-KEY(Q, r, 0) O(lgV)

7. while Q   Executed |V| times

Min-heap
operations:
8. do u ← EXTRACT-MIN(Q) Takes O(lgV) O(VlgV)
9. for each v  Adj[u] Executed O(E) times total

10. do if v  Q and w(u, v)<key[v] Constant O(ElgV)

11. then π[v] ← u Takes O(lgV)

12. DECREASE-KEY(Q, v, w(u, v))

 PRIM’S ALGORITHM
Prim’s algorithm is a “greedy” algorithm

• Greedy algorithms find solutions based on a sequence of

choices which are “locally” optimal at each step.
Nevertheless, Prim’s greedy strategy produces a globally
optimum solution!
 KRUSKAL’S ALGORITHM

 Creates a forest of trees. Initially the forest consists of

n single node trees (and no edges).
 At each step, we add one safe edge (the cheapest one)
so that it joins two trees together.
 KRUSKAL’S ALGORITHM
How is it different from Prim’s algorithm?
• Prim’s algorithm grows one
tree all the time
• Kruskal’s algorithm grows
tree1
multiple trees (i.e., a forest)
at the same time.
• Trees are merged together u
using safe edges
• Since an MST has exactly |V| - 1 v
edges, after |V| - 1 merges, tree2

we would have only one component

 KRUSKAL’S ALGORITHM
8 7
Start with each vertex being its b c d
4 9
own component 2
a 11 i 4 14 e
Repeatedly merge two 8
7 6
10
components into one by h g f
1 2
choosing the light edge that
We would add
connects them edge (c, f)

Which components to consider at

each iteration?
• Scan the set of edges in
monotonically increasing
order by weight
 EXAMPLE

8
Initially we have 9 set of vertices:
b c 7 d
4 9 {a}, {b}, {c}, {d}, {e}, {f}, {g}, {h},
2 {i}
a 11 i 4 14 e
7 6 Edges arranged in increasing order
8 10
h g f according to their weights:
1 2
1: (h, g) 2: (c, i), (g, f)
1: (h, g) 8: (a, h), (b, c)
2: (c, i), (g, f) 9: (d, e) 4: (a, b), (c, f) 6: (i, g)
4: (a, b), (c, f) 10: (e, f) 7: (c, d), (i, h) 8: (a, h), (b, c)
6: (i, g) 11: (b, h) 9: (d, e) 10: (e, f)
7: (c, d), (i, h) 14: (d, f) 11: (b, h) 14: (d, f)
{a}, {b}, {c}, {d}, {e}, {f}, {g}, {h}, {i}
 IMPLEMENTATION OF KRUSKAL’S ALGORITHM
• Uses a disjoint-set data structure to determine
whether an edge connects vertices in different
components.
• Disjoint-set: is a data structure that tracks a set of
elements partitioned into a number of disjoint
(non-overlapping) subsets.
• A disjoint-set forest consists of a number of
elements each of which stores a parent pointer.
 IMPLEMENTATION OF KRUSKAL’S ALGORITHM
• If an element's parent pointer points to no other
element, then the element is the root of a tree
and is the representative member of its set.
• Each element in the set that has a particular
property representative member.
OPERATIONS ON DISJOINT DATA SETS
MAKE-SET(u) – creates a new set whose only member is u
FIND-SET(u) – returns a representative element from the set
that contains u
• E.g.: Su = {r, s, t, u}
FIND-SET(u) = r FIND-SET(s) = r
• FIND-SET has to return the same value for the elements of
the same set
OPERATIONS ON DISJOINT DATA SETS
UNION(u, v) – unites the dynamic sets that contain u and v,
say Su and Sv

• E.g.: Su = {r, s, t, u}, Sv = {v, x, y}

UNION (u, v) = {r, s, t, u, v, x, y}

Running time for FIND-SET and UNION depends

on implementation. Can be O(lg n).
KRUSKAL(V, E, W)
1. A ← 
2. for each vertex v  V
3. do MAKE-SET(v)
4. sort E into non-decreasing order by w
5. for each (u, v) taken from the sorted list
6. do if FIND-SET(u)  FIND-SET(v)
7. then A ← A  {(u, v)}
8. UNION(u, v)
9. return A
 KRUSKAL(V, E, W)
1. A← 
2. for each vertex v  V
3. do MAKE-SET(v)
O(V)
4. sort E into non-decreasing order by w O(ElgE)
5. for each (u, v) taken from the sorted list O(E)
6. do if FIND-SET(u)  FIND-SET(v)
7. then A ← A  {(u, v)}
O(lgV) O(ElgV)
8. UNION(u, v)
9. return A

Running time: O(V+ElgE+ElgV)

RUNNING TIME

- Running time: O(V+ElgE+ElgV)

- Since E= O(V2),
we have lgE=O(2lgV)=O(lgV)

The total time for Kruskal's algorithm is O(E lg

V), which is asymptotically the same as of
Prim's algorithm.
KRUSKAL’S ALGORITHM

Kruskal’s algorithm is a “greedy” algorithm

Kruskal’s greedy strategy produces a globally optimum

solution
S

u y

v
V-S
REFERENCES

Introduction to Algorithms, Second Edition by Thomas H.

Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford
Stein, The MIT Press © 2001
Chapter 23.2