Greedy Technique
Constructs a solution to an optimization problem piece by piece
through a sequence of choices that are:
• feasible
• locally optimal
• irrevocable
For some problems, yields an optimal solution for every
instance.
For most, does not but can be useful for fast approximations.
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 1
Applications of the Greedy Strategy
• Optimal solutions:
• change making for “normal” coin denominations
• minimum spanning tree (MST)
• single-source shortest paths
• simple scheduling problems
• Huffman codes
• Approximations:
• traveling salesman problem (TSP)
• knapsack problem
• other combinatorial optimization problems
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 2
Change-Making Problem
Given unlimited amounts of coins of denominations d1 > … > dm,
give change for amount n with the least number of coins
Example: d1 = 25c, d2 =10c, d3 = 5c, d4 = 1c and n = 48c
Greedy solution:
Greedy solution is
• optimal for any amount and “normal’’ set of denominations
• may not be optimal for arbitrary coin denominations
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 3
Minimum Spanning Tree (MST)
• Spanning tree of a connected graph G: a connected acyclic
subgraph of G that includes all of G’s vertices
• Minimum spanning tree of a weighted, connected graph G: a
spanning tree of G of minimum total weight
Weighted Graph Minimum Spanning Tree Spanning Tree Spanning Tree
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 4
Prim’s MST algorithm
• Start with tree T1 consisting of one (any) vertex and “grow”
tree one vertex at a time to produce MST through a series of
expanding subtrees T1, T2, …, Tn
• On each iteration, construct Ti+1 from Ti by adding vertex not
in Ti that is closest to those already in Ti (this is a “greedy”
step!)
• Stop when all vertices are included
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 5
Example
4 c
a
1
6
2
b d
3
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 6
Example
4 c
a
1
6
2
b d
3
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 7
Example
4 c
a
1
6
2
b d
3
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 8
Example
4 c
a
1
6
2
b d
3
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 9
Example
c
a
1
2
b d
3
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 10
Example
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 11
Example
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 12
Notes about Prim’s algorithm
• Proof by induction that this construction actually yields MST
• Needs priority queue for locating closest fringe vertex
• Efficiency
• O(n2) for weight matrix representation of graph and array
implementation of priority queue
• O(m log n) for adjacency list representation of graph with n vertices
and m edges and min-heap implementation of priority queue
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 13
Another greedy algorithm for MST: Kruskal’s
• Sort the edges in nondecreasing order of lengths
• “Grow” tree one edge at a time to produce MST through a
series of expanding forests F1, F2, …, Fn-1
• On each iteration, add the next edge on the sorted list unless
this would create a cycle. (If it would, skip the edge.)
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 14
Kruskal Algorithm
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 15
Example
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 16
Example
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 17
Acyclic Subgraph Detection
If the edge's vertices (u and v) are in the same set, it causes a cycle
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 18
Implementation (union-find)
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 19
Implementation – data structure
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 20
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg} }
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 21
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg}, {ic} }
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 22
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf}, {ic} }
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 23
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf}, {ic}, {ab} }
union(c,f) takes the union of two disjoint sets {hg, gf} and {ic}
{hg, gf, ic, cf}
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 24
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf, ic, cf}, {ab} }
find(i) and find(g) return the same disjoint set, therefore, it forms a cycle.
Therefore, “ig” cannot be part of MST.
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 25
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf, ic, cf}, {ab} }
c and d are not in the same set, so add “cd” into the set.
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 26
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf, ic, cf, cd}, {ab} }
h and i are in the same set, so it causes a cycle, skip “hi”
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 27
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf, ic, cf, cd}, {ab} }
b and c are in different sets, so take union(b,c) and add “bc”
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 28
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf, ic, cf, cd, ab, bc} }
b and c are in different sets, so take union(b,c) and add “bc”
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 29
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf, ic, cf, cd, ab, bc} }
skip the edge “ah”, since “a” and “h” are in the same set
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 30
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf, ic, cf, cd, ab, bc} }
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 31
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf, ic, cf, cd, ab, bc} }
“d” and “e” are not in the same set. Therefore, add “de” into the list
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 32
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf, ic, cf, cd, ab, bc, de} }
add “de” since “d” and “e” are not in the same set.
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 33
Example: Kruskal’s MST
8 7
b c d
4 9
2
a 11 i 4 14 e
7 6
8 10
h g f
1 2
Sorted edge list:
hg ic gf ab cf ig cd hi bc ah de fe bh df
1 2 2 4 4 6 7 7 8 8 9 10 11 14
Edge List = { {hg, gf, ic, cf, cd, ab, bc, de} }
Since there are 8 edges (v–1=9–1=8). MST is built already.
No need to check remaining edges, “fe, bh, df”
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 34
Notes about Kruskal’s algorithm
• Algorithm looks easier than Prim’s but is harder to implement
(checking for cycles!)
• Cycle checking: a cycle is created iff added edge connects
vertices in the same connected component
• Union-find algorithms – see section 9.2
A.Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.9 ©2012 Pearson Education, Inc. 35
