Design and Analysis of Algorithms
CoSc 3094
Unit-3
Greedy Algorithms
Computer Science Department
Oda Bultum University, Chiro, Ethiopia
[email protected] 915980493
Outline
Looping
General Characteristics of greedy algorithms
Minimum Spanning trees (MST)
Kruskal’s algorithm
Prim’s algorithm
Shortest Paths
Scheduling(reading assignment)
Characteristics of Greedy Algorithms
Greedy algorithms are characterized by the following features.
1. Greedy approach forms a set or list of candidates 𝑪.
2. Once a candidate is selected in the solution, it is there forever: once a candidate is excluded from the
solution, it is never reconsidered.
3. To construct the solution in an optimal way, Greedy Algorithm maintains two sets.
4. One set contains candidates that have already been considered and chosen, while the other set contains
candidates that have been considered but rejected.
The greedy algorithm consists of four functions.
i. Solution Function:- A function that checks whether chosen set of items provides a solution.
ii. Feasible Function:- A function that checks the feasibility of a set.
iii. Selection Function:- The selection function tells which of the candidates is the most promising.
iv. Objective Function:- An objective function, which does not appear explicitly, but gives the value of a
solution.
Hadi Hussen Unit 3 – Greedy Algorithms 4
Introduction to Minimum Spanning Tree (MST)
Let 𝐺 = 𝑁, 𝐴 be a connected, undirected graph where,
1. N is the set of nodes and
2. A is the set of edges.
Each edge has a given positive length or weight.
A spanning tree of a graph 𝐺 is a sub-graph which is basically a tree and it contains all the
vertices of 𝐺 but does not contain cycle.
A minimum spanning tree (MST) of a weighted connected graph 𝐺 is a spanning tree with
minimum or smallest weight of edges.
Two Algorithms for constructing minimum spanning tree are,
1. Kruskal’s Algorithm
2. Prim’s Algorithm
Hadi Hussen Unit 3 – Greedy Algorithms 6
Spanning Tree Examples
Graph A Spanning Tree A
B C B C
D E F G D E F G
H H
Graph Spanning Tree
A
A
B C E
B C E
D F
D F
Hadi Hussen Unit 3 – Greedy Algorithms 7
Kruskal’s Algorithm for MST – Example 1
A Step 2: Taking next Step 4: Taking next
4 5
6 min edge (B,C) min edge (A,B)
6
B 3
E B A
4
5 7 2
2
C D B E
C D 1 3
1
2
Step 3: Taking next
Step 1: Taking min
min edge (B,E) C D
edge (C,D) B E 1
3
So, we obtained a
C D 2
1 C D minimum
1 spanning tree of cost:
4 + 2 + 1 + 3 = 10
Hadi Hussen Unit 3 – Greedy Algorithms 8
Step:1
Kruskal’s Algorithm for MST – Example 2
Sort the edges in increasing order of their weight.
Edges Weight
1 2 {1, 2} 1
1 2 3 {2, 3} 2
{4, 5} 3
4 6 4 5 6
{6, 7} 3
{1, 4} 4
4 3 5 8 6 {2, 5} 4
{4, 7} 4
7 {3, 5) 5
4 3
{2, 4} 6
{3, 6} 6
7
{5, 7} 7
{5, 6} 8
Hadi Hussen Unit 3– Greedy Algorithms 9
Step:2
Kruskal’s Algorithm for MST – Example 2
Select the minimum weight edge but no cycle.
Edges Weight
1 2 {1, 2} 1
1 2 3 {2, 3} 2
{4, 5} 3
4 6 4 5 6
{6, 7} 3
{1, 4} 4
4 3 5 8 6 {2, 5} 4
{4, 7} 4
7 {3, 5) 5
4 3
{2, 4} 6
{3, 6} 6
7
{5, 7} 7
{5, 6} 8
Hadi Hussen Unit 3– Greedy Algorithms 10
Step:3
Kruskal’s Algorithm for MST – Example 2
The minimum spanning tree for the given graph.
Edges Weight
1 2 {1, 2} 1
1 2 3 {2, 3} 2
{4, 5} 3
4
{6, 7} 3
{1, 4} 4
4 3 5 6 {4, 7} 4
4 3 Total Cost = 17
Hadi Hussen Unit 3– Greedy Algorithms 11
Kruskal’s Algorithm – Example 2
Step Edges considered - Connected Components Edges Weight
{u, v}
Init. - {1} {2} {3} {4} {5} {6} {7} {1, 2} 1
1 {1,2} {1,2} (3} {4} {5} {6} {7} {2, 3} 2
2 {2,3} {1,2,3} {4} {5} {6} {7} {4, 5} 3
3 {4,5} {1,2,3} {4,5} {6} {7} {6, 7} 3
4 {6,7} {1,2,3} {4,5} {6,7} {1, 4} 4
5 {1,4} {1,2,3,4,5} {6,7} {4, 7} 4
6 {2,5} Rejected Total Cost = 17
7 {4,7} {1,2,3,4,5,6,7}
Hadi Hussen Unit 3 – Greedy Algorithms 12
Exercises – Home Work
The complexity for the Kruskal’s algorithm is in 𝜽(𝒂 𝒍𝒐𝒈 𝒏) where 𝒂 is total number of edges
and 𝒏 is the total number of nodes in the graph 𝐺.
Write the kruskal’s Algorithm to find out Minimum Spanning Tree. Apply the same and find MST
for the graph given below.
𝟏. 𝟐.
10
3
3 A 1
F C
A 4 3 7
B 2 4
C 5
8 4
5 6
4 B D D E F G
4
H 3 2 8
2 1 9
3
G E
3 H
Hadi Hussen Unit 3 – Greedy Algorithms 14
Prim’s Algorithm
In Prim's algorithm, the minimum spanning tree grows in a natural way, starting from an
arbitrary root.
At each stage we add a new branch to the tree already constructed; the algorithm stops when
all the nodes have been reached.
The complexity for the Prim’s algorithm is 𝜽(𝒏𝟐 ) where 𝑛 is the total number of nodes in the
graph 𝐺.
Hadi Hussen Unit 3 – Greedy Algorithms 15
Prim’s Algorithm for MST – Example 1 Step:1 Select an arbitrary node.
Node - Set B Edges
1 2
1 2 3 1
4 6 4 5 6
4 3 5 8 6
7
4 3
Hadi Hussen Unit 3– Greedy Algorithms 16
Prim’s Algorithm for MST – Example 1 Step:2 Find an edge with minimum weight.
Node - Set B Edges
1 2
1 2 3 1 {1, 2}, {1, 4}
4 6 1, 2 {1, 4}, {2, 3} {2, 4}, {2, 5}
6 4 5
1, 2, 3 {1,4}, {2,4}, {2,5}, {3,5},
4 3 5 8 6 {3,6}
1, 2, 3, 4 {2,4} {2,5} {3,5} {3,6} {4,5}
7 {4,7}
4 3 1, 2, 3, 4, 5 {2,4} {2,5} {3,5} {3,6} {4,7}
{5,6} {5,7}
7 1, 2, 3, 4, 5, 7 {2,4} {2,5} {3,5} {3,6} {5,6}
{5,7} {6,7}
1, 2, 3, 4, 5, 6, 7
Hadi Hussen Unit 3– Greedy Algorithms 17
Prim’s Algorithm for MST – Example 1 Step:3
The minimum spanning tree for the given graph.
Node Edges
1 2
1 2 3 1
1, 2 {1, 2}
4
1, 2, 3 {2, 3}
1, 2, 3, 4 {1, 4}
4 3 5 6 1, 2, 3, 4, 5 {4, 5}
1, 2, 3, 4, 5, 7 {4, 7}
4 3 1, 2, 3, 4, 5, 6, 7 {6, 7}
Total Cost = 17
7
Hadi Hussen Unit 3– Greedy Algorithms 18
Cost = 17
Prim’s Algorithm – Example 1
Step Edge Set B Edges Considered
Selected
{u, v}
Init. - {1} --
1 {1, 2} {1,2} {1,2} {1,4}
2 {2, 3} {1,2,3} {1,4} {2,3} {2,4} {2,5}
3 {1, 4} {1,2,3,4} {1,4} {2,4} {2,5} {3,5} {3,6}
4 {4, 5} {1,2,3,4,5} {2,4} {2,5} {3,5} {3,6} {4,5} {4,7}
5 {4, 7} {1,2,3,4,5,7} {2,4} {2,5} {3,5} {3,6} {4,7} {5,6} {5,7}
6 {6,7} {1,2,3,4,5,6,7} {2,4} {2,5} {3,5} {3,6} {5,6} {5,7} {6,7}
Exercises – Home Work
Write the Prim’s Algorithm to find out Minimum Spanning Tree. Apply the same and find MST
for the graph given below.
𝟏. 𝟐.
10
3
3 A 1
F C
A 4 3 7
B 2 4
C 5
8 4
5 6
4 B D D E F G
4
H 3 2 8
2 1 9
3
G E
3 H
Hadi Hussen Unit 3 – Greedy Algorithms 21
Introduction
Consider now a directed graph 𝐺 = (𝑁, 𝐴) where 𝑁 is the set of nodes and 𝐴 is the set of
directed edges of graph 𝐺.
Each edge has a positive length.
One of the nodes is designated as the source node.
The problem is to determine the length of the shortest path from the source to each of the
other nodes of the graph.
Dijkstra’s Algorithm is for finding the shortest paths between the nodes in a graph.
For a given source node, the algorithm finds the shortest path between the source node and
every other node.
The algorithm maintains a matrix 𝑳 which gives the length of each directed edge:
𝐿[𝑖, 𝑗] ≥ 0 if the edge (𝑖, 𝑗) ∈ 𝐴, and
𝐿[𝑖, 𝑗] = ∞ otherwise.
Hadi Hussen Unit 3 – Greedy Algorithms 23
Dijkstra’s Algorithm - Example
1 Single source shortest path algorithm
10 50
5 30 2
100
Source node = 1
10 5 Step v C 2 3 4 5
20
Init. - {2, 3, 4, 5} 50 30 100 10
4 3
50 1 5 {2, 3, 4} 50 30 20 10
Compare cost of 1–5–4
Is there path from 1 - 5 - 432 No
Yes (20) and 1–4 (100)
Hadi Hussen Unit 3 – Greedy Algorithms 24
Dijkstra’s Algorithm - Example
1 Single source shortest path algorithm
10 50
5 30 2
100
Source node = 1
10 5 Step v C 2 3 4 5
20
Init. - {2, 3, 4, 5} 50 30 100 10
4 3
50 1 5 {2, 3, 4} 50 30 20 10
2 4 {2, 3} 40 30 20 10
1–4–2
Compare cost of 1–4–3
Is there
Is there path
path from
from 11 -- 44 -- 532 Yes
No
(40) and
(70) and 1–3
1–2(50)
(30)
Hadi Hussen Unit 3 – Greedy Algorithms 25
Dijkstra’s Algorithm - Example
1 Single source shortest path algorithm
10 50
5 30 2
100
Source node = 1
10 5 Step v C 2 3 4 5
20
Init. - {2, 3, 4, 5} 50 30 100 10
4 3
50 1 5 {2, 3, 4} 50 30 20 10
2 4 {2, 3} 40 30 20 10
3 3 {2} 35 30 20 10
Compare cost of
Is there path from 1 - 3 - 542 Yes
No
1–3–2 and 1–2
Hadi Hussen Unit 3 – Greedy Algorithms 26
Exercises – Home Work
Write Dijkstra’s Algorithm for shortest path. Use the algorithm to find the shortest path from the
following graph.
1. 2.
2
2 A B
B D
10 4 1 3 10
1 4 8 7 9 2 2
A C D E
3 2 5 8 4 6
C E
1
F G
Hadi Hussen Unit 3 – Greedy Algorithms 27
Assignment(Individual Assignment)
1. Write an algorithm for:
Prims algorithm
Kruskal’s algorithm
2. Scheduling(Reading Assignment)
Hadi Hussen Unit 3 – Greedy Algorithms 53
