CSE 304

Design & Analysis of Algorithms

Greedy Algorithms (Part 2)

 Greedy Algorithm
• Greedy algorithms make the choice that looks
best at the moment.
• This locally optimal choice may lead to a globally
optimal solution (i.e. an optimal solution to the
entire problem).

2
 When can we use Greedy algorithms?

We can use a greedy algorithm when the following are true:

1) The greedy choice property: A globally optimal solution

can be arrived at by making a locally optimal (greedy) choice.

2) The optimal substructure property: The optimal solution

contains within its optimal solutions to subproblems.

3
 Designing Greedy Algorithms
1. Cast the optimization problem as one for which:
• we make a choice and are left with only one subproblem
to solve
2. Prove the GREEDY CHOICE
• that there is always an optimal solution to the original
problem that makes the greedy choice
3. Prove the OPTIMAL SUBSTRUCTURE:
• the greedy choice + an optimal solution to the resulting
subproblem leads to an optimal solution

4
 Example: Making Change
• Instance: amount (in cents) to return to customer
• Problem: do this using fewest number of coins
• Example:
– Assume that we have an unlimited number of coins of
various denominations:
– 1c (pennies), 5c (nickels), 10c (dimes), 25c (quarters), 1$
(loonies)
– Objective: Pay out a given sum $5.64 with the
smallest number of coins possible.

5
 The Coin Changing Problem
• Assume that we have an unlimited number of coins of various
denominations:
• 1c (pennies), 5c (nickels), 10c (dimes), 25c (quarters), 1$ (loonies)
• Objective: Pay out a given sum S with the smallest number of
coins possible.

• The greedy coin changing algorithm:

• This is a Θ(m) algorithm where m = number of denominations.

while S > 0 do
c := value of the largest coin no larger than S;
num := S / c;
pay out num coins of value c;
S := S - num*c;

6
 Example: Making Change
• E.g.:
$5.64 = $2 +$2 + $1 +
.25 + .25 + .10 +
.01 + .01 + .01 +.01

7
 Making Change – A big problem
• Example 2: Coins are valued $.30, $.20, $.05,
$.01
– Does not have greedy-choice property, since $.40 is
best made with two $.20’s, but the greedy solution will
pick three coins (which ones?)

8
 The Fractional Knapsack Problem
• Given: A set S of n items, with each item i having
– bi - a positive benefit
– wi - a positive weight
• Goal: Choose items with maximum total benefit but with weight at
most W.
• If we are allowed to take fractional amounts, then this is the fractional
knapsack problem.
– In this case, we let xi denote the amount we take of item i

– Objective: maximize

– Constraint:

9
 Example
• Given: A set S of n items, with each item i having
– bi - a positive benefit
– wi - a positive weight
• Goal: Choose items with maximum total benefit but with total weight at
most W.

“knapsack”

Solution: P
• 1 ml of 5 50$
Items:
1 2 3 4 5 • 2 ml of 3 40$
• 6 ml of 4 30$
Weight: 4 ml 8 ml 2 ml 6 ml 1 ml • 1 ml of 2 4$
Benefit: $12 $32 $40 $30 $50 10 ml
•Total Profit:124$
Value: 3 4 20 5 50
($ per ml)

10
 The Fractional Knapsack Algorithm
• Greedy choice: Keep taking item with highest value (benefit to
weight ratio)
– Since

Algorithm fractionalKnapsack(S, W)
Input: set S of items w/ benefit bi and weight wi; max. weight W
Output: amount xi of each item i to maximize benefit w/ weight at most W

for each item i in S

xi ← 0
vi ← bi / wi {value}
w←0 {total weight}
while w < W
remove item i with highest vi
xi ← min{wi , W - w}
w ← w + min{wi , W - w}
11
 The Fractional Knapsack Algorithm
• Running time: Given a collection S of n items, such that each item i
has a benefit bi and weight wi, we can construct a maximum-benefit
subset of S, allowing for fractional amounts, that has a total weight W in
O(nlogn) time.
– Use heap-based priority queue to store S
– Removing the item with the highest value takes O(logn) time
– In the worst case, need to remove all items

12
 An Activity Selection Problem
(Conference Scheduling Problem)

• Input: A set of activities S = {a1,…, an}

• Each activity has start time and a finish time
– ai=(si, fi)
• Two activities are compatible if and only if their
interval does not overlap
• Output: a maximum-size subset of mutually
compatible activities

13
 The Activity Selection Problem
• Here are a set of start and finish times

• What is the maximum number of activities that can be

completed?
• {a3, a9, a11} can be completed
• But so can {a1, a4, a8’ a11} which is a larger set
• But it is not unique, consider {a2, a4, a9’ a11}

14
 The Activity Selection Problem
Input: list of time-intervals L

Output: a non-overlapping subset S of the intervals

Objective: maximize |S| 3,7

2,4
5,8
6,9
1,11
10,12
0,3 15
 The Activity Selection Problem
Input: list of time-intervals L

Output: a non-overlapping subset S of the intervals

Objective: maximize |S| 3,7

2,4
5,8
6,9
1,11
Answer = 3 10,12
0,3 16
 The Activity Selection Problem
Algorithm 1:

1. sort the activities by the starting time

2. pick the first activity a
3. remove all activities conflicting with a
4. repeat

17
 The Activity Selection Problem
Algorithm 1:

1. sort the activities by the starting time

2. pick the first activity “a”
3. remove all activities conflicting with “a”
4. repeat

18
 The Activity Selection Problem
Algorithm 1:

1. sort the activities by the starting time

2. pick the first activity “a”
3. remove all activities conflicting with “a”
4. repeat

19
 The Activity Selection Problem
Algorithm 2:

1. sort the activities by length

2. pick the shortest activity “a”
3. remove all activities conflicting with “a”
4. repeat

20
 The Activity Selection Problem
Algorithm 2:

1. sort the activities by length

2. pick the shortest activity “a”
3. remove all activities conflicting with “a”
4. repeat

21
 The Activity Selection Problem
Algorithm 2:

1. sort the activities by length

2. pick the shortest activity “a”
3. remove all activities conflicting with “a”
4. repeat

22
 The Activity Selection Problem
Algorithm 3:

1. sort the activities by ending time

2. pick the activity which ends first
3. remove all activities conflicting with a
4. repeat

23
 The Activity Selection Problem
Algorithm 3:

1. sort the activities by ending time

2. pick the activity which ends first
3. remove all activities conflicting with a
4. repeat

24
 The Activity Selection Problem
Algorithm 3:

1. sort the activities by ending time

2. pick the activity which ends first
3. remove all activities conflicting with a
4. repeat

25
 The Activity Selection Problem
Algorithm 3:

1. sort the activities by ending time

2. pick the activity which ends first
3. remove all activities conflicting with a
4. repeat

26
 The Activity Selection Problem
Algorithm 3:

1. sort the activities by ending time

2. pick the activity a which ends first
3. remove all activities conflicting with a
4. repeat

Theorem:

Algorithm 3 gives an optimal solution to

the activity selection problem.
27
 Activity Selection Algorithm

Idea: At each step, select the activity with the smallest finish time
that is compatible with the activities already chosen.

Greedy-Activity-Selector(s, f)
n <− length[s]
A <− {1} {Automatically select first activity}
j <− 1 {Last activity selected so far}
for i <− 2 to n do
if si >= fj then
A <− A U {i} {Add activity i to the set}
j <− i {record last activity added}
return A

28
 The Activity Selection Problem
• Here are a set of start and finish times

• What is the maximum number of activities that can be

completed?
• {a3, a9, a11} can be completed
• But so can {a1, a4, a8’ a11} which is a larger set
• But it is not unique, consider {a2, a4, a9’ a11}

29
Interval Representation

30
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3115
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3215
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3315
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3415
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3515
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3615
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 3715
 Why this Algorithm is Optimal?
• We will show that this algorithm uses the
following properties
• The problem has the optimal substructure
property
• The algorithm satisfies the greedy-choice
property
• Thus, it is Optimal

38
 Greedy-Choice Property

• Show there is an optimal solution that begins with a greedy

choice (with activity 1, which as the earliest finish time)
• Suppose A ⊆ S in an optimal solution
– Order the activities in A by finish time. The first activity in A is k
• If k = 1, the schedule A begins with a greedy choice
• If k ≠ 1, show that there is an optimal solution B to S that begins with the
greedy choice, activity 1
– Let B = A – {k} ∪ {1}
• f1 ≤ f k activities in B are disjoint (compatible)
• B has the same number of activities as A
• Thus, B is optimal

39
 Optimal Substructures
– Once the greedy choice of activity 1 is made, the problem reduces
to finding an optimal solution for the activity-selection problem over
those activities in S that are compatible with activity 1
• Optimal Substructure
• If A is optimal to S, then A’ = A – {1} is optimal to S’={i ∈S: si ≥ f1}
• Why?
– If we could find a solution B’ to S’ with more activities than A’, adding
activity 1 to B’ would yield a solution B to S with more activities than A
contradicting the optimality of A
– After each greedy choice is made, we are left with an optimization
problem of the same form as the original problem
• By induction on the number of choices made, making the greedy choice
at every step produces an optimal solution

40