Chapter VI DP and Network
Chapter VI DP and Network
3
Forward and backward recursion
Both the forward and backward recursions yield the same solution.
Although the forward procedure appears more logical, DP literature
commonly uses backward recursion.
The reason for this preference is that, in general, backward recursion
may be more efficient computationally.
This course uses the backward recursion to present the DP
computations in a compact tabular form.
4
Basic elements of DP models
When DP problems are solved, pay special attention to the three basic
elements of the DP model:
1) The stages
2) The decision alternatives at each stage
3) The states for each stage
Of the three elements, the definition of the state is usually the most
subtle, the definition of the state varies depending on the situation
being modeled.
5
6.2 Selected applications
6
1. Stagecoach (shortest path) problem
It is a network problem to determine the shortest route between the
origin and the destination, provided that a series of paths from the
origin to the destination.
These two points are not directly connected and each path has a
value linked to it (e.g. distance, cost, time, etc.).
The idea for this problem is that a salesperson is traveling from one
town to another town; each leg of his trip costs a certain amount and
s/he wants to find the minimum cost of his trip, given multiple paths.
7
Example
Find the shortest route between the origin 1 and the destination 7 using DP.
The distances are given in km.
2 25
5
17
16 8
14
35
1 3 7
17
9 14
26
4 22 6
Solution:
The objective of the problem is to minimize the travel distance between
nodes 1 and 7 due to time or cost constraints.
Decompose the problem into the required stages and apply backward
recursion.
8
2 25
5
17
16 8
14
35
1 3 7
17
9 14
26
4 22 6
10
Stage 3
Use as input the optimum (shortest) alternative values of each state
from stage 2
Optimum (shortest)
S3 D3 R3 S2 *R2 *R2+R3
alternative *R3
1-2 16 2 31 31+16 = 47
1 1-3 35 3 22 22+35 = 57 43 (route 1-4-5-7)
1-4 9 4 34 34+9 = 43
The shortest path is route 1-4-5-7 and its objective function value is
43 kms.
11
Exercise: Find the shortest path from town 1 to town 10. The values
between each pair of nodes are time in hours.
(Hint: decompose the problem into four stages ).
12
2. Resources allocation problem
Example: A transport company has subdivided its market into three
regions; North, East and South. The company has three buses, and it
wants to allocate the buses in a manner that results in maximum
revenue. The company will not restrict the number of buses that can be
assigned in anyone of the regions. In an extreme case, all the three
buses could be assigned in one region. The return that will be monthly
generated in each region from each possible combination are as show
below. Decision alternative Return from each region (*1000)
for each region North East South
0 0 0 0
1 7 9 6
2 12 17 10
3 20 18 16
13
Solution
The problem is a resource allocation problem.
The objective is to maximize the monthly return the from three market
segments through optimal assignment of the available buses.
Maximize z = R1 + R2 + R3
subject to
D1 + D2 + D3 ≤ 3
D1, D2, D3 ≥0 and integers
Where
R1, R2 and R3 are returns from the regions South, East and North respectively.
D1, D2 and D3 are allocation decisions into the South, East and North regions
respectively.
14
Decompose the problem into three regions (stages).
Decision alternative Return from each region (*1000)
for each region North East South
0 0 0 0
1 7 9 6
2 12 17 10
3 20 18 16
Stage 3 Stage 2 Stage 1
15
Stage 1 – allocation into the South
State S1 Decision D1
Return R1 Optimum *R1
(available buses) (allocated buses)
0 0 0 0
1 1 6 6
2 2 10 10
3 3 16 16
16
Stage 2 - allocation into the East
Use as input the optimum (maximum) values of each state from stage 1.
Optimum (maximum)
S2 D2 R2 S1 *R1 *R1+R2
return *R2
0 0 0 0 0 0 0
0 0 1 6 6
1 9
1 9 0 0 9
0 0 2 10 10
2 1 9 1 6 15 17
2 17 0 0 17
0 0 3 16 16
1 9 2 10 19
3 23
2 17 1 6 23
3 18 0 0 18
17
Stage 3 - allocation into the North
Use as input the optimum (maximum) alternative values of each state
from stage 2.
Optimum (maximum)
S3 D3 R3 S2 *R2 *R2+R3
return *R3
0 0 3 23 23
1 7 2 17 24
3 24
2 12 1 9 21
3 20 0 0 20
The optimal solution is allocating 1 bus to North, 2 buses to East and 0
to South (D3 = 1, D2 = 2, D1 = 0).
The maximum return is 24.
Represent your solution using networks.
18
Exercise: Assigning four salesmen into three regions. The table below
compiles the problem situation.
Salesmen per region Return from each region (*1000)
East South West
0 0 0 0
1 22 17 25
2 51 48 45
3 65 71 58
4 82 90 75
19
Exercise 2: A college student has one week (7 days) before final examinations begin in her four
courses, and she wants to allocate this study time as effectively as possible. She needs at least 1 day
on each course, and she likes to concentrate on just one course each day, so she wants to allocate 1,
2, 3, or 4 days to each course. Having recently taken an OR course, she decides to use dynamic
programming to make these allocations to maximize the total grade points to be obtained from the
four courses. She estimates that the alternative allocations for each course would yield the number
of grade points shown in the following table:
20
3. Knapsack/Fly-away/Cargo loading problem
― The knapsack model classically deals with the situation in which a
soldier must decide on the most valuable items to carry in a
backpack.
― The knapsack problem is also known in the literature as the fly-away
kit problem, in which a jet pilot must determine the most valuable
items to take aboard a jet.
― The cargo-loading problem, in which a vessel with limited volume or
weight capacity is loaded with the most valuable cargo items.
― It is a resource allocation model in which a single limited resource is
assigned to a number of alternatives with the objective of
maximizing the total return.
21
The general problem is represented by the following ILP
Maximize z = R1D1 + R2D1 + … + RnDn
subject to
w1D1 +w2D2 + … + wnDn ≤ W (total capacity)
D1, D2, …, Dn ≥ 0 and integers
The three elements of the model are
1) The stage i is represented by its item
2) The decision alternatives at stage i are represented by Di, the
number of units of an item included in the knapsack.
3) The state at stage i is represented by Si, the total weight assigned
(available) to stages (items).
22
Example:
A 4-ton vessel can be loaded with one or more of three items. The
following table gives the unit weight, wi, in tons and the unit revenue in
thousands of dollars, Ri, for each item at its stage i. How should the
vessel be loaded to maximize the total return?
Item Weight [ton] Return
1 2 31
2 3 47
3 1 14
23
Solution
The problem can be modelled as:
Maximize z = 14D1 + 47D2 + 31D3
subject to
D1 + 3D2 + 2D3 ≤ 4
D1, D2, D3 ≥ 0 and integer
When the vessel is totally loaded with one item type, its capacity is 2 units of item
1, 1 unit of item 2 and 4 units of item 3.
The three items are the three stages.
The state (S) in each stage is the total weight/capacity allocated to the stages ( it is
the same at every stage).
The decision alternative (D) at each stage the number of units of item to be
shipped.
24
Decompose the problem into three stages.
Item Weight [ton] Return
Stage 3 1 2 31
Stage 2 2 3 47
Stage 1 3 1 14
25
Stage 2 - assigning item 2 for shipping
Use as input the optimum (maximum) alternative values of each state
from stage 1.
S2 D2 R2 S1 *R1 *R1 + R2 Optimum *R2
0 0 0 0 0 0 0
1 0 0 1 14 14 14
2 0 0 2 28 28 28
0 0 3 42 42
3 47
1 47 0 0 47
0 0 4 56 56
4 61
1 47 1 14 61
26
Stage 3 - assigning item 1 for shipping
Use as input the optimum (maximum) alternative values of each state from
stage 2.
S3 D3 R3 S2 *R2 *R2 + R3 Optimum *R3
0 0 0 0 0 0 0
1 0 0 1 14 14 14
0 0 2 28 28
2 31
1 31 0 0 31
0 0 3 47 47
3 47
1 31 1 14 45
0 0 4 61 61
4 1 31 2 28 59 62
2 62 0 62 62
The optimal solution is assigning 2 units of item 1 (D3 =2) and nothing from
items 2 and 3 (D2 = D1 = 0) and z = 62.
Solve this problem when the vessel capacity is 3 tons, 5 tons, etc.
27
Exercise:
A wilderness hiker must pack three items: food, first-aid kits, and
clothes. The backpack has a capacity of 3 m3. Each unit of food takes 1
m3. A first-aid kit occupies 1/4 m3 and each piece of cloth takes about ½
m3. The hiker assigns the priority weights 3,4, and 5 to food, first aid,
and clothes, which means that clothes are the most valuable of the
three items. From experience, the hiker must take at least one unit of
each item and no more than two first-aid kits. How many of each item
should the hiker take?
28
4. Workforce size problem
― In some industrial/construction projects, hiring and firing are exercised
to maintain a labor force that meets the needs of the project.
― Given that the activities of hiring and firing both incur additional costs,
how should the labor force be maintained throughout the life of the
project?
― Let us assume that the project will be executed over the span of n
periods and that the minimum labor-force required in every period is b
laborers.
― Theoretically, hiring and firing is required to keep the workforce in
each planned period exactly equal to the corresponding b.
― Alternatively, it may be more economical to maintain a labor-force
larger than the minimum requirements through new hiring.
29
The elements of the DP model are defined as follows:
1) The stage i is represented by each planned period .
2) The alternatives at stage i are Di, the actual number of laborers at
stage i.
3) The state at stage i is represented by the number of laborers available
at the next stage (previous period ).
Given that bi is the minimum number of laborers required at stage i
(corresponding period), two costs can be incurred at stage i:
1) The cost of maintaining an excess labor-force = C1(Di - bi) and
2) The cost of hiring additional laborers with reference to the previous
period (next stage) = C2(Di – Si)
It is assumed that no additional cost is incurred when employment is
discontinued for the next specific problem.
30
The general DP model is given as:
Minimize z =
Subject to
Di ≥ bi
The DP recursive equation is given as:
*Ri = min }, i = 1, 2,…, n
R0 = 0
The computations start at stage 1 with D1 = b1 and terminate
at stage n.
31
Example: A contractor estimates that the size of the workforce needed
over the next five weeks to be 5, 7, 8, 4 and 6 workers, respectively.
Excess labor kept on the force will cost Birr 300 per worker per week,
and new hiring in any week will incur a fixed cost of Birr 400 plus Birr
200 per worker per week. The data of the problem are summarized as:
Weeks 1 2 3 4 5
Laborers (b) 5 7 8 4 6
Solution:
Decompose the problem into five stages based on weeks (periods)
Stage 5 Stage 4 Stage 3 Stage 2 Stage 1
Weeks 1 2 3 4 5
Laborers (b) 5 7 8 4 6
32
Stage 1- week 5 ( b1 = 6, minimum requirement of laborers)
State S1 Decision D1 Return R1
Optimum *R1
(laborers in week 4) (required laborers) (Cost *100)
4 6 (2 hire) 8 8
5 6 (1 hire) 6 6
6 6 (0 hire) 0 0
Weeks 1 2 3 4 5
b) Laborers (b) 8 4 7 8 2
36
5. Equipment replacement problem
― The longer a machine stays in service, the higher is its maintenance
cost, and the lower its productivity.
― When a machine reaches a certain age, it may be more economical to
replace it.
― The problem is to determine the most economical age of a machine.
― Suppose that we are studying the machine replacement problem over
a span of n years.
― At the start of each year, we decide whether to keep the machine in
service an extra year or to replace it with a new one.
― Let r(t), c(t), and s(t) represent the yearly revenue, operating cost, and
salvage value of a t-year-old machine.
― The cost of acquiring a new machine in any year is I.
37
The elements of the DP model are:
1) The stage i is represented by a decision year, i = 1,2, ... , n.
2) The decision alternatives at stage (decision year) i call for either
keeping or replacing the machine at the start of each decision year
i.
3) The state at stage i is the age of the machine at the start of every
decision year i.
38
The general DP model is given as:
Maximize z =
The DP recursive equation is given as:
*Ri =
R0 = 0
39
Example: A company needs to determine the optimal replacement
policy for a current 3-year-old machine over the next 4 years (n = 4).
The company requires that a 6-year-old machine be replaced. The cost
of a new machine is Birr 100,000. The following table gives the data of
the problem.
Age, t (year) Revenue, r(t) Operating cost, c(t) Salvage value, s(t)
0 20,000 200 -
1 19,000 600 80,000
2 18,500 1,200 60,000
3 17,200 1,500 50,000
4 15,500 1,700 30,000
5 14,000 1,800 10,000
6 12,200 2,200 5,000
40
Solution
At the start of decision year 1, we have a 3-year-old machine, then we
can either replace it (R) or keep it (K) for another year.
At the start of decision year 2, if replacement occurs, the new
machine will be 1 year old; otherwise, the old machine will be 4 years
old.
The same logic applies at the start of decision years 2 to 4.
If a 1-year-old machine is replaced at the start of decision years 2,3,
or 4, its replacement will be 1 year old at the start of the next decision
years.
Also, at the start of year 4, a 6-year-old machine must be replaced,
and at the end of year 4 (end of the planning horizon), we salvage (s)
the machines.
41
Representation of machine age as a function of decision year
42
The solution of the network is equivalent to finding the longest route (i.e.,
maximum revenue) from the start of year 1 to the end of year 4.
Decompose the problem into four stages depending on the four decision stages
Stage 1- start of decision year 4
The net return
State S1 Decision D1
Return R1 (net revenue Optimum
(age at start of (keep or
*1000) *R1
decision year 4) replace)
Keep 19.0 + 60 - 0.6 = 78.4
1 79.8
Replace 20 + 80 + 80 - 0.2 - 100 = 79.8
Keep 18.5 + 50 - 1.2 = 67.3
2 67.3
Replace 20 + 60 + 80 - 0.2 - 100 = 59.8
Keep 17.2 + 30 - 1.5 = 45.7
3 49.8
Replace 20 + 50 + 80 - 0.2 - 100 = 49.8
Keep Must be replaced
6 4.8
Replace 20 + 5 + 80 - 0.2 - 100 = 4.8 43
Stage 2- start of decision year 3
The net return
44
Stage 3- start of decision year 2
The net return
45
Stage 4- start of decision year 1
The net return
46
Exercise
In each of the following cases, develop the network, and find the
optimal solution for the model in the previous example:
1) The machine is 2 years old at the start of year 1.
2) The machine is 1 year old at the start of year 1.
3) The machine is bought new at the start of year 1.
47
6.3 Network models
A network is an arrangement of paths connected at various points,
through which one or more items move from one point to another;
like such networks as highway systems, telephone networks, railroad
systems, and television networks.
The purpose of the network is to determine the shortest distance,
shortest length of time, or lowest cost between points in the network.
A class of network models that are directed at the flow of items
through a system are referred to as network flow models.
A network to be directed or oriented if it allows positive flow in one
direction and zero flow in the opposite direction.
48
Network components
Networks are illustrated as diagrams consisting of two main
components: nodes and branches.
Nodes represent junction points for example, an intersection of
several streets, and they are denoted in the network diagram by
circles.
Branches connect the nodes and reflect the flow from one point in
the network to another, and they are represented by lines connecting
the nodes.
49
A network has an origin and destinations.
The node representing Atlanta is referred to as the origin, and any of
the three remaining nodes could be the destinations.
A number is assigned to each node to provide a more convenient
means of identifying the nodes and branches.
Typically, a value that represents a distance, length of time, or cost is
assigned to each branch.
For example, the values 4, 6, 3, and 5, corresponding to the four
branches, represent the lengths of time, in hours, between the
attached nodes.
50
1. Shortest route model
The shortest route problem is to determine the shortest distance between
an originating point and several destination points.
It is applicable to distribute goods from a distribution center to various
demand centers or retail centers.
The steps of the shortest route solution method are as follows:
1) Select the node with the shortest direct route from the origin.
2) Establish a permanent set with the origin node and the node that was
selected in step 1.
3) Determine all nodes directly connected to the permanent set nodes.
4) Select the node with the shortest route (branch) from the group of
nodes directly connected to the permanent set nodes.
5) Repeat steps 3 and 4 until all nodes have joined the permanent set
51
Example:
A shipping company manager wants to determine the best routes (in
terms of the minimum travel time) for the trucks to take to reach their
destinations from the origin node 1. Solve the problem using the
shortest route solution technique. All measurement are time in hrs.
25
2 5
12
21 10 18
30 9
1 4 7
17 12 10
15
3 25 6
52
Exercises:
1) The Burger Doodle restaurant franchises in Los Angeles are supplied
from a central warehouse in Inglewood. The location of the warehouse
and its proximity, in minutes of travel time, to the franchises are shown
in the following network:
53
2) The network below gives the distances in miles between pairs of
cities 1,2, . . , and 8. Find the shortest route between the following
cities:
a) Cities 1 and 8
b) Cities 1 and 6
Hint: the flows are only directed according to the directions of their arrows.
54
3) In the network below, branches 7-6 and 6-4 are unidirectional
(directed), and all the distances are in miles. Determine the shortest
route between the following pairs of nodes:
a) From node 1 to node 7.
b) From node 7 to node 1.
c) From node 6 to node 7.
55
2. Minimal spanning tree model
In the shortest route problem, the objective was to determine the
shortest routes between the origin and the destination nodes in the
network.
The minimal spanning tree problem is similar to the shortest route
problem, except that the objective is to connect all the nodes in the
network so that the total branch lengths are minimized.
The resulting network spans (connects) all the points in the network
at a minimum total distance (or length).
Applicable in connecting different cities/towns using different
infrastructures.
56
The steps of the minimal spanning tree solution method are as follows:
1) Select any starting node (conventionally, node 1 is selected).
2) Select the node closest to the starting node to join the spanning
tree.
3) Select the closest node not presently in the spanning tree.
4) Repeat step 3 until all nodes have joined the spanning tree.
57
Example:
The Metro Cable Television Company is to install a television cable system in a
community consisting of seven suburbs. Each of the suburbs must be connected to
the main cable system. The cable television company wants to lay out the main
cable network in a way that will minimize the total length of cable that must be
installed. The possible paths available to the cable television company and the feet
of cable (in thousands of feet) required for each path are shown in Figure below.
Find the total length of cable required to connect the seven suburbs.
25
2 5
12
21 10 18
30 9
1 4 7
17 12 10
15
3 25 6
58
Exercise:
1) In intermodal transportation, loaded truck
trailers are Shipped between railroad
terminals on special flatbed carts. The
figure shows the location of the main
railroad terminals in the United States and
the existing railroad tracks. The objective is
to decide which tracks should be
"revitalized" to handle the intermodal
traffic. In particular, the Los Angeles (LA)
terminal must be linked directly to Chicago
(CH) to accommodate expected heavy
traffic. Other than that, all the remaining
terminals can be linked, directly or
indirectly, such that the total length (in
miles) of the selected tracks is minimized.
Determine the segments of the railroad
tracks that must be included in the
revitalization program.
59
2) A major hotel chain is constructing a new resort hotel complex in Greenbranch
Springs, West Virginia. The resort is in a heavily wooded area, and the
developers want to preserve as much of the natural beauty as possible. To do
so, the developers want to connect all the various facilities in the complex with
a combination walking–riding path that will minimize the amount of pathway
that will have to be cut through the woods. The following network shows
possible connecting paths and corresponding distances (in yards) between the
facilities. Determine the path that will connect all the facilities with the
minimum amount of construction and indicate the total length of the pathway.
60
3) The network below gives the
mileage of the feasible links
connecting nine offshore natural gas
wellheads with an inshore delivery
point. Because wellhead 1 is the
closest to shore, it is equipped with
sufficient pumping and storage
capacity to pump the output of the
remaining eight wells to the delivery
point. Determine the minimum
pipeline network that links the
wellheads to the delivery point.
61
3. Maximal flow problem
The maximal flow problem is to maximize the amount of flow of items
from an origin to a destination.
Maximal flow problems can involve the flow of water, gas, or oil
through a network of pipelines; the flow of traffic through a road
network; or the flow of products through a production line system.
In each of these examples, the branches of the network have limited
and often different flow capacities.
Given these conditions, the decision maker wants to determine the
maximum flow that can be obtained through the system.
62
The steps of the maximal flow solution method are as follows:
1) Arbitrarily select any path in the network from origin to
destination.
2) Adjust the capacities at each node by subtracting the maximal
flow for the path selected in step 1.
3) Add the maximal flow along the path in the opposite direction at
each node to indicate the maximum traffic at each node.
4) Repeat steps 1, 2, and 3 until there are no more paths with
available flow capacity.
63
Example:
Given the following network, with the indicated flow capacities of each
branch, determine the maximum flow from source node 1 to
destination node 6 and the flow along each branch:
64
Exercise 1:
Given the following network, with the indicated flow capacities of each
branch, determine the maximum flow from source node 1 to
destination node 6 and the flow along each branch:
65
Exercise 2:
The FAA has granted a license to a new airline, Omni air, and awarded it
several routes between Los Angeles and Chicago. The flights per day for
each route are shown in the following network. Determine the
maximum number of flights the airline can schedule per day from
Chicago to Los Angeles and indicate the number of flights along each
route.
66