C BYREGOWDA INSTITUTE OF TECHNOLOGY

Department of CSE & AIML

Module-4: Dynamic Programming

Contents

1. General method with Examples.

2. Multistage Graphs.
3. Transitive Closure:
 Warshall’s Algorithm.
4. All Pairs Shortest Paths:

 Floyd's Algorithm
5. Knapsack problem,
6. Bellman-Ford Algorithm,
7. Travelling Sales Person problem.
8. Space-Time Tradeoffs: Introduction,
 Sorting by Counting,
 Input Enhancement in String Matching Harspool’s algorithm.

Dept. of CSE, AIML, CBIT, Kolar Page 1

 DAA-21CS42—MODULE-4 2023

1. Introduction to Dynamic Programming

Dynamic programming is a technique for solving problems with overlapping sub
problems. Typically, these sub problems arise from a recurrence relating a given problem’s
solution to solutions of its smaller sub problems. Rather than solving overlapping sub
problems again and again, dynamic programming suggests solving each of the smaller sub
problems only once and recording the results in a table from which a solution to the original
problem can then be obtained.
The Dynamic programming can also be used when the solution to a problem can be viewed
as the result of sequence of decisions. Here are some examples.

Example 1

Example 2

Example 3

Example4

Dept. of CSE AIML, CBIT, Kolar Page 2

 DAA-21CS42—MODULE-4 2023

Dept. of CSE AIML, CBIT, Kolar Page 3

 DAA-21CS42—MODULE-4 2023

2. Multistage Graphs

Dept. of CSE AIML, CBIT, Kolar Page 4

 DAA-21CS42—MODULE-4 2023

Figure: Five stage graph

Dept. of CSE AIML, CBIT, Kolar Page 5

 DAA-21CS42—MODULE-4 2023

Dept. of CSE AIML, CBIT, Kolar Page 6

 DAA-21CS42—MODULE-4 2023

Backward Approach

Dept. of CSE AIML, CBIT, Kolar Page 7

 DAA-21CS42—MODULE-4 2023

3. Transitive Closure using Warshall’s Algorithm

Definition: The transitive closure of a directed graph with n vertices can be defined as then
× n Boolean matrix T = {tij }, in which the element in the ith row and the jth column is 1 if
there exists a nontrivial path (i.e., directed path of a positive length) from the i th vertex to the
jth vertex; otherwise, tij is 0.
Example: An example of a digraph, its adjacency matrix, and its transitive closure is given
below.

(a) Digraph. (b) Its adjacency matrix. (c) Its transitive closure.

We can generate the transitive closure of a digraph with the help of depth first search or
breadth-first search. Performing either traversal starting at the ith vertex gives the information
about the vertices reachable from it and hence the columns that contain 1’s in the ith row of
the transitive closure. Thus, doing such a traversal for every vertex as a starting point yields
the transitive closure in its entirety.
Since this method traverses the same digraph several times, we can use a better algorithm
called Warshall’s algorithm. Warshall’s algorithm constructs the transitive closure through
a series of n × n boolean matrices:

Each of these matrices provides certain information about directed paths in the digraph.
(k)in the ith row and jth column of matrix R(k) (i, j = 1, 2, . . . , n, k
Specifically, the element rij
= 0, 1, . . . , n) is equal to 1 if and only if there exists a directed path of a positive length from
the ith vertex to the jth vertex with each intermediate vertex, if any, numbered not higher than
k.
Thus, the series starts with R(0) , which does not allow any intermediate vertices in its paths;
hence, R(0) is nothing other than the adjacency matrix of the digraph. R(1) contains the
information about paths that can use the first vertex as intermediate. The last matrix in the
series, R(n) , reflects paths that can use all n vertices of the digraph as intermediate and hence
is nothing other than the digraph’s transitive closure.
This means that there exists a path from the ith vertex vi to the jth vertex vj with each
Intermediate vertex numbered not higher than k:
vi, a list of intermediate vertices each numbered not higher than k, vj . --- (*)
Two situations regarding this path are possible.

Dept. of CSE AIML, CBIT, Kolar Page 8

 DAA-21CS42—MODULE-4 2023

1. In the first, the list of its intermediate vertices does not contain the kth vertex. Then this
path from vi to vj has intermediate vertices numbered not higher than k−1. i.e. r(k–1)
ij =1
2. The second possibility is that path (*) does contain the kth vertex vk among the
intermediate vertices. Then path (*) can be rewritten as;
vi, vertices numbered ≤ k − 1, vk, vertices numbered ≤ k − 1, vj .
i.e r(k–1) = 1 and r(k–1) = 1
ik kj

Thus, we have the following formula for generating the elements of matrix R(k) from the
elements of matrix R(k−1)

The Warshall’s algorithm works based on the above formula.

As an example, the application of Warshall’s algorithm to the digraph is shown below. New
1’s are in bold.

Dept. of CSE AIML, CBIT, Kolar Page 9

 DAA-21CS42—MODULE-4 2023

Analysis
Its time efficiency is Θ(n3). We can make the algorithm to run faster by treating matrix rows
as bit strings and employ the bitwise or operation available in most Modern computer
languages.
Space efficiency: Although separate matrices for recording intermediate results of the algorithm
are used, that can be avoided.

4. All Pairs Shortest Paths using Floyd's Algorithm

Problem definition: Given a weighted connected graph (undirected or directed), the all-pairs
shortest paths problem asks to find the distances—i.e., the lengths of the shortest paths - from
each vertex to all other vertices.
Applications: Solution to this problem finds applications in communications, transportation
networks, and operations research. Among recent applications of the all-pairs shortest-path
problem is pre-computing distances for motion planning in computer games.
We store the lengths of shortest paths in an n x n matrix D called the distance matrix: the
element dij in the ith row and the jth column of this matrix indicates the length of the shortest
path from the ith vertex to the jth vertex.

(a) Digraph. (b) Its weight matrix. (c) Its

distance matrix
We can generate the distance matrix with an algorithm that is very similar to Warshall’s
algorithm. It is called Floyd’s algorithm.
Floyd’s algorithm computes the distance matrix of a weighted graph with n vertices through a
series of n × n matrices:

Dept of CSE AIML,CBIT, Kolar Page 10

 DAA-21CS42—MODULE-4 2023

The element d(k)in the ith row and the jth column of matrix D(k) (i, j = 1, 2, . . .,n, k = 0,1,
. . . , n) is equal to the length of the shortest path among all paths from the i vertex to the jth
th

vertex with each intermediate vertex, if any, numbered not higher than k.
As in Warshall’s algorithm, we can compute all the elements of each matrix D(k) from its
immediate predecessor D(k−1)

If d(k)
ij = 1, then it means that there is a path;

vi, a list of intermediate vertices each numbered not higher than k, vj .

We can partition all such paths into two disjoint subsets: those that do not use the kth vertex vk
as intermediate and those that do.
i. Since the paths of the first subset have their intermediate vertices numbered not higher
than k − 1, the shortest of them is, by definition of our matrices, of lengthd(k–1)
ij
ii. In the second subset the paths are of theform
vi, vertices numbered ≤ k − 1, vk, vertices numbered ≤ k − 1, vj .

Taking into account the lengths of the shortest paths in both subsets leads to the following
recurrence:

Analysis: Its time efficiency is Θ(n3), similar to the warshall’s algorithm.

Dept of CSE AIML,CBIT, Kolar Page 11

 DAA-21CS42—MODULE-4 2023

Application of Floyd’s algorithm to the digraph is shown below. Updated elements are shown
in bold.

5. Knapsack problem
We start this section with designing a dynamic programming algorithm for the knapsack
problem: given n items of known weights w1, . . . , wn and values v1, . . . , vn and a knapsack
of capacity W, find the most valuable subset of the items that fit into theknapsack.
To design a dynamic programming algorithm, we need to derive a recurrence relation that
Expresses a solution to an instance of the knapsack problem in terms of solutions to its
smaller sub instances.

Dept of CSE AIML,CBIT, Kolar Page 12

 DAA-21CS42—MODULE-4 2023

Let us consider an instance defined by the first i items, 1≤ i ≤ n, with weights w 1, . . . , wi,
values v1, . . . , vi , and knapsack capacity j, 1 ≤ j ≤ W. Let F(i, j) be the value of an optimal
solution to this instance. We can divide all the subsets of the first i items that fit the knapsack
of capacity j into two categories: those that do not include the ith item and those that do. Note
the following:
i. Among the subsets that do not include the ith item, the value of an optimal subset is,
by definition, F(i − 1,j).
ii. Among the subsets that do include the ith item (hence, j − wi ≥ 0), an optimal subset is
made up of this item and an optimal subset of the first i−1 items that fits into the
knapsack of capacity j − wi . The value of such an optimal subset is vi + F(i − 1, j −
wi).

Thus, the value of an optimal solution among all feasible subsets of the first I items is the
maximum of these two values.

It is convenient to define the initial conditions as follows:

F(0, j) = 0 for j ≥ 0 and F(i, 0) = 0 for i ≥ 0.
Our goal is to find F(n, W), the maximal value of a subset of the n given items that fit into
the knapsack of capacity W, and an optimal subsetitself.

Example-1: Let us consider the instance given by the following data:

The dynamic programming table, filled by applying formulas is given below

Dept of CSE AIML,CBIT, Kolar Page 13

 DAA-21CS42—MODULE-4 2023

Thus, the maximal value is F(4, 5) = $37.

We can find the composition of an optimal subset by backtracing the computations of this
entry in the table. Since F(4, 5) > F(3, 5), item 4 has to be included in an optimal solution
along with an optimal subset for filling 5 − 2 = 3 remaining units of the knapsack capacity.
The value of the latter is F(3, 3). Since F(3, 3) = F(2, 3), item 3 need not be in an optimal
subset. Since F(2, 3) > F(1, 3), item 2 is a part of an optimal selection, which leaves element
F(1, 3 − 1) to specify its remaining composition. Similarly, since F(1, 2) > F(0, 2), item 1 is
the final part of the optimal solution {item 1, item 2, item 4}.

Analysis
The time efficiency and space efficiency of this algorithm are both in Θ(nW). The time
needed to find the composition of an optimal solution is in O(n).

Memory Functions
The direct top-down approach to finding a solution to such a recurrence leads to an algorithm
that solves common sub problems more than once and hence is very inefficient.
The classic dynamic programming approach, on the other hand, works bottom up: it fills a
table with solutions to all smaller sub problems, but each of them is solved only once. An
unsatisfying aspect of this approach is that solutions to some of these smaller sub problems
are often not necessary for getting a solution to the problem given. Since this drawback is not
present in the top-down approach, it is natural to try to combine the strengths of the top-down
and bottom-up approaches. The goal is to get a method that solves only sub problems that are
necessary and does so only once. Such a method exists; it is based on using memory
functions.

Dept of CSE AIML,CBIT, Kolar Page 14

 DAA-21CS42—MODULE-4 2023

The following algorithm implements this idea for the knapsack problem. After initializing the
table, the recursive function needs to be called with i = n (the number of items) and j = W
(the knapsackcapacity).
Algorithm MFKnapsack(i, j )
//Implements the memory function method for the knapsack problem
//Input: A nonnegative integer i indicating the number of the first items being
considered and a nonnegative integer j indicating the knapsack capacity
//Output: The value of an optimal feasible subset of the first i items
//Note: Uses as global variables input arrays Weights[1..n], V alues[1..n], and
table F[0..n, 0..W ] whose entries are initialized with −1’s except for
row 0 and column 0 initialized with 0’s

Example-2 Let us apply the memory function method to the instance considered in Example
1. The table in Figure given below gives the results. Only 11 out of 20 nontrivial values (i.e.,
not those in row 0 or in column 0) have been computed. Just one nontrivial entry, V (1, 2), is
retrieved rather than being recomputed. For larger instances, the proportion of such entries
can be significantlylarger.

Figure: Example of solving an instance of the knapsack problem by the memory function algorithm

In general, we cannot expect more than a constant-factor gain in using the memory function
method for the knapsack problem, because its time efficiency class is the same as that of the
bottom-up algorithm.

Dept of CSE AIML,CBIT, Kolar Page 15

 DAA-21CS42—MODULE-4 2023

6. Bellman-Ford Algorithm (Single source shortest path with –ve weights)

Problem definition
Single source shortest path - Given a graph and a source vertex s in graph, find shortest paths
from s to all vertices in the given graph. The graph may contain negative weight edges.
Note that we have discussed Dijkstra’s algorithm for single source shortest path problem.
Dijksra’s algorithm is a Greedy algorithm and time complexity is O(VlogV). But Dijkstra
doesn’t work for graphs with negative weight edges.
Bellman-Ford works for such graphs. Bellman-Ford is also simpler than Dijkstra and suites
well for distributed systems. But time complexity of Bellman-Ford is O(VE), which is more
than Dijkstra.
How it works?
Like other Dynamic Programming Problems, the algorithm calculates shortest paths in
bottom-up manner. It first calculates the shortest distances for the shortest paths which have
at-most one edge in the path. Then, it calculates shortest paths with at-most 2 edges, and so
on. After the ith iteration of outer loop, the shortest paths with at most i edges are calculated.
There can be maximum |V| – 1 edges in any simple path, that is why the outer loop runs |v| –
1 times. The idea is, assuming that there is no negative weight cycle, if we have calculated
shortest paths with at most i edges, then an iteration over all edges guarantees to give shortest
path with at-most (i+1)edges

Bellman-Ford algorithm to compute shortest path

Dept of CSE AIML,CBIT, Kolar Page 16

 DAA-21CS42—MODULE-4 2023

Dept. of CSE, AIML, CBIT, Kolar Page 17

 DAA-21CS42—MODULE-4 2023

7. Travelling Sales Person problem (T2:5.9),

Dept. of CSE, AIML, CBIT, Kolar Page 18

 DAA-21CS42—MODULE-4 2023

Dept. of CSE, AIML, CBIT, Kolar Page 19

 DAA-21CS42—MODULE-4 2023

Dept. of CSE, AIML, CBIT, Kolar Page 20

 DAA-21CS42—MODULE-4 2023

8. Space and time trade-offs

Space and time trade-offs in algorithm design are a well-known issue for both theoreticians and
practitioners of computing. Consider, as an example, the problem of computing values of a function
at many points in its domain. If it is time that is at a premium, we can pre-compute the function’s
values and store them in a table. This is exactly what human computers had to do before the advent
of electronic computers, in the process burdening libraries with thick volumes of mathematical
tables.

8.1. Sorting by counting

As a first example of applying the input-enhancement technique, we discuss its application to

the sorting problem.

Comparison Counting Sort

One rather obvious idea is to count, for each element of a list to be sorted, the total number of
elements smaller than this element and record the results in a table. These numbers will indicate
the positions of the elements in the sorted list: e.g., if the count is 10 for some element, it should
be in the 11th position (with index 10, if we start counting with 0) in the sorted array. Thus, we
will be able to sort the list by simply copying its elements to their appropriate positions in a new,
sorted list. This algorithm is called comparison counting sort. As shown in figure.

What is the time efficiency of this algorithm? It should be quadratic because the algorithm
considers all the different pairs of an n-element array. More formally, the number of times its
basic operation, the comparison A[i] < A[j ] , is executed is equal to the sum we have encountered
several times already:

Dept. of CSE, AIML, CBIT, Kolar Page 21

 DAA-21CS42—MODULE-4 2023

Thus, the algorithm makes the same number of key comparisons as selection sort and in addition
uses a linear amount of extra space. On the positive side, the algorithm makes the minimum
number of key moves possible, placing each of them directly in their final position in a sorted
array.

The counting idea does work productively in a situation in which elements to be sorted
belong to a known small set of values. Assume, for example, that we have to sort a list whose
values can be either 1 or 2. Rather than applying a general sorting algorithm, we should be able to
take advantage of this additional information about values to be sorted.

Distribution Counting

Let us consider a more realistic situation of sorting a list of items with some other
information associated with their keys so that we cannot overwrite the list’s elements. Then we
can copy elements into a new array S[0..n−1]to hold the sorted list as follows. The elements of A
whose values are equal to the lowest possible value l are copied into the first F[0]elements of S,
i.e., positions 0 through F[0]− 1; the elements of value l + 1 are copied to positions from F[0] to
(F[0]+ F[1]) − 1; and so on. Since such accumulated sums of frequencies are called a distribution
in statistics, the method itself is known as distribution counting.

Note that the distribution values indicate the proper positions for the last occurrences of their
elements in the final sorted array. If we index array positions from 0 to n − 1, the distribution
values must be reduced by 1 to get corresponding element positions.

Dept. of CSE, AIML, CBIT, Kolar Page 22

 DAA-21CS42—MODULE-4 2023

It is more convenient to process the input array right to left. For the example, the last
element is 12, and, since its distribution value is 4, we place this 12 in position 4 − 1= 3 of the
array S that will hold the sorted list. Then we decrease the 12’s distribution value by 1 and
proceed to the next (from the right) element in the given array. The entire processing of this
example is depicted in Figure.

Assuming that the range of array values is fixed, this is obviously a linear algorithm
because it makes just two consecutive passes through its input array A. This is a better time-

Dept. of CSE, AIML, CBIT, Kolar Page 23

 DAA-21CS42—MODULE-4 2023

efficiency class than that of the most efficient sorting algorithms—mergesort, quicksort, and
heapsort—we have encountered. It is important to remember, however, that this efficiency is
obtained by exploiting the specific nature of inputs for which sorting by distribution counting
works, in addition to trading space for time.

8.2. Input Enhancement in String Matching:

In this section, we see how the technique of input enhancement can be applied to the problem
of string matching. The brute-force algorithm for this problem in Section 3.2: it simply matches
corresponding pairs of characters in the pattern and the text left to right and, if a mismatch occurs,
shifts the pattern one position to the right for the next trial. Since the maximum number of such
trials is n − m + 1 and, in the worst case, m comparisons need to be made on each of them, the
worst-case efficiency of the brute-force algorithm is in the O(nm) class.

Several faster algorithms have been discovered. Most of them exploit the input-enhancement
idea: pre-process the pattern to get some information about it, store this information in a table, and
then use this information during an actual search for the pattern in a given text. This is exactly the
idea behind the two best known algorithms of this type: the Knuth-Morris-Pratt algorithm and the
Boyer-Moore algorithm.

Horspool’s Algorithm:

Consider, as an example, searching for the pattern BARBER in some text:

s0 . . . c . . . sn−1
BARBER

 Starting with the last R of the pattern and moving right to left, we compare the
corresponding pairs of characters in the pattern and the text.
 If all the pattern’s characters match successfully, a matching substring is found. Then the
search can be either stopped altogether or continued if another occurrence of the same
pattern is desired.
 If a mismatch occurs, we need to shift the pattern to the right. Clearly, we would like to
make as large a shift as possible without risking the possibility of missing a matching
substring in the text.
 Horspool’s algorithm determines the size of such a shift by looking at the character c of
the text that is aligned against the last character of the pattern.

In general, the following four possibilities can occur.

Case 1 If there are no c’s in the pattern—e.g., c is letter S in our example— we can safely shift
the pattern by its entire length (if we shift less, some character of the pattern would be aligned
against the text’s character c that is known not to be in the pattern):

Dept. of CSE, AIML, CBIT, Kolar Page 24

 DAA-21CS42—MODULE-4 2023

Case 2 If there are occurrences of character c in the pattern but it is not the last one there—e.g., c
is letter B in our example—the shift should align the rightmost occurrence of c in the pattern with
the c in the text:

Case 3 If c happens to be the last character in the pattern but there are no c’s among its other
m − 1 characters—e.g., c is letter R in our example—the situation is similar to that of Case 1 and
the pattern should be shifted by the entire pattern’s length m:

Case 4 Finally, if c happens to be the last character in the pattern and there are other c’s among its
first m − 1 characters—e.g., c is letter R in our example— the situation is similar to that of Case 2
and the rightmost occurrence of c among the first m − 1 characters in the pattern should be aligned
with the text’s c:

These examples clearly demonstrate that right-to-left character comparisons can lead to
farther shifts of the pattern than the shifts by only one position always made by the brute-force
algorithm.

Fortunately, the idea of input enhancement makes repetitive comparisons unnecessary. We

can pre-compute shift sizes and store them in a table. The table will be indexed by all possible
characters that can be encountered in a text, including, for natural language texts, the space,
punctuation symbols, and other special characters.

The table’s entries will indicate the shift sizes computed by the formula.

For example, for the pattern BARBER, all the table’s entries will be equal to 6, except for
the entries for E, B, R, and A, which will be 1, 2, 3, and 4, respectively.

Dept. of CSE, AIML, CBIT, Kolar Page 25

 DAA-21CS42—MODULE-4 2023

Here is a simple algorithm for computing the shift table entries. Initialize all the entries to
the pattern’s length m and scan the pattern left to right repeating the following step m − 1 times:
for the jth character of the pattern (0 ≤ j ≤ m − 2), overwrite its entry in the table with m − 1− j ,
which is the character’s distance to the last character of the pattern.

Note, that since the algorithm scans the pattern from left to right, the last overwrite will
happen for the character’s rightmost occurrence—exactly as we would like it to be.

Horspool’s algorithm:

Step 1: For a given pattern of length m and the alphabet used in both the pattern and text,
construct the shift table as described above.

Step 2 Align the pattern against the beginning of the text.

Step 3 Repeat the following until either a matching substring is found or the pattern reaches
beyond the last character of the text. Starting with the last character in the pattern, compare the
corresponding characters in the pattern and text until either all m characters are matched (then
stop) or a mismatching pair is encountered. In the latter case, retrieve the entry t (c) from the c’s
column of the shift table where c is the text’s character currently aligned against the last character
of the pattern, and shift the pattern by t (c) characters to the right along the text.

Dept. of CSE, AIML, CBIT, Kolar Page 26

 DAA-21CS42—MODULE-4 2023

EXAMPLE As an example of a complete application of Horspool’s algorithm, consider

searching for the pattern BARBER in a text that comprises English letters and spaces (denoted by
underscores). The shift table, as we mentioned, is filled as follows:

A simple example can demonstrate that the worst-case efficiency of Horspool’s algorithm
is in O(nm) (Problem 4 in this section’s exercises). But for random texts, it is in _(n), and,
although in the same efficiency class, Horspool’s algorithm is obviously faster on average than
the brute-force algorithm.

***********************************************

Dept. of CSE, AIML, CBIT, Kolar Page 27