Design and Analysis of Algorithms

Unit -4
Bharathi R
Department of Computer Science & Engineering
DESIGN AND ANALYSIS OF ALGORITHMS

Unit 4: Space and Time Tradeoffs

Space and Time Tradeoffs - Sorting by Counting

Bharathi R
Department of Computer Science & Engineering
 Design and Analysis of Algorithms
Space and Time Tradeoffs- Introduction

• Space and time trade-offs in algorithm design are a well-

known issue for both theoreticians and practitioners of
computing.

• As an algorithm design technique, trading space for

time is much more prevalent than trading time for
space.
Design and Analysis of Algorithms

Principal Varieties for trading space for time in Algorithm design

Achieving time efficiency in

computation

1.Input enhancement 2.Prestructuring 3.Dynamic

programming

a) Hashing b) B-Trees
a) Counting b) String
Methods for matching
Sorting Algorithm
Design and Analysis of Algorithms

1.Input enhancement

● Input Enhancement 1.Input enhancement

 Preprocess the problem’s input, in
whole or in part, and store the
additional information obtained a) Counting b) String
to accelerate solving the problem Methods for matching
Sorting Algorithm
afterward.
 Eg:
1. Comparison counting sort
2. Distribution Counting,
3. Horspool’s algorithm,
4. Boyer-Moore’s algorithm
Design and Analysis of Algorithms
1.Input enhancement
a)Sorting by Counting
1. Comparison Counting Sorting
I. For each element of the list, count the total number of
elements smaller than this element.
II. These numbers will indicate the positions of the elements in
the sorted list.

2. Distribution Counting Sorting

I. Suppose the elements of the list to be sorted belong to a finite
set (aka domain).
II. Count the frequency of each element of the set in the list to
be sorted.
III. Scan the set in order of sorting and print each element of the
set according to its frequency, which will be the required
sorted list.
Design and Analysis of Algorithms
1.Input Enhancement
→ Sorting by Counting
→→ Comparison Counting Sorting

1. Find the numbers that are less than a[0] i.e, 62, by scanning the array from the index 1 to 5

a[0] a[1] a[2] a[3] a[4] a[5]

Array a 62 31 84 96 19 47

2. Maintain another array called Count the elements that are lesser than 62

a[0] a[1] a[2] a[3] a[4] a[5]

Array a 62 31 84 96 19 47
Array 3
Count
Design and Analysis of Algorithms

Example of sorting by comparison counting

Design and Analysis of Algorithms

Algorithm for Sorting by Counting

Design and Analysis of Algorithms
Time Efficiency

It should be quadratic because the algorithm considers all the

different pairs of an n-element array
Design and Analysis of Algorithms
Drawbacks

1. Thus, the algorithm makes the same

number of key comparisons as selection
sort,
2. In addition it uses a linear amount of extra
space.
Design and Analysis of Algorithms
References

“Introduction to the Design and Analysis of Algorithms”, Anany Levitin,

Pearson Education, Delhi (Indian Version), 3rd edition, 2012.
Chapter- 7
THANK YOU

Bharathi R
Department of Computer Science & Engineering
[email protected]
Design and Analysis of Algorithms
Unit -4
Bharathi R
Department of Computer Science & Engineering
DESIGN AND ANALYSIS OF ALGORITHMS

Unit 4: Space and Time Tradeoffs

Distribution Counting Sort

Slides from Anany Levitin

Bharathi R
Department of Computer Science & Engineering
Design and Analysis of Algorithms
1.Input enhancement
a)Sorting by Counting
1. Comparison Counting Sorting
I. For each element of the list, count the total number of
elements smaller than this element.
II. These numbers will indicate the positions of the elements in
the sorted list.

2. Distribution Counting Sorting

I. Suppose the elements of the list to be sorted belong to a finite
set (aka domain).
II. Count the frequency of each element of the set in the list to
be sorted.
III. Scan the set in order of sorting and print each element of the
set according to its frequency, which will be the required
sorted list.
Design and Analysis of Algorithms
1. Input Enhancement
→ Sorting by Counting →→ ii )Distribution Counting Sorting

• A sorting method in which, with the help of some associated

information of the elements , the elements can be placed in an
array at their relative positions.

• The required information which is used to place the elements at

proper positions is accumulated sum of frequencies which is
also called as distribution in statistics.

• Hence this method is called as Distribution counting method for

sorting.
Design and Analysis of Algorithms
1. Input Enhancement
→ Sorting by Counting →→ ii )Distribution Counting Sorting

Consider sorting the array whose values are known to come from the set
{11, 12, 13} and should not be overwritten in the process of sorting. The
frequency and distribution arrays are as follows:

=
Design and Analysis of Algorithms
Example of sorting by distribution counting.

The distribution values being

decremented are shown in bold.
Design and Analysis of Algorithms
Algorithm for Distribution Counting
Design and Analysis of Algorithms
Distribution Counting Sorting

1. This is a better time-efficiency class than that of the most

efficient sorting algorithms—mergesort, quicksort, and
heapsort—we have encountered.

2. It is important to remember, however, that this efficiency is

obtained by exploiting the specific nature of inputs for which
sorting by distribution counting works, in addition to trading
space for time.
Design and Analysis of Algorithms
References

“Introduction to the Design and Analysis of Algorithms”, Anany Levitin,

Pearson Education, Delhi (Indian Version), 3rd edition, 2012. Chapter- 7
THANK YOU

Bharathi R
Department of Computer Science & Engineering
[email protected]
Design and Analysis of Algorithms
Unit -4
Bharathi R
Department of Computer Science & Engineering
DESIGN AND ANALYSIS OF ALGORITHMS

Unit 4: Space and Time Tradeoffs

Input Enhancement in String Matching

Bharathi R
Department of Computer Science & Engineering
Design and Analysis of Algorithms
String Matching

String matching requires finding an occurrence of a given string of m

characters called the pattern in a longer string of n characters called
the text.
 Design and Analysis of Algorithms
Input Enhancement in String Matching

Input-enhancement idea: 1.Input enhancement

Preprocess the pattern to get
some information about it, store
this information in a table, and
b) String matching
then use this information during Algorithm
an actual search for the pattern
in a given text.
1. Horspool’s Algorithm 2. Boyer-Moore Algorithm
 Design and Analysis of Algorithms
String matching by Brute force

pattern: a string of m characters to search for

text: a (long) string of n characters to search in

Brute force algorithm

Step 1 Align pattern at beginning of text

Step 2 Moving from left to right, compare each character of pattern to

the corresponding character in text until either all characters are
found to match (successful search) or a mismatch is detected

Step 3 While a mismatch is detected and the text is not yet exhausted,
realign pattern one position to the right and repeat Step 2

Time complexity (worst-case): O(mn)

 Design and Analysis of Algorithms
String matching by preprocessing

Several string searching algorithms are based on the input enhancement

idea of preprocessing the pattern

1. Knuth-Morris-Pratt (KMP) algorithm preprocesses pattern left to

right to get useful information for later searching

2. Boyer -Moore algorithm preprocesses pattern right to left and store

information into two tables

3. Horspool’s algorithm simplifies the Boyer-Moore algorithm by using

just one table
 Design and Analysis of Algorithms
Horspool’s Algorithm

A simplified version of Boyer-Moore algorithm:

Preprocesses pattern to generate a “shift table” that determines how

much to shift the pattern when a mismatch occurs

Always makes a shift based on the text’s character c aligned with the
last compared (mismatched) character in the pattern according to the
shift table’s entry for c
Design and Analysis of Algorithms
How far to shift?

Look at first (rightmost) character in text that was compared:

1. The character is not in the pattern
........c...................... (c not in pattern)
≠
BAOBAB

2. The character is in the pattern (but not the rightmost)

.....O...................... (O occurs once in pattern)
BAOBAB

.....A...................... (A occurs twice in pattern)

BAOBAB

3. The rightmost characters do match

.....B......................
BAOBAB
Design and Analysis of Algorithms
Shift Table

Shift sizes can be precomputed by the formula

By scanning pattern before search begins and stored in a table called shift
table. After the shift, the right end of pattern is t(c) positions to the right of
the last compared character in text.

Shift table is indexed by text and pattern alphabet Eg: for BAOBAB:

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

1 2 6 6 6 6 6 6 6 6 6 6 6 6 3 6 6 6 6 6 6 6 6 6 6 6
Design and Analysis of Algorithms
Shift Table
Design and Analysis of Algorithms
Horspool Matching
Design and Analysis of Algorithms
Horspool Matching -Example
Design and Analysis of Algorithms
Example of Horspool’s algorithm

_
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

1 2 6 6 6 6 6 6 6 6 6 6 6 6 3 6 6 6 6 6 6 6 6 6 6 6 6

BARD LOVED BANANAS

BAOBAB
BAOBAB
BAOBAB
BAOBAB (unsuccessful search)
Design and Analysis of Algorithms
References

“Introduction to the Design and Analysis of Algorithms”, Anany

Levitin, Pearson Education, Delhi (Indian Version), 3rd edition,
2012. Chapter- 7
THANK YOU

Bharathi R
Department of Computer Science & Engineering
[email protected]
Design and Analysis of Algorithms
Unit -4
Bharathi R
Department of Computer Science & Engineering
DESIGN AND ANALYSIS OF ALGORITHMS

Unit 4: Space and Time Tradeoffs

Input Enhancement in String Matching- The Boyer-Moore Algorithm

Bharathi R
Department of Computer Science & Engineering
 Design and Analysis of Algorithms
The Boyer-Moore algorithm

Based on the same two ideas:

comparing pattern characters to text from right to left

precomputing shift sizes in two tables

1. bad-symbol table indicates how much to shift based on

text’s character causing a mismatch

2. good-suffix table indicates how much to shift based on

matched part (suffix) of the pattern (taking advantage of
the periodic structure of the pattern)
Design and Analysis of Algorithms
Bad-symbol shift in Boyer-Moore algorithm

1. If the rightmost character of the pattern doesn’t match, BM algorithm acts

as Horspool’s
2. If the rightmost character of the pattern does match, BM compares
preceding characters right to left until either all pattern’s characters match
or a mismatch on text’s character c is encountered after k > 0 matches

In this situation, the Boyer-Moore algorithm determines the shift size by considering
two quantities. The first one is guided by the text’s character c that caused a
mismatch with its counterpart in the pattern. Accordingly, it is called the bad symbol
shift.
Design and Analysis of Algorithms
Bad-symbol shift in Boyer-Moore algorithm

If c is not in the pattern, we shift the pattern to just pass this c in the text.
Conveniently, the size of this shift can be computed by the formula
t1(c) − k where t1(c) is the entry in the precomputed table used by
Horspool’s algorithm and k is the number of matched characters:
Design and Analysis of Algorithms
Bad-symbol shift in Boyer-Moore algorithm- Example
Design and Analysis of Algorithms
Bad-symbol shift in Boyer-Moore algorithm- Example
 Design and Analysis of Algorithms
Good-suffix shift in Boyer-Moore algorithm and when to use?

Erroneous shift will happen.

To avoid that a shift based on matched suffix of the pattern.
Called “Good Suffix shift”. It is denoted by d2.
 Design and Analysis of Algorithms
Good-suffix shift in Boyer-Moore algorithm

d2(k) = Distance between matched suffix of size k and its rightmost

occurrence in the pattern that is not preceded by the same character as the suffix
 Design and Analysis of Algorithms
Good-suffix shift in Boyer-Moore algorithm

• Good-suffix shift d2 is applied after 0 < k < m last characters were

matched

• d2(k) = the distance between (the last letter of) the matched
suffix of size k and (the last letter of ) its rightmost occurrence in
the pattern that is not preceded by the same character
preceding the suffix

Example: CABABA d2(1) = 4

• If there is no such occurrence, match the longest part (tail) of the

k-character suffix with corresponding prefix;
if there are no such suffix-prefix matches, d2 (k) = m

Example: WOWWOW d2(2) = 5, d2(3) = 3, d2(4) = 3, d2(5) = 3

 Design and Analysis of Algorithms
Boyer-Moore algorithm

Step 1 Fill in the bad-symbol shift table

Step 2 Fill in the good-suffix shift table
Step 3 Align the pattern against the beginning of the text
Step 4 Repeat until a matching substring is found or text ends:
Compare the corresponding characters right to left.
If no characters match, retrieve entry t1(c) from the bad-
symbol table for the text’s character c causing the mismatch
and shift the pattern to the right by t1(c).
If 0 < k < m characters are matched, retrieve entry t1(c) from
the bad-symbol table for the text’s character c causing the
mismatch and entry d2(k) from the good-suffix table and shift
the pattern to the right by
d = max {d1, d2}
where d1 = max{t1(c) - k, 1}.
Design and Analysis of Algorithms
Example of Boyer-Moore alg. application
Design and Analysis of Algorithms
Analysis

• The worst-case efficiency of the Boyer-Moore algorithm is known to

be linear.
• Though this algorithm runs very fast, especially on large alphabets
(relative to the length of the pattern),many people prefer its
simplified versions, such as Horspool’s algorithm, when dealing with
natural-language–like strings.
 Design and Analysis of Algorithms
References

Chapter 7 ,Introduction to The Design and Analysis of Algorithms by

Anany Levitin
THANK YOU

Bharathi R
Department of Computer Science & Engineering
[email protected]
Design and Analysis of Algorithms
Unit -4
Bharathi R
Department of Computer Science & Engineering
DESIGN AND ANALYSIS OF ALGORITHMS

Unit 4: Greedy Technique

Bharathi R
Department of Computer Science & Engineering
Design and Analysis of Algorithms
The Greedy Approach

The greedy approach suggests constructing a solution through a

sequence of steps, each expanding a partially constructed solution
obtained so far, until a complete solution to the problem is reached.
On each step, the choice made must be:

-feasible: it has to satisfy the problem’s constraints

-locally optimal: it has to be the best local choice among all
feasible choices available on that step
- irrevocable: once decision was made, it cannot be changed on
subsequent steps of the algorithm
Design and Analysis of Algorithms
Examples of Greedy Algorithms:

● Coin-change problem

● Minimum Spanning Tree (MST)

○ Prim’s Algorithm
○ Kruskal’s Algorithm

● Single-source shortest paths

○ Dijkstra’s Algorithm

● Huffman codes
Design and Analysis of Algorithms
Coin Change Problem

A greedy algorithm to find the minimum number of coins

for making the change of a given amount of money.
Usually, this problem is referred to as the change-making
problem.
 Design and Analysis of Algorithms
Change Making Problem

• In the change-making problem, we’re provided with an array, D = { d1, d2,

d3,…..dm} of m distinct coin denominations.

• Now we need to find an array(subset) s having minimum number of coins that

add up to a given amount of money n, provided that there exists a viable
solution.

• Let’s consider a real-life example for a better understanding of the change-

making problem.

• Let’s assume that we’re working at a cash counter and have an infinite supply of
D ={1,2,5,10} valued coins.

• A person buys things worth Rs. 72 and gives a Rs. 100 bill. How does the cashier
give change for Rs. 28?
 Design and Analysis of Algorithms
Change Making Problem

Change-making problem:
How can a given amount of money be
made with the least number of coins
of given denominations?
Option Choosen Coins
Example:
Change for Rs. 28 28-10 = 18 10

18-10 =8 10 10

8-5 =3 10 10 5

3-2 = 1 10 10 5 2

1-1 =0 10 10 5 2 1
Design and Analysis of Algorithms
Change Making Problem

Iteration 1 Iteration 5
D ={1,2,5,10} D ={1,2,5,10}
n = 28 and i=4 n = 1 and i=0
Select D[3] =10 as 28>= 10 Select D[0] =1 as 1>= 1
Decrease n by 10 ( n= 18) Decrease n by 1 ( n= 0)
Add 10 to set S = {10} Add 1 to set S = {10, 10, 5,2,1}
Iteration 2
Iteration 4
D ={1,2,5,10}
D ={1,2,5,10}
n = 28 and i=3
n = 3 and i=1
Select D[3] =10 as 18>= 10
Select D[1] =2 as 3>= 2
Decrease n by 10 ( n= 8)
Decrease n by 2 ( n= 1)
Add 10 to set S = {10, 10}
Add 2 to set S = {10, 10, 5,2}
Iteration 3
D ={1,2,5,10}
n = 8 and i=2
Select D[2] =5 as 8>= 5
Decrease n by 5 ( n= 3)
Add 5 to set S = {10, 10,5}
Design and Analysis of Algorithms
Summary

The greedy technique suggests constructing a solution to an

optimization problem through a sequence of steps, each expanding a
partially constructed solution obtained so far, until a complete
solution to the problem is reached.

On each step, the choice made must be feasible, locally optimal, and
irrevocable.
Design and Analysis of Algorithms
Text Books

Chapter 9 ,Introduction to The Design and Analysis of Algorithms by Anany Levitin

THANK YOU

Bharathi R
Department of Computer Science & Engineering
[email protected]
Design and Analysis of Algorithms
Unit -4
Bharathi R
Department of Computer Science & Engineering
DESIGN AND ANALYSIS OF ALGORITHMS

Unit 4: Greedy Technique

Prim’s algorithm

Bharathi R
Department of Computer Science & Engineering
Design and Analysis of Algorithms
Minimum Spanning Tree : DEFINITIONs

A spanning tree of an undirected connected graph is its connected

acyclic subgraph (i.e., a tree) that contains all the vertices of the
graph.

Example

Minimum Spanning Tree (MST) of a weighted,

connected graph G is a spanning tree of G with
minimum total weight.

Minimum Spanning Tree

Design and Analysis of Algorithms
Prim’s Algorithm : Greedy Approach

● Start with a tree, T1, consisting of one vertex (V).

● Adjacent vertices of the vertex in T1 are “fringe” vertices of T1.
● For i = 1 to |V|-1 do
○ Construct Ti from Ti-1 by adding the fringe vertex with the
minimum weight edge from the set. The vertex is removed
from the set of fringe vertices.
○ Add the adjacent vertices of the vertex to the set of fringe
vertices which are not in Ti.
○ Remove vertices from the set of fringe vertices where the
new vertex is one of the terminal vertex of the edge.
● Return Tn which is a minimum spanning tree.
Design and Analysis of Algorithms
Prim’s Algorithm

a
Design and Analysis of Algorithms
Prim’s Algorithm
Design and Analysis of Algorithms
Prim’s Algorithm
Design and Analysis of Algorithms
Prim’s Algorithm
Design and Analysis of Algorithms
Prim’s Algorithm
Design and Analysis of Algorithms
How efficient is Prim’s Algorithm

• The answer depends on the data structures chosen for the graph
itself and for the priority queue of the set V − VT whose vertex
priorities are the distances to the nearest tree vertices.

• If a graph is represented by its weight matrix and the priority

queue is implemented as an unordered array, the algorithm’s
running time will be in

• If a graph is represented by its adjacency lists and the priority

queue is implemented as a min-heap, the running time of the
algorithm is in O(|E| log |V |).
Design and Analysis of Algorithms
Summary

Prim's algorithm is very similar to Kruskal's: whereas Kruskal's

"grows" a forest of trees, Prim's algorithm grows a single tree until it
becomes the minimum spanning tree.

Both algorithms use the greedy approach - they add the cheapest
edge that will not cause a cycle. But rather than choosing the
cheapest edge that will connect any pair of trees together, Prim's
algorithm only adds edges that join nodes to the existing tree.

(In this respect, Prim's algorithm is very similar to Dijkstra's

algorithm for finding shortest paths.)
Design and Analysis of Algorithms
Text Books

Chapter 9 ,Introduction to The Design and Analysis of Algorithms by Anany Levitin

THANK YOU

Bharathi R
Department of Computer Science & Engineering
[email protected]
Design and Analysis of Algorithms
Unit -4
Bharathi R
Department of Computer Science & Engineering
DESIGN AND ANALYSIS OF ALGORITHMS

Unit 4: Greedy Technique

Bharathi R
Department of Computer Science & Engineering
Design and Analysis of Algorithms
Minimum Spanning Tree : Kruskal’s Algorithm

This algorithm was described by Joseph Kruskal in 1956.

Prim’s algorithm and Kruskal’s algorithm solve the same problem

( Minimum Spanning Tree) by applying the greedy approach in two
different ways, and both of them always yield an optimal solution.
Design and Analysis of Algorithms
Minimum Spanning Tree : Kruskal’s Algorithm

Kruskal's algorithm initially places all the nodes of the original graph
isolated from each other, to form a forest of single node trees, and
then gradually merges these trees, combining at each iteration any
two of all the trees with some edge of the original graph.

Before the execution of the algorithm, all edges are sorted by weight
(in non-decreasing order).

Then begins the process of unification: pick all edges from the first to
the last (in sorted order), and if the ends of the currently picked edge
belong to different subtrees, these subtrees are combined, and the
edge is added to the answer.

After iterating through all the edges, all the vertices will belong to
the same sub-tree, and we will get the answer.
 Design and Analysis of Algorithms
Kruskal’s Algorithm

Let G = {V,E} be weighted connected graph

The vertices are numbered 0 through n − 1.
V = {0,1,….n-1} a nd E =[e1,e2,….en}
Sort the edges in non-decreasing order of their weights.
•Initially, trees of the forest are the vertices (no edges).
•Start with an empty minimum spanning tree and mark each edge as
unvisited,
•While there are edges not yet visited or while the minimum spanning
tree does not have n − 1 edges:
• Find the edge with minimum weight that has not yet been
visited.
• Mark that edge as visited.
• If adding that edge does not create a cycle (that is, the two
vertices are not already connected), add that edge to the
minimum spanning tree.
Design and Analysis of Algorithms
Kruskal’s Algorithm

Given : Graph <V,E>

1
b c
6
3 4 4
5
a 5
f d

2
6 8

e
Design and Analysis of Algorithms
Kruskal’s Algorithm

Given : Graph<V,E> Sorted list of edges in non

decreasing order

1 Edge(u,v) Weight
b c
6 bc 1
3 4 4
ef 2
5 ab 3
a 5
f d
bf 4
cf 4
2
6 8
af 5
df 5
e
ae 6
cd 6
de 8
Design and Analysis of Algorithms
Kruskal’s Algorithm

Given : Graph<V,E> Sorted list of edges in non

decreasing order

1 Edge(u,v) Weight
b c
bc 1
ef 2

a f d ab 3
bf 4
cf 4
af 5
e df 5
ae 6
cd 6
de 8
Design and Analysis of Algorithms
Kruskal’s Algorithm

Given : Graph<V,E> Sorted list of edges in non

decreasing order

1 Edge(u,v) Weight
b c
bc 1
ef 2
a f d ab 3
bf 4
2 cf 4
af 5
e df 5
ae 6
cd 6
de 8
Design and Analysis of Algorithms
Kruskal’s Algorithm

Given : Graph<V,E> Sorted list of edges in non

decreasing order

1 Edge(u,v) Weight
b c
3 bc 1
ef 2

a f d ab 3
bf 4
2 cf 4
af 5
e df 5
ae 6
cd 6
de 8
Design and Analysis of Algorithms
Kruskal’s Algorithm

Given : Graph<V,E> Sorted list of edges in non

decreasing order

1 Edge(u,v) Weight
b c
3 bc 1
4
ef 2
a f d ab 3
bf 4
2 cf 4
af 5
e df 5
ae 6
cd 6
de 8
Design and Analysis of Algorithms
Kruskal’s Algorithm

Given : Graph<V,E> Sorted list of Edges in non

decreasing Order

1 Edge(u,v) Weight
b c
3 bc 1
4
ef 2
a f d ab 3
bf 4
2 Cyclic cf 4 X
af 5
e
df 5
ae 6
cd 6
de 8
Design and Analysis of Algorithms
Kruskal’s Algorithm

Given : Graph<V,E> Sorted list of edges in non

decreasing order

1 Edge(u,v) Weight
b c
3 bc 1
4
ef 2
a f d ab 3
bf 4
2 cf 4
Cyclic af 5 X
e df 5
ae 6
cd 6
de 8
Design and Analysis of Algorithms
Kruskal’s Algorithm

Sorted list of edges in non

Given : Graph<V,E>
decreasing order

1 Edge(u,v) Weight
b c
3 bc 1
4 ef 2
5
a f d ab 3
bf 4
2 cf 4
af 5
e Acyclic df 5
ae 6
cd 6
de 8
 Design and Analysis of Algorithms
Data Structure

Before formalizing the above idea, lets quickly review the disjoint-set
data structure.

1. makeset(v): Create a new set whose only member is pointed to

by v. Note that for this operation v must already be in a set.
2. find(v): Returns a pointer to the set containing v.
3. union(u,v): Unites the dynamic sets that contain u and v into a
new set that is union of these two sets.
Design and Analysis of Algorithms
Disjoint Subsets and Union-Find Algorithms

For example, let S = {1, 2, 3, 4, 5, 6}.

Then makeset(i) creates the set {i} and applying this operation six times
initializes the structure to the collection of six singleton sets:
{1}, {2}, {3}, {4}, {5}, {6}.

Performing union(1, 4) and union(5, 2) yields

{1, 4}, {5, 2}, {3}, {6},

and, if followed by union(4, 5) and then by union(3, 6), we end up with

the disjoint subsets
{1, 4, 5, 2}, {3, 6}.
Design and Analysis of Algorithms
Disjoint Subsets and Union-Find Algorithms

(a) Forest representation of subsets

(b) Result of union(5, 6).
{1, 4, 5, 2} and {3, 6}
Design and Analysis of Algorithms
Kruskal’s Algorithm
Design and Analysis of Algorithms
Kruskal’s Algorithm

With an efficient Sorting algorithm and an Union-Find algorithm.

Design and Analysis of Algorithms
Text Book

Chapter 9 ,Introduction to The Design and Analysis of Algorithms by Anany Levitin

THANK YOU

Bharathi R
Department of Computer Science & Engineering
[email protected]
Design and Analysis of Algorithms
Unit -4
Bharathi R
Department of Computer Science & Engineering
DESIGN AND ANALYSIS OF ALGORITHMS

Unit 4: Greedy Technique

Dijkstra’s Algorithm

Bharathi R
Department of Computer Science & Engineering
Design and Analysis of Algorithms
Dijkstra’s Algorithm

Single Source Shortest Paths Problem: Given a weighted connected

(directed) graph G, find shortest paths from source vertex s to each of
the other vertices

Dijkstra’s algorithm: Similar to Prim’s MST algorithm, with

a different way of computing numerical labels: Among vertices
not already in the tree, it finds vertex u with the smallest sum
dv + w(v,u)
where
v is a vertex for which shortest path has been already found
on preceding iterations (such vertices form a tree rooted at s)
dv is the length of the shortest path from source s to v
w(v,u) is the length (weight) of edge from v to u
Design and Analysis of Algorithms
Dijkstra’s Algorithm

Example
Design and Analysis of Algorithms
Dijkstra’s Algorithm-Example

The next closest vertex is shown in

bold.
Design and Analysis of Algorithms
Dijkstra’s Algorithm
Design and Analysis of Algorithms
Dijkstra’s Algorithm- Time Efficiency

The time efficiency of Dijkstra’s algorithm depends on the data

structures used for implementing the priority queue and for
representing an input graph itself.

• O(|V|2) for graphs represented by weight matrix and array

implementation of priority queue
• O(|E|log|V|) for graphs represented by adj. lists and min-heap
implementation of priority queue
Design and Analysis of Algorithms
Notes on Dijkstra’s Algorithm

• Correctness can be proven by induction on the number of

vertices.

• Doesn’t work for graphs with negative weights (whereas Floyd’s

algorithm does, as long as there is no negative cycle).
• Applicable to both undirected and directed graphs

• Don’t mix up Dijkstra’s algorithm with Prim’s algorithm!

Design and Analysis of Algorithms
Exercises

Solve the following instances of the single-source shortest-paths

problem with vertex a as the source:
Design and Analysis of Algorithms
References

Chapter-9 Greedy Technique

Introduction to the Design & Analysis of Algorithms- Anany Levitin
THANK YOU

Bharathi R
Department of Computer Science & Engineering
[email protected]
Design and Analysis of Algorithms
Unit -4
Bharathi R
Department of Computer Science & Engineering
DESIGN AND ANALYSIS OF ALGORITHMS

Unit 4: Greedy Technique- Huffman Trees

Bharathi R
Department of Computer Science & Engineering
Design and Analysis of Algorithms
Huffman Trees

The Huffman trees are constructed for encoding a given text of n

characters.

While encoding a given text, each character is associated with some

of bits called the codeword.

● Fixed length encoding: Assigns to each character a bit string of

the same length.
● Variable length encoding: Assigns codewords of different lengths
to different characters.
● Prefix free code: In Prefix free code, no codeword is a prefix of a
codeword of another character.
Design and Analysis of Algorithms
Binary prefix code :

● The characters are associated with the leaves of a binary

tree.
● All left edges are labeled as 0, and right edges as 1.
● Codeword of a character is obtained by recording the
labels on the simple path from the root to the
character’s leaf.
● Since, there is no simple path to a leaf that continues to
another leaf, no codeword can be a prefix of another
codeword.
 Design and Analysis of Algorithms
About Huffman Algorithm:

● Invented by David A Huffman in 1951.

● Constructs binary prefix code tree.
● Huffman’s algorithm achieves data compression by finding the best
variable length binary encoding scheme for the symbols that occur in
the file to be compressed. Huffman coding uses frequencies of the
symbols in the string to build a variable rate prefix code
○ Each symbol is mapped to a binary string
○ More frequent symbols have shorter codes
○ No code is a prefix of another code (prefix free code)
● Huffman Codes for data compression achieves 20-90% Compression.
Design and Analysis of Algorithms
Huffman Algorithm:

Input: Alphabet and frequency of each symbol in the text.

Step 1: Initialize n one-node trees (forest) and label them with the
symbols of the alphabet given. Record the frequency of each
symbol in its tree’s root to indicate the tree’s weight. (Generally,
the weight of a tree will be equal to the sum of the frequencies in
the tree’s leaves.)

Step 2: Repeat the following operation until a single tree is

obtained. Find two trees with the smallest weight. Make them the
left and right subtree of a new tree and record the sum of their
weights in the root of the new tree as its weight.
 Design and Analysis of Algorithms
Huffman Algorithm:

1.First arrange the characters in ascending order of their probabilities

Combine these two nodes and form a single node

Design and Analysis of Algorithms
Huffman Algorithm:

2.Again arrange probabilities in ascending order

Combine these nodes

 Design and Analysis of Algorithms
Huffman Algorithm:

3.Again arrange probabilities in ascending order

Combine these 2 trees

4. Arrange probabilities in ascending order

Combine these 2 trees

Design and Analysis of Algorithms
Huffman Tree

DAD is encoded as 011101, and

10011011011101 is decoded as BAD_AD

The resulting codewords are as follows

 Design and Analysis of Algorithms
Compression Ratio

Compression Ratio : A standard measure of a compression algorithm’s efficiency.

Design and Analysis of Algorithms
Features of Huffman’s Encoding

• Huffman’s encoding is one of the most important file-

compression methods.

• In addition to its simplicity and versatility, it yields an optimal,

i.e., minimal-length, encoding (provided the frequencies of
symbol occurrences are independent and known in advance).

• The simplest version of Huffman compression calls, in fact, for a

preliminary scanning of a given text to count the frequencies of
symbol occurrences in it. Then these frequencies are used to
construct a Huffman coding tree and encode the text.
Design and Analysis of Algorithms
Text Books

Chapter-9 Greedy Technique

Introduction to the Design & Analysis of Algorithms- Anany Levitin
THANK YOU

Bharathi R
Department of Computer Science & Engineering
[email protected]
 UE19CS251 DAA
Assignment II
Max Marks: 20(will be scaled down)

Implement Dijkstra’s algorithm to solve Single Destination Shortest Path

problem. Single destination shortest path is finding shortest path from all the
vertices to the given vertex
Input graph to be read from the file adjacencylist.txt
Input file
 Vertices are numbered 1 to n
 First line represents number of vertices
 This is followed by a set of lines
 Each line starts with a integer (any integer from 1 to n in any order) which
represents vertex number (v_id) followed by space followed by a set of d
pairs where d represents outdegree of the vertex v_id .First number in the
pair represents neighbour vertex id and second number weight of the
edge which connects vertex to the neighbour. Weight is always a non-
negative integer
Sample Input file
4
1 2 43 5 4 5
4 1 73 3
2 1 3 4 10
Represents the graph
The input file provided in the assignment folder should be read to create a graph
which should be represented in memory using adjacency list representation
Implementation
 Program should consider the last vertex (v_id =n )as the destination
(Example destination is 4 for the graph mentioned in example above)
 Using Dijkstra’s algorithm Program should determine shortest distance
from all the vertices to the destination
 Priority queue used by dijkstra’s algorithm should be implemented
using min heap. Each node in the heap contains vertex number, d value
(distance of path from source to the vertex), and p value (Vertex id of
predecessor vertex on the path from the source to the vertex). d value
should be used as key to perform various operations
(insert/delete/update)
 Program should print shortest path from all the vertices to the
destination.
Output Format
 Each paths to be printed on new line (starting with path from 1 to
destination then from 2,3---------n-1).If no path exists from a particular
vertex to the destination vertex ‘NO PATH’ should be printed on the
respective line number
 Each line should print a single path and length of the path (space
separated), where the vertices in a path are space separated
(No other format for output is accepted)
Sample output:
1 1 45
2 2 14 8
3 NO PATH
C programming language should be used for implementation
Implementation should consists of three files
 Header file (function prototypes, user defined data type definitions)
 Implementation File (function definitions)
 Client file (Driver function)
The file names should be
SRN_H.h
SRN_F.c
SRN_C.c
List of functionalities your code should support
1. Reading data from file and creating Graph
(Adjacency list and not adjacency matrix)
2. Implementation for Priority queue using min heap
(Insert, delete, search update)
3. Implementation of Dijkstra’s algorithm to determine shortest path from
all vertices to nth vertex in the list
(Single Destination Shortest path)
4. Determining shortest path distance and shortest path from all the vertices
to destination vertex (Dijkstra’s module to be invoked only once)
5. Printing shortest path and shortest path distance from all the vertices to
the destination starting with vertex numbered 1,2,3---n-1 as per the
output format specified

Marks distribution

Functionality Max Marks

1+5 5 marks
2, 3, 4 5 marks each