Module 3

Transform and Conquer

• That’s the secret to life . . . replace one

worry with another.
—Charles M. Schulz (1922–2000),
American cartoonist, the creator of Peanuts
 Transform-and-conquer

•Works as two-stage procedures.

•First, in the transformation stage, the
problem’s instance is modified to be, for
one reason or another, more amenable
to solution.
•Then, in the second or conquering stage,
it is solved.
•There are three major variations of this idea
that differ by what we transform a given
instance to :
•Transformation to a simpler or more
convenient instance of the same problem—
instance simplification. (Eg. Presorting)
•Transformation to a different representation of
the same instance— representation
change.(Eg. Heap)
•Transformation to an instance of a different
problem for which an algorithm is already
available—problem reduction.(Eg. lcm(m, n)
= m•n/gcd(m, n))
 Binary Search Tree
Arrange keys in a binary tree with the binary search tree
property:
K

<K >K

Example: 5, 3, 1, 10, 12, 7, 9

(Which could/should be the root? Best/Worst case?)

6
 Dictionary Operations on Binary Search Trees

Searching – straightforward
Insertion – search for key, insert at leaf where search terminated
Deletion – 3 cases:
deleting key at a leaf
deleting key at node with single child
deleting key at node with two children

Efficiency depends of the tree’s height: ⎣log2 n⎦ ≤ h ≤

n-1,
with height average (random files) be about 3log2 n
Maximum number of nodes if height k?
Thus all three operations have
– worst case efficiency: Θ(n)
– average case efficiency: Θ(log n)

Bonus: inorder traversal produces sorted list

A. Levitin “Introduction to the Design &
Analysis of Algorithms,” 3rd ed., Ch. 6
7
©2012 Pearson Education, Inc. Upper
 Balanced Search Trees
Attractiveness of binary search tree is marred by the
bad (linear) worst-case efficiency. Two ideas to
overcome it (ie keep BST close to balanced) are:

● to restructure BST to maintain balance when an

insertion makes the tree “too unbalanced”
– AVL trees
– red-black trees [come back to after 2-3-4 trees]

● to allow more than one key per node of a search tree

– 2-3 trees [Chapter 6]
– 2-3-4 trees [Chapter ?]
– B-trees [Chapter 7] (Red black tree = 234 in binary tree)
A. Levitin “Introduction to the Design &
Analysis of Algorithms,” 3rd ed., Ch. 6
8
©2012 Pearson Education, Inc. Upper
 Balanced trees: AVL trees
Definition An AVL tree is a binary search tree in which, for every
node, the difference between the heights of its left and right
subtrees, called the balance factor, is at most 1 (with the
height of an empty tree defined as -1)

Tree (a) is an AVL tree; tree (b) is not an AVL tree

A. Levitin “Introduction to the Design &
Analysis of Algorithms,” 3rd ed., Ch. 6
9
©2012 Pearson Education, Inc. Upper
 Rotations
If a key insertion violates the balance requirement at some node, the subtree
rooted at that node is transformed via one of the four rotations. (The rotation is
always performed for a subtree rooted at an “unbalanced” node closest to the
new leaf.)

Single R-rotation Double LR-rotation

A. Levitin “Introduction to the Design &

Analysis of Algorithms,” 3rd ed., Ch. 6
10
©2012 Pearson Education, Inc. Upper
General case: Single R-rotation

A. Levitin “Introduction to the Design &

Analysis of Algorithms,” 3rd ed., Ch. 6
11
©2012 Pearson Education, Inc. Upper
General case: Double LR-rotation

A. Levitin “Introduction to the Design &

Analysis of Algorithms,” 3rd ed., Ch. 6
12
©2012 Pearson Education, Inc. Upper
AVL tree construction - an example [SKIP]
Construct an AVL tree for the list 5, 6, 8, 3, 2, 4, 7

A. Levitin “Introduction to the Design &

Analysis of Algorithms,” 3rd ed., Ch. 6
13
©2012 Pearson Education, Inc. Upper
AVL tree construction - example

A. Levitin “Introduction to the Design &

Analysis of Algorithms,” 3rd ed., Ch. 6
14
©2012 Pearson Education, Inc. Upper
 Analysis of AVL trees
● h ≤ 1.4404 log2 (n + 2) - 1.3277
average height: 1.01 log2n + 0.1 for large n (found
empirically)

● Search and insertion are O(log n)

● Deletion is more complicated but is also O(log n)

● Disadvantages:
– frequent rotations
– complexity

● A similar idea: red-black trees (height of subtrees is

allowed to differ by up“Introduction
A. Levitin to a factor
to the Designof
& 2)
Analysis of Algorithms,” 3rd ed., Ch. 6
15
©2012 Pearson Education, Inc. Upper
 Multiway Search Trees
Definition A multiway search tree is a search tree that allows
more than one key in tree nodes.

Definition A node of a search tree is called an n-node if it

contains n-1 ordered keys (which divide the entire key range
into n intervals pointed to by the node’s n links to its children):

k1 < k2 < … < kn-1

< k1 [k1, k2 ) ≥ kn-1

Note: Every node in a classical binary search tree is a 2-node

A. Levitin “Introduction to the Design &
Analysis of Algorithms,” 3rd ed., Ch. 6
16
©2012 Pearson Education, Inc. Upper
 2-3 Tree
Definition A 2-3 tree is a search tree that
● may have 2-nodes and 3-nodes (How many keys per node?)
● height-balanced (all leaves are on the same level !! )

A 2-3 tree is constructed by successive insertions of keys given, with a

new key always inserted into a leaf of the tree. If the leaf is a 3-node, it’s
split into two with the middle key promoted
A. Levitin “Introduction to the Design &to the parent. Splitting can
Analysis of Algorithms,” 3rd ed., Ch. 6
be repeated up the tree,©2012
as needed.
Pearson Education, Inc. Upper
17
2-3 tree construction – an example

Construct a 2-3 tree the list 9, 5, 8, 3, 2, 4, 7

A. Levitin “Introduction to the Design &

Analysis of Algorithms,” 3rd ed., Ch. 6
©2012 Pearson Education, Inc. Upper LEAVES?18
 Analysis of 2-3 trees
● Consider:
– Shortest tree with 8 nodes (add 1,4,10 to [3,8/2-5-9])
– Tallest tree with 7 nodes [which tree]

● log3 (n + 1) - 1 ≤ h ≤ log2 (n + 1) - 1
– Does this work for examples above? What is h?

● Search, insertion, and deletion are in Θ(log n)

● The idea of 2-3 tree can be generalized by allowing

more keys per node
– 2-3-4 trees
– B-trees
A. Levitin “Introduction to the Design &
Analysis of Algorithms,” 3rd ed., Ch. 6
19
©2012 Pearson Education, Inc. Upper
 Heap and Heapsort
• Notion of the Heap
• DEFINITION: A heap can be defined as a binary tree with keys
assigned to its nodes, one key per node, provided the following
two conditions are met:
1. The shape property—the binary tree is essentially complete (or
simply complete), i.e., all its levels are full except possibly the
last level, where only some rightmost leaves may be missing.
2. The parental dominance or heap property—the key in each
node is greater than or equal to the keys in its children. (This
condition is considered automatically satisfied for all leaves.)
 • Important properties of heaps
1. The root of a heap always contains its largest element.

2. A node of a heap considered with all its descendants is also a

heap.

3. A heap can be implemented as an array by recording its

elements in the topdown, left-to-right fashion.
It is convenient to store the heap’s elements in positions 1
through n of such an array, leaving H[0] either unused or putting
there a sentinel whose value is greater than every element in the
heap. In such a representation,
a. the parental node keys will be in the first n/2 positions of the
array, while the leaf keys will occupy the last n/2 positions;
b. the children of a key in the array’s parental position i (1≤ i
≤ n/2) will be in positions 2i and 2i + 1, and, correspondingly, the
parent of a key in position i (2 ≤ i ≤ n) will be in position
i/2.
 Bottom-up heap construction
algorithm
• It initializes the essentially complete binary tree with n
nodes by placing keys in the order given and then
“heapifies” the tree as follows:
• Starting with the last parental node, the algorithm checks
whether the parental dominance holds for the key in this
node. If it does not, the algorithm exchanges the node’s
key K with the larger key of its children and checks
whether the parental dominance holds for K in its new
position.
• This process continues until the parental dominance for K
is satisfied. (Eventually, it has to because it holds
automatically for any key in a leaf.)
• After completing the “heapification” of the subtree rooted
at the current parental node, the algorithm proceeds to
do the same for the node’s immediate predecessor.
• The algorithm stops after this is done for the root of the
tree.
2,9,7,6,5,8
 Bottom-up vs. Top-down Heap
Construction

• Bottom-up: Put everything in the array and then

heapify/fix the trees in a bottom-up way.
• Top-down: Heaps can be constructed by successively
inserting elements into an (initially) empty heap.
 Top-down heap construction algorithm

• The alternative (and less efficient) algorithm constructs a heap

by successive insertions of a new key into a previously
constructed heap;
• So how can we insert a new key K into a heap?
• First, attach a new node with key K in it after the last leaf of the
existing heap.
• Then shift K up to its appropriate place in the new heap as
follows.
❖ Compare K with its parent’s key: if the latter is greater than or
equal to K, stop (the structure is a heap); otherwise, swap these
two keys and compare K with its new parent.
❖ This swapping continues until K is not greater than its last
parent or it reaches the root
• Maximum Key Deletion from a heap
• Step 1 Exchange the root’s key with the last key K
of the heap.
• Step 2 Decrease the heap’s size by 1.
• Step 3 “Heapify” the smaller tree by shifting K
down the tree exactly in the same way as in the
bottom-up heap construction algorithm ie., verify
the parental dominance for K: if it holds, we are
done; if not, swap K with the larger of its children
and repeat this operation until the parental
dominance condition holds for K in its new
position.
 Heapsort Algorithm

▪ The algorithm
▪ (Heap construction) Build heap for a given array (either
bottom-up or top-down)
▪ (Maximum deletion) Apply the root-deletion operation
n-1 times to the remaining heap until heap contains just
one node.
▪ An example: 2 9 7 6 5 8
 SPACE AND TIME TRADEOFF

• The idea is to preprocess the problem’s input,

• in whole or in part, and store the additional information obtained to accelerate
solving the problem afterward. We call this approach input enhancement
• The other type of technique that exploits space-for-time trade-offs simply uses extra
space to facilitate faster and/or more flexible access to the data.We call this
approach prestructuring.
• The two resources—time and space—do not have to compete with each other in all
design situations. In fact, they can align to bring an algorithmic solution that
minimizes both the running time and the space consumed. Such a situation arises, in
particular, when an algorithm uses a space efficient data structure to represent a
problem’s input, which leads, in turn, to a faster algorithm.
 Input enhancement – Sorting by Counting

• In sorting problem, One rather obvious idea is to count, for each element of a list to
be sorted, the total number of elements smaller than this element and record the
results in a table. These numbers will indicate the positions of the elements in the
sorted list: e.g., if the count is 10 for some element, it should be in the 11th position
(with index 10, if we start counting with 0) in the sorted array. Thus, we will be able
to sort the list by simply copying its elements to their appropriate positions in a
new, sorted list. This algorithm is called comparison counting sort
 DISTRIBUTION COUNT SORT
• The counting idea does work productively in a situation in which elements to
be sorted belong to a known small set of values. Assume, for example, that we
have to sort a list whose values can be either 1 or 2. Rather than applying a
general sorting algorithm, we should be able to take advantage of this
additional information about values to be sorted. Indeed, we can scan the list to
compute the number of 1’s and the number of 2’s in it and then, on the second
pass, simply make the appropriate number of the first elements equal to 1 and
the remaining elements equal to 2.
• More generally, if element values are integers between some lower bound l and
upper bound u, we can compute the frequency of each of those values and
store them in array F[0..u − l]. Then the first F[0] positions in the sorted list
must be filled with l, the next F[1] positions with l + 1, and so on. All this can
be done, of course, only if we can overwrite the given elements.
• Let us consider a more realistic situation of sorting a list of
items with some other information associated with their keys
so that we cannot overwrite the list’s elements.Then we can
copy elements into a new array S[0..n−1]to hold the sorted list
as follows.
• The elements of A whose values are equal to the lowest
possible value l are copied into the first F[0]elements of S, i.e.,
positions 0 through F[0]− 1; the elements of value l + 1 are
copied to positions from F[0] to (F[0]+ F[1]) − 1; and so on.
Since such accumulated sums of frequencies are called a
distribution in statistics, the method itself is known as
distribution counting.
• Note that the distribution values indicate the proper positions for the last
occurrences of their elements in the final sorted array. If we index array
positions from 0 to n − 1, the distribution values must be reduced by 1 to get
corresponding element positions.
• It is more convenient to process the input array right to left. For the example,
the last element is 12, and, since its distribution value is 4, we place this 12 in
position 4 − 1= 3 of the array S that will hold the sorted list. Then we decrease
the 12’s distribution value by 1 and proceed to the next (from the right)
element in the given array.
• Assuming that the range of array values is fixed, this is obviously a linear
algorithm because it makes just two consecutive passes through its input array
A. This is a better time-efficiency class than that of the most efficient sorting
algorithms—mergesort, quicksort, and heapsort—we have encountered. It is
important to remember, however, that this efficiency is obtained by exploiting
the specific nature of inputs for which sorting by distribution counting works,
in addition to trading space for time.
 Input Enhancement in String Matching

• Input-enhancement idea: preprocess the pattern to get some information about it, store
this information in a table, and then use this information during an actual search for
the pattern in a given text.
• Consider, as an example, searching for the pattern BARBER in some text:

• Starting with the last R of the pattern and moving right to left, we compare the
corresponding pairs of characters in the pattern and the text. If all the pattern’s
characters match successfully, a matching substring is found. Then the search can be
either stopped altogether or continued if another occurrence of the same pattern is
desired. If a mismatch occurs, we need to shift the pattern to the right. Clearly, we
would like to make as large a shift as possible without risking the possibility of
missing a matching substring in the text. Horspool’s algorithm determines the size of
such a shift by looking at the character c of the text that is aligned against the last
character of the pattern. This is the case even if character c itself matches its
counterpart in the pattern
•In general, the following four possibilities can occur.
•Case 1 If there are no c’s in the pattern—e.g., c is letter S in our example— we can
safely shift the pattern by its entire length (if we shift less, some character of the pattern
would be aligned against the text’s character c that is known not to be in the pattern):
These examples clearly demonstrate that right-to-left character
comparisons can lead to farther shifts of the pattern than the shifts by
only one position always made by the brute-force algorithm
• We can precompute shift sizes and store them in a table. The table will be indexed
by all possible characters that can be encountered in a text, including, for natural
language texts, the space, punctuation symbols, and other special characters.
• The table’s entries will indicate the shift sizes computed by the formula:
• Here is a simple algorithm for computing the shift table entries.
• Initialize all the entries to the pattern’s length m and scan the pattern left to
right repeating the following step m − 1 times: for the jth character of the
pattern (0 ≤ j ≤ m − 2), overwrite its entry in the table with m − 1− j , which is
the character’s distance to the last character of the pattern. Note that since the
algorithm scans the pattern from left to right, the last overwrite will happen for
the character’s rightmost occurrence.
• Horspool’s algorithm
• Step 1 For a given pattern of length m and the alphabet used in both the pattern and
text, construct the shift table as described above.
• Step 2 Align the pattern against the beginning of the text.
• Step 3 Repeat the following until either a matching substring is found or the pattern
reaches beyond the last character of the text. Starting with the last character in the
pattern, compare the corresponding characters in the pattern and text until either all
m characters are matched (then stop) or a mismatching pair is encountered. In the
latter case, retrieve the entry t (c) from the c’s column of the shift table where c is
the text’s character currently aligned against the last character of the pattern, and
shift the pattern by t (c) characters to the right along the text.
The worst-case efficiency of Horspool’s algorithm is in O(nm) and for
random texts its theta(n)