Design and Analysis of Algorithm

Course Code : 5009

Lecture 2

Dr. Huma Qayyum

Department of Software Engineering
[email protected]
More Analysis of the Algorithm
 Iterative Binary Search
int bsearch(int* a[],int size,int x) {
int low=0, high=size-1;

while (low<=higt) {
int mid=(low+high)/2;
if (a[mid]<x)
low=mid+1;
else if (x<a[mid])
high=mid-1;
else return mid;
}
return -1
}
 What is a recurrence relation?

• A recurrence relation, T(n), is a recursive function of integer variable n.

• Like all recursive functions, it has both recursive case and base case.
• Example:

• The portion of the definition that does not contain T is called the base case of
the recurrence relation; the portion that contains T is called the recurrent or
recursive case.
• Recurrence relations are useful for expressing the running times (i.e., the
number of basic operations executed) of recursive algorithms
Forming Recurrence Relations
• For a given recursive method, the base case and the
recursive case of its recurrence relation correspond
directly to the base case and the recursive case of the
method.
• Example 1: Write the recurrence relation for the
following method.
• The base case is reached when n == 0. The method
performs one comparison. Thus, the number of
public void f (int n) {
operations when n == 0, T(0), is some
if (n constant
> 0) { a.
• When n > 0, the method performs twoSystem.out.println(n);
basic operations
f(n-1);
and then calls itself, using ONE recursive
} call, with a
}
parameter n – 1.
• Therefore the recurrence relation is:
Forming Recurrence Relations
• Example 2: Write the recurrence relation for the
following method.

public int g(int n) {

if (n == 1)
return 2;
else
return 3 * g(n / 2) + g( n / 2) + 5;
}

• The base case is reached when n == 1. The method performs one

comparison and one return statement. Therefore, T(1), is constant
c.
• When n > 1, the method performs TWO recursive calls, each with
the parameter n / 2, and some constant # of basic operations.
• Hence, the recurrence relation is:
 Solving Recurrence Relations
• To solve a recurrence relation T(n) we need to derive a form of
T(n) that is not a recurrence relation. Such a form is called a
closed form of the recurrence relation.

• There are four methods to solve recurrence relations that represent

the running time of recursive methods:
 Iteration method (unrolling and summing)
 Substitution method (Intelligent Guess Work)
 Recursion tree method
 Master method
Solving Recurrence Relations - Iteration method

• Steps:
 Expand the recurrence
 Express the expansion as a summation by plugging the
recurrence back into itself until you see a pattern.
 Evaluate the summation
• In evaluating the summation one or more of the
following summation formulae may be used:
• Arithmetic series: •Special Cases of Geometric Series:

 Geometric Series:
Solving Recurrence Relations - Iteration
method
 Harmonic Series:

 Others:
Analysis Of Recursive Binary Search
public int binarySearch (int target, int[] array,
int low, int high) {
if (low > high)
return -1;
else {
int middle = (low + high)/2;
if (array[middle] == target)
return middle;
else if(array[middle] < target)
return binarySearch(target, array, middle + 1, high);
else
return binarySearch(target, array, low, middle - 1);
}
}
• The recurrence relation for the running time of the method is:
T(1) = a if n = 1 (one element array)
T(n) = T(n / 2) + b if n > 1
 Recursive binary search:
int bsearch(int* a[],int low, int
high, int x) {

if (low>high) return -1; O(1)

else int mid=(low+high)/2; O(1)
if (x=a[mid]) return mid;
else
if(a[mid]<x)
bsearch(a,mid+1,high,x); T(N/2)
else
T (1)  1
N
T (N ) T ( )  1
bsearch(a,low,mid-1);
} 2
Analysis Of Recursive Binary Search
Expanding:
T(n) = T(n / 2) + b
= [T(n / 4) + b] + b = T (n / 22) + 2b
= [T(n / 8) + b] + 2b = T(n / 23) + 3b
= ……..
= T( n / 2k) + kb

When n / 2k = 1  n = 2k  k = log2 n, we have:

T(n) = T(1) + b log2 n

= a + b log2 n

Therefore, Recursive Binary Search is O(log n)

 Insertion sort
Run this algorithm of following input
5 7 0 3 4 2 6 1
Example Insertion Sort
• 57034261

57034261 (0)
05734261 (2)
03574261 (2)
03457261 (2)
02345761 (4)
02345671 (1)
01234567 (6)
16
Mathematical Tools for Design and Analysis of
Algorithm
• When analyzing an algorithm, the amount of
resources required is usually expressed as a function
of the input size.
• A non trivial algorithm typically consists of repeating
a set of instructions either iteratively, e.g. by
executing a for or while loop.

17
Mathematical Tools for Design and Analysis of
Algorithm
• Or recursively by invoking the same algorithm again
and again, each time reducing the input size until it
becomes small enough, in which case the algorithm
solves the input instance using a straightforward
method.
• This implies that the amount of resources used by an
algorithm can be expressed in the form of summation
or recursive formula.

18
Mathematical Tools for Design and Analysis of
Algorithm
• This mandates the need for the basic mathematical
tools that are necessary to deal with these summations
and recursive formulas in the process of analyzing an
algorithm.
• Hence we review some of the mathematical
preliminaries and discuss briefly some of these
mathematical tools that are frequently employed in
the analysis of algorithms.

19
 Sequence of Mathematical Tools

• Sets
• Sequences
• Order pairs
• Cross Product
• Relation
• Functions
• Operators over above structure
Sets, Relations and Functions
• When analyzing an algorithm, its input is considered
to be a set drawn from some particular domain, e.g.
the set of integers.
• An algorithm, in the formal sense, can be thought of
as a function, which is a constrained relation, that
maps each possible input to a specific output.
• Thus, sets and functions are at the heart of
algorithmic analysis.

21
 SET

• The term set is used to refer to any collection of objects,

which are called members or elements of the set.
• A set is called finite if it contains n elements, for some
constant n >=0, and infinite otherwise.
• Examples of infinite sets include the set of natural numbers
{1; 2; : : :} and the sets of integers, rational and reals.

22
 Representation of Sets
• Three different ways to built a set
• Descriptive Form
• Set Builder Form (Comprehension Set)
• Tabular Form

• Descriptive Form
• E.g S= Set of all Prime Numbers (not useful from
computer point of view)

• Tabular Form
• S={2,3,5,7,11,13……} ->:Little bit informative from
Computer Tool point of view But
• What about large set (Finite, Infinite) cannot be write
down.
23
 Representation of Sets
• Set Builder Notation (very useful)
•

• Write properties of set rather than their name

• There any prime list can be generated through this property

• Indirect pseudo code
• Here input/output and relationship is define with the help of set of well define
properties. Relation ship is define through predicate

24
 Partitioning of a Set
• A partition P of a Set A is a collection of { A1, A2, A3, ….An }
such that following are satisfied e.g.
1. Disk partitioning
2. Task partitioning on different machines
Rules :

• ∀ Ai ∈ P, Ai ≠ Φ. (should not be empty)

• Ai ∩ Aj = Φ ∀ i,j ∈ {1,2,3,….n} and i ≠ j. (do not overwrites)

• A= A1 ∪ A2 ∪ A3 ∪….. An. (give same set)

26
 Function : Partial Functions

DEFINITION

In mathematics, a partial function from X to Y is a function ƒ: X' → Y, where X' is a subset

of X.

27
 Functions

DEFINITION

It is a relation in which each element of X is related to the unique element or image of Y

Then partial function is a total function denoted by X Y . Here domain of
function is X .

Advanced Algorithms Analysis and Design

28
 Relation VS Functions

Q) Algorithm single input generate no output ? Good or not

Advanced Algorithms Analysis and Design

29
 Relation VS Functions

Q) Algorithm single input generate multiple outputs. Meaning generating

different output in different timing ? Good or not

Advanced Algorithms Analysis and Design

30
 Sequences

It is sometime necessary to record the order in which the objects are arranged. e.g.

• Data may be indexed by an ordered collection of keys.

• Messages may be stored in the order of arrival.
• Task may be performed in the order of importance.
• Names may be sorted in the order of alphabets and so on….

Definition:

A group of elements in a specified order is called a sequence. A sequence can have a repeated
elements. i.e. multiset

Operations apply on sequences

Merge
indexing (search)
filter
extraction

31
 Sequence

• Notation: Sequence is defined by listing elements in

order, enclosed in parentheses. e.g.
S = (a, b, c), T = (b, c, a), U = (a, a, b, c)
• Sequence is a set
S = {(1, a), (2, b), (3, c)} = {(3, c), (2, b), (1, a)}

• Permutation: If all elements of a finite sequence are

distinct, that sequence is said to be a permutation of
the finite set consisting of the same elements.
• No. of Permutations: If a set has n elements, then
there are n! distinct permutations over it.
 Operators over Sequences

• Operators are for manipulations

Concatenation
• (a, b, c )  ( d, a ) = ( a, b, c, d, a )
Other operators
• Extraction of information: ( a, b, c, d, a )(2) = b
• Filter Operator: {a, d}  ( a, b, c, d, a ) = (a, d, a)
Note:
• We can think how resulting theory of sequences
falls within our existing theory of sets
• And how operators in set theory can be used in
case of sequences
 Tuples and Cross Product

• A tuple is a finite sequence.

• Ordered pair (x, y), triple (x, y, z), quintuple
• A k-tuple is a tuple of k elements.
Construction to ordered pairs
• The cross product of two sets, say A and B, is
A  B = {(x, y) | x  A, y  B}
| A  B | = |A| |B|

• Some times, A and B are of same set, e.g.,

Z  Z, where Z denotes set of Integers
 Binary Relations

Definition: If X and Y are two non-empty sets, then

X  Y = {(x, y) | x X and y  Y}
Example: If X = {a, b}, Y = {0,1} Then
X  Y = {(a, 0), (a, 1), (b, 0), (b, 1)}

Definition: A subset of X  Y is a relation over X  Y

Example: Compute all relations over X  Y, where
X = {a, b}, Y = {0,1}
R1 = , R2 = {(a, 0)}, R3 = {(a, 1)}
R4 = {(b, 0)}, R5 = {(b, 1)}, R6 = {(a, 0), (b, 0)}, . . .
There will be 24 = 16 number of relations

Advanced Algorithms Analysis and Design

 Applications : Order Pairs in Terms of String
Definition
• A relation R over X, Y, Z is some subset of X x Y x Z and so on
Example 1
• If  = {0, 1}, then construct set of all strings of length 2 and 3
• Set of length 2 =  x  = {0,1} x {0,1} = {(0,0), (0,1), (1,0), (1,1)}
= {00, 01, 10, 11}
• Set of length 3 =  x  x  = {0, 1} x {0, 1} x {0,1}
= {(0,0,0), (0,0,1), (0,1,0), (0,1,1), (1,0,0), (1,0,1), (1,1,0), (1,1,1)}
= {000, 010, 100, 110, 001, 011, 101, 111}
Example 2
• If  = {0, 1}, then construct set of all strings of length ≤ 3
• Construction = {}     x    x  x 
Similarly we can construct collection of all sets of length n

Advanced Algorithms Analysis and Design

 Logic and Proving Techniques

i.e. How can we prove that algorithm

that we have design is correct
Tools used for proving algorithms
• Propositional Logic
• Predicate Logic
• Proofs using
• Truth Tables
• Logical Equivalences
• Counter Example
Propositional and Predicate Logic
 Logical Connectives

• Proposition: Simplest statements also called atomic formula

• Propositions may be connected based on atomic formula.
• Logical connectives, in descending order of operator precedence

Symbol Name Pronunciation

 (uniary) negation not

 (binary) conjunction and
 disjunction or
 implication implies
 equivalence if and only if
 Negation, Conjunction and Disjunction

Negation
Conjunction
• The conjunction p  q is true only if p and q both are
true otherwise false
• The conjunction follows the commutative property,
i.e. p  q = q  p

Disjunction
• The disjunction p  q is false if both p and q are false
otherwise true
• The disjunction follows the commutative property as
well, i.e. p  q = q  p
 Implication

• The p is antecedent and q is consequent

• The antecedent is stronger than consequent.
• Commutative property does not hold, i.e.
(p  q)  (q  p)
p q pq qp pq
t t t t t
t f f t f
f t t f t
f f t t t
 Bi-implication

The equivalence p  q means p  q & q  p

Commutative property does hold, i.e.
(p  q) = (q  p)

p q pq qp pq&qp

t t t t t
t f f t f
f t t f f
f f t t t
 Predicates and Quantifiers

Predicate: P(x)  x < 5

Example:  x : N | x2 = x  x < 2
For all quantifier
•  x, P(x) is true  P(x) is true for all x.
Existential Quantifier
•  x, P(x) is true  P(x) is true for some value of x.
Logical Equivalences
•  x, P(x) is logically equivalent to  ( x, P(x))
•  x, P(x) is logically equivalent to ( x, P(x))
•  x, (P(x)  Q(x)) means x, P(x)  Q(x)
Proving Techniques
Proof using Truth Table: (p  q  r)  (p  (q  r))

p q r (p  q  r)  (p  (q  r))
t t t t t t t tt
t t f t f t f ft
t f t f t t t tt
t f f f t t t tt
f t t f t t t tt
f t f f t t t ft
f f t f t t t tt
f f f f t t t tt
 De Morgan’s Laws

1. (p  q) =  p   q

p q pq (p  q) p q p  q

t t t f f f f
t f f t f t t
f t f t t f t
f f f t t t t
 De Morgan’s Laws

2. (p  q) =  p   q

p q pq (p  q) p q p  q

t t t f f f f
t f t f f t f
f t t f t f f
f f f t t t t
 Proof using Counter Example, Contraposition

Counter Example
Is used when we have prove general statement is false.
To prove  x (A(x)  B(x)) is false, we show some
object x for which A(x) is true and B(x) is false.
Proof
 ( x (A(x)  B(x))) 
 x, (A(x)  B(x))) 
 x, (A(x)  B(x)) 
 x, A(x)  B(x))
50
Recursive Call

• function Fibrec(n)

• if n < 2 then return n

• else return Fibrec(n - 1) + Fibrec(n - 2)

• Let T(n) be the time taken by a call on Fibrec(n)

• If n < 2, the algorithm simply returns n, which takes some time constant
time a

• Most of the work is spent in the two recursive calls which take time T(n
- 1) and T(n - 2)

• One addition involving fn-1 and fn-2 (values returned by the recursive
calls)
51
Recursive Fibonacci (cont)

52
53
54
Supplementary (Example)

55
Example Selection Sort

Original Array
6354927
1st pass -> 2 3 5 4 9 6 7 (2 and 6 were swapped)
2nd pass -> 2 3 4 5 9 6 7 (4 and 5 were swapped)
3rd pass -> 2 3 4 5 6 9 7 (6 and 9 were swapped)
4th pass -> 2 3 4 5 6 7 9 (7 and 9 were swapped)
5th pass -> 2 3 4 5 6 7 9 (no swap)
6th pass -> 2 3 4 5 6 7 9 (no swap)

56
57
58