CS-311 Design and Analysis of

Algorithms

Lecture 2
Analysis Example

• Find the value of the largest element in a

list of n numbers.

MaxElement(A[0..n-1)
maxVal = A[0];
for(I = 1; I < n; I++)
if(A[I] > maxVal)
maxVal = A[I];
return maxVal
Analysis Example
• Check whether all the elements in a given array
are distinct.

UniqueElements(A[0..n-1])
for(I = 0; i<n-1; i++)
for(j = i+1; j<n; j++)
if(A[i] = A[j])
return false
return true
Analysis Example
Max-Subsequence-Sum(Array, N) //Where N is size of Array

{ int this-sum = 0, Max-sum = 0;

for(int i = 0; i < N; i++)
{ for(int j = i; j < N; j++)
{
this-sum = 0;
for(int k = i; k <= j; k++)
this-sum = this-sum + Array[k];
if(this-sum > Max-sum)
Max-sum = this-sum;
}
}
return(Max-sum);
}
 A Maximum Sub Sequence
Sum Problem
Analysis:
• Instructions inside the third loop take constant time.
10

1  10  5  1  6
n n j

 cons tan t =1

i 1 j i k i
k 5

• First Sum j
N

i 
N ( N  1)
2
1 
k i
j  i 1 i 1

• Next Sum n
(n  i  1)(n  i  2)
when i = n, n-i+1
j i
j  i 1 
2
n
(n  i  1)(n  i  2)
i1 2
A Maximum Sub Sequence
Sum Problem
• Solve the summation
= c1n3 + c2n2 + c3n
= O(n3)
• If n = 1000, Time constant = 1msec
=> T  109/(24*1000*60*60 )
 11 days
Greatest Common Devisor Problem

• Greatest common devisor of two non

negative, not-both-zero integers m and n,
denoted gcd(m, n) is defined as
– The largest integer that divides both m and n
evenly, i.e. with a remainder of zero.
Greatest Common Devisor Problem
Consecutive Integer Checking Algorithm
• This algorithm is based on the definition of GCD
i.e.
– The GCD of two numbers m and n is the
largest integer that divides both numbers
evenly
• This common divisor can not be greater than the
smaller of these numbers.
• It can be denoted as t = min(m, n)
• We can start by checking whether t divides both
m and n: if it does
– t is the answer
• If it does not, we simply decrease t by 1 and try
again
Greatest Common Devisor Problem
Consecutive Integer Checking Algorithm
Step 1 Assign the value of min(m, n) to t
Step 2 Divide m by t. if the remainder of this division
is 0, go to Step 3; otherwise, go to Step 4.
Step 3 Divide n by t, if the remainder of this division
is 0, return the value of t as the answer and stop;
otherwise, proceed to Step 4
Step 4 Decrease the value of t by 1. Go to Step 2.
• This algorithm does not work correctly when one of its input
numbers is 0.
• The range of inputs for which an algorithm works has
to be specified carefully.
Greatest Common Devisor Problem
Middle School Procedure
Step 1 Find the prime factors of m
Step 2 Find the prime factors of n
Step 3 Identify all the common factors in the two
prime expansions found in Step 1 and Step 2.
Step 4 Compute the product of all the common
factors and return it as the greatest common divisor
of the numbers given.
• Can we directly implement this procedure on
computer?
• The non ambiguity requirement for each step of an
algorithm cannot be compromised.
Greatest Common Devisor Problem
Euclid’s Algorithm
Step 1 if n = 0, return the value of m as
the answer and stop; otherwise
proceed to step 2.
Step 2 Divide m by n and assign the
value of the remainder to r.
Step 3 Assign the value of n to m and the
value of r to n. Go to Step 1
Greatest Common Devisor Problem
Euclid’s Algorithm
• This algorithm is based on applying
repeatedly the equality
gcd(m, n) = gcd(n, m mod n)
Until m mod n is equal to 0;
• gcd(m, 0) = m
Greatest Common Devisor Problem
Euclid’s Algorithm
• Alternatively, we can express the same algorithm in a
pseudo code:
Euclid(m, n)
//Compute gcd(m, n) by Euclid’s algorithm
//Input: Two nonnegative, not-both-zero integers m and n
//Output: Greatest common divisor of m and n
While n != 0
r = m mod n
m=n
n=r
return m
• The same algorithm can be represented in several
different ways
 Conclusions so far
• The non ambiguity requirement for each step of an
algorithm cannot be compromised.
• The range of inputs for which an algorithm works has to
be specified carefully.
• The same algorithm can be represented in several
different ways.
• Several algorithms for solving the same problem may
exist.
• Algorithms for the same problem can be based on very
different ideas and can solve the problem with
dramatically different speeds.
Design & Analysis Example
• Suppose you have an array of n distinct
elements.
• Write an algorithm that picks ith array
element and place it at the position such
that all array elements present at left are
less then it and all array elements present
at right are greater then it.
• Also gives the time complexity of your
algorithm in  notation.
Design & Analysis Example….
number = A[i];
smaller[n], greater[n], s = 1, g = 1;
for(j = 1; j <= n; j++)
if(A[j] < number)
smaller[s++] = A[j];
else if(A[j] > number)
greater[g++] = A[j];
for(j = 1; j < s; j++)
A[j] = smaller[j];
A[j] = number;
for(k = 1; k < g; k++)
A[++j] = greater[k];
 Algorithms Analysis: Reference Functions

n f(n) = lgn f(n) = n f(n) = nlgn f(n) = n2 f(n) = n3 f(n) = 2n

10 0.003 micro sec 0.01 micro sec 0.033 micro sec 0.1 micro sec 1 micro sec 1 micro sec
20 0.004 micro sec 0.02 micro sec 0.086 micro sec 0.4micro sec 8 micro sec 1 milli sec
30 0.005 micro sec 0.03 micro sec 0.147 micro sec 0.9 micro sec 27micro sec 1 sec
40 0.005 micro sec 0.04 micro sec 0.213 micro sec 1.6 micro sec 64 micro sec 18.3 min
50 0.006 micro sec 0.05 micro sec 0.282 micro sec 2.5 micro sec 125 micro sec 13 days
1 02 0.007 micro sec 0.10 micro sec 0.664 micro sec 10 micro sec 1 milli sec 4 exp 13 years
1 03 0.010 micro sec 1.00 micro sec 9.966 micro sec 1 milli sec 1 sec
1 04 0.013 micro sec 10 micro sec 130 micro sec 100 milli sec 16.7 min
1 05 0.017 micro sec 0.10 milli sec 1.67 milli sec 10 s 11.6 days
1 06 0.020 micro sec 1 milli sec 19.93 milli sec 16.7 min 31.7 years
1 07 0.023 micro sec 0.01sec 0.23 sec 1.16 days 31709 years
1 08 0.027 micro sec 0.10 sec 2.66 sec 115.7 days 3.17 exp 7 years
1 09 0.030 micro sec 1 sec 29.90 sec 31.7 years
 Algorithms Analysis Framework
Asymptotic Notations (Upper Bound)
There may be a situation, e.g.
g(n)

f(n)
Tt

1 n0 n
f(n) <= g(n) for all n >= n0 Or
f(n) <= cg(n) for all n >= n0 and c = 1
g(n) is an asymptotic upper bound on f(n).
f(n) = O(g(n)) iff there exist two positive constants c and n 0 such that
Algorithms Analysis Framework
Asymptotic Notations (Lower Bound)

Asymptotic Lower Bound: f(n) = (g(n)),

iff there exit positive constants c and n0 such that
f(n) >= cg(n) for all n >= n0

f(n)

g(n)

n0 n
Algorithms Analysis Framework
Asymptotic Notations (Tight Bound)
Asymptotically Tight Bound: f(n) = (g(n)),
iff there exit positive constants c1 and c2 and n0 such that
c1 g(n) <= f(n) <= c2g(n) for all n >= n0

c2g(n)
f(n)
c1g(n)

n0 n
Algorithms Analysis Framework
Asymptotic Notations (Example)
Example: show that (1/2)n2 – 3n = (n2)
 To do so we must determine positive constants c1, c2
and n0 such that c1n2 <= (1/2)n2 – 3n <= c2n2, n >=
n0
 Dividing by n2 c1 <= ½ - 3/n <= c2
 Right Hand Inequality ½ <= c2 + 3/n
 For positive n, if c2 >= ½ then the inequality holds.
 Left Hand Inequality c1 + 3/n <= ½
 For n = 7 and c1 <= 1/14, the inequality holds.
 c1 <= 1/14, c2 >= ½ and n = 7
Algorithms Analysis Framework
Asymptotic Notations (Rules)
1. If T1(n) = O(f(n)) and T2(n) = O(g(n))
Then T1(n) + T2(n) = Max(O(f(n)), O(g(n)))
T1(n) * T2(n) = O(f(n) *g(n))
2. If T(x) is a polynomial of degree n, then
T(x) = (xn)
3. logk(n) = O(n) for any constant k. This tells
that logarithms grow very slowly.
4. Do not include any constants or low order
terms inside a big-Oh, e.g.,
T(n) = O(2n2) -------- wrong
T(n) = O(n2 + n) ----- wrong
 Algorithms Analysis

• The complexity of an algorithm is a function

g(n) that gives the upper bound of the number
of operation (or running time) performed by an
algorithm when the input size is n.
• There are two interpretations of upper bound.
• Worst-case Complexity
– The running time for any given size input will be
lower than the upper bound except possibly for some
values of the input where the maximum is reached.
• Average-case Complexity
– The running time for any given size input will be the
average number of operations over all problem
instances for a given size.