AD3351-

DESIGN AND ANALYSIS OF

ALGORITHMS

Handling By,
B.SANKARALAKSHMI,
AP/AI &DS,
RIT.
 UNIT - I
INTRODUCTION
Algorithm - Definition

? An algorithm is a sequence of unambiguous

instructions for solving a problem ie, for obtaining a
required output for any legitimate input in a finite
amount of time.
? A finite sequence of steps (or) instructions to solve a
given problem (or) to accomplish a task.
Design of Algorithm

? Understand the problem statement.

? Develop an efficient algorithm.
? A single problem - Variety of algorithms.
? An efficient algorithm – converted into a program.
Analysis of Algorithm

? Able to measure the efficiency of the algorithm.

? Analysis – in terms of time and space aspects.
? Time Efficiency (or) Time Complexity – time taken
to execute and produce the output.
? Space Efficiency (or) Space Complexity – memory
space occupied.
Why should we study Algorithms?

? A computer science person – able to design

algorithms and analyze its efficiency.
? No computer programs without algorithms.
? The Study of algorithms – Algorithmics – The Spirit
of Computing.
? Special kind of solutions to problems.
? Same problem – Different algorithms available.
Difference between Algorithm and Program

S.No.: Algorithm Program

1. Step-by-step instruction to Sequence of instructions
solve a problem executed by a computer
2. First algorithm is designed Algorithm is implemented with
the help of any programming
language
3. No syntax rule is followed Syntax rule is strictly followed
4. Doesn’t raise errors Raises errors and Debug it
5. Not executed by computer Executed by computer
6. Written in any natural language Written in any programming
language
7. Analyzed in terms of efficiency Testing is done to ensure the
requirements
Continued…
Algorithm Program
Eg: Eg:

1.Start the program. print(“Enter two numbers:”)

2.Input a and b values. n1=int(input())
3.If a>b then print a is greatest. n2=int(input())
4.Else print b is greatest. If n1>n2:
5.Stop the program. print(“\n Largest Number=“,n1)
else:
print(“\n Largest Number=“,n2)
Pseudo code

? Step–by-step description of an algorithm.

? Written in natural language along with mathematical
logic.
? Mainly for human understanding than machine
understanding.
? Intermediate state between algorithm and program.
? Doesn’t use any syntax.
Difference between Algorithm and Pseudo code
S.No.: Algorithm Pseudo code
1. Step-by-step procedure to Step-by-step description of
solve a problem algorithms
2. Special kind of solution to Method of writing algorithm with
problems logic
3. Not required to specify aim, Specify aim, input and output in
input and output beginning
4. Written in natural language Written in natural language with
mathematical logic
5. No specific syntax rules, No specific syntax rules, doesn’t
doesn’t raise error and not raise error and not required to
required to execute execute
Example : Pseudo code

//Computes Greatest number

//Input: Two values a & b
//Output: Greatest number a or b
if(a>b)
print “a is Greatest”
else
print “b is Greatest”
Some Important Points on Algorithm
Properties (or) Characteristics of Algorithms

? Input: Zero or more inputs.

? Output: At least one output and correct output.
? Definiteness: Unambiguous and clear instruction.
? Effectiveness: Simple instructions.
? Finiteness: Finite sequence of instructions.
Notion of Algorithm
Continued..

To illustrate the notion of the algorithm we consider 3

methods to solve the same problem – Computing the
greatest common divisor of 2 non-negative, not-both-
zero integers m,n => gcd(m,n)

? Euclid’s Algorithm
? Consecutive Integer Checking Algorithm
? Middle-School Algorithm
GCD(m,n) - Definition

? gcd(m,n) – the largest integer that divides both m and

n evenly, ie., with a remainder of zero.
? General method – gcd(m,n)

Eg: Compute GCD(60,24)

• Find common factors of 2 gn nos:

• Product of those common factors – gcd

=> gcd(24,18)=?
Euclid’s Algorithm:

* invented by Euclid of Alexandria – called Euclid’s

algm.
Compute gcd(60,24) using Euclid’s algorithm.
Compute gcd(60,24) using Euclid’s algorithm

Given: m=60 and n=24

? Iteration 1:

n=24≠0 -> r=60 mod 24 -> r=12

Now, m=n=24 and n=r=12
? Iteration 2:

n=12≠0 -> r=24 mod 12 -> r=0

Now, m=n=12 and n=r=0
? Iteration 3:

n=0, condition fails and returns m value

Therefore, gcd(60,24)=12.
Middle-School Algorithm

Compute gcd(24,12) using Middle-School algorithm.

GCD(24,12)

Given: m=24 and n=12

Step 1: prime factors of m => 2,2,2,3
Step 2: prime factors of n => 2,2,3
Step 3: common factors => 2,2,3
Step 4: product of common factors =>
2x2x3=12.
Demerits:

? Much more complex and slower than Euclid’s

algorithm
? Lower than Euclid’s algorithm in efficiency – inferior
efficiency
? Not a legitimate algorithm – ambiguous steps
? If large integers – complicated – list of prime
numbers
Consecutive Integer Checking Algorithm
Continued…

Eg: (i) Compute gcd(9,6) using Consecutive Integer

Checking Algorithm
(i) Compute gcd(24,12) using Consecutive Integer
Checking Algorithm
(iii) Compute gcd(60,24) using Consecutive Integer
Checking Algorithm

Drawback:

Doesn’t work properly when one of its inputs is ZERO

Sieve of Eratosthenes

? Simple algorithm for generating consecutive primes

not exceeding any given integer n>1
? Starts by initializing a list of primes from 2 to n
? First Iteration – eliminate all multiples of 2 except 2
? Next Iteration - eliminate all multiples of 3 except 3
? Continuous until no more elements to be eliminated
? Remaining integers – primes
Algorithm : Sieve of Eratosthenes

Step1: Create a list of consecutive integers from 2 through n –

(1,2,3,…..,n).
Step 2: Initially let p=2 the smallest prime no.
Step 3: Enumerate the multiples of p by counting in increments
of p from 2p to n and mark them in the list (these will be
2p,3p,4p,…, the p itself should not be marked).
Step 4: Find the smallest no in the list greater than p that is not
marked. If there is no such no stop. Otherwise let p now equal
to this new no and repeat from Step 3.
Step 5: When the algorithm terminates the numbers remaining
not marked in the list are all the primes below n.
Continued…

? Incorporate Eratosthenes algorithm in middle-school

algm –> make it as a legitimate one
Fundamentals of Algorithmic Problem Solving
? Sequence of steps to design and analyze an algorithm
1. Understanding the Pbm

? Before designing algm – understand pbm

? Read pbm descriptions – special cases
? Specify range of i/ps – works properly
? Know strength and weaknesses of algm – to produce
correct o/p
? An i/p – instance of pbm
? Important to specify set of instances
? Correct algm – works properly for all legitimate i/ps
2.(i) Ascertaining the capabilities of the
Computational Device
? Choose computational device depends on type of
algm
? 2 types: i) Sequential Algm ii) Parallel Algm
? One after another – Sequential
? Concurrently - Parallel
? For complex pblms – huge volumes of data – speed
and m/y available
2.(ii) Choosing between exact and approximate
problem solving:
? Choose among solving exactly or approximately
? Exact pbm solving – exact algm
? Approximate pbm solving – approximation algm
? Exact algms – can’t solve exactly bcz pbm
complexity
? Approximate algm – sometime solves exactly
? Approximate solving – Eg: extracting square roots,
solving non-linear eqns and evaluating definite
integrals
2.(iii) Algorithm Design Techniques

? To solve pbm algorithmically

? Defn – An algm design tech (or) strategy (or)
paradigm is a general approach to solve pbms
algorithmically – applicable to variety of pblms
? Design Tech – Dynamic Pgmmg, Greedy Tech,
Divide and Conquer etc.,
? Designing algms for new pblms – no satisfactory
algm
? Can classify algms a/c to designing tech
3.(i) Designing an Algorithm and Data
Structures
? Choose data structure for operations performed by
algm
? Data Structure – store, organize and retrieve data in
efficient manner ie., space and time
? Eg: Array, LL, Stack, Q etc.,
? Sieve of Eratosthenes – used array – LL, instead of
array – run longer
? Pgm = algm + data structure
? DS – important for both design and analysis of algms
3.(ii) Methods of specifying an algorithm

? Need to specify an algm in some style

? Eg: Euclid’s algm specified in words and pseudocode

? 3 ways to specify an algm

(a) Natural Language (English)

(b) Pseudocode
(c) Flowcharts
(a) Natural Language:
? Specified in English –> step-by-step procedure
Continued…

(b) Pseudo code:

?Mixture of natural language and programming

language
?Consists of pbm description, i/p to be supplied and o/p

to be produced – beginning
?Followed by instructions of algm

?Use “<-” - assignment operation

“//” - comments
Continued…

(c) Flowchart:
?Ancient method to specify an algm

?Pictorial representation

?Collection of connected geometric shapes containing

descriptions of algm’s steps

?Flow of data – links

* First 2 ways widely used to specify algms

* Finally algms converted into pgms – implementation
4. Proving an Algorithm’s correctness
? Once an algm specified – prove its correctness
? ie., prove it yields required result for every legitimate
i/p in finite amount of time
? Eg: proving correctness of Euclid’s algm –
correctness of gcd(m,n)=gcd(n, m mod n)
? Second integer gets smaller on every iteration and
stops when it becomes 0
? Just for one instance of i/p algm fails – incorrect algm
? Proving correctness – not straight forward for
approximation algms – error produced not exceed
predefined limit
? Proving correctness – straight forward for exact
algms
5. Analyzing an Algorithm
? An algm – several qualities
? After correctness – important quality is efficiency
? 2 kinds of efficiency – time and space efficiency
? Time efficiency – how fast it runs
? Space efficiency – how much extra m/y it requires
? Simplicity – easier to understand and program
? Generality – generality of problem and set of i/ps
accepts
? If not satisfied with efficiency (or) simplicity (or)
generality – redesign algm
6. Coding an Algorithm
? Once time and space efficiency satisfied –
transformed into pgm
? Once pgm written – testing is done - validity
? If any errors – debug it – check validity
Important Problem Types

? Important problem types are:

? Sorting
? Searching
? String Processing
? Graph Problems
? Combinatorial Problems
? Geometric Problems
? Numerical problems
(a) Sorting
? Rearrange the given elements - ascending or descending order
? Sorting of lists of no:s, characters, strings and student records
? Student records - sorting based on a key value
? Eg: alphabetical order - names, reg no, marks
? Based on which attribute - key attribute
? Why do we sort elements? - to make many tasks easier, Eg:
Searching
? Eg: Dictionaries and telephone dictionaries - alphabetical order
- searching name or word easier
? Sorting is important
Continued…
? Sorting algms available - Eg: Insertion sort, Merge sort,
Quick sort etc.,
? No algm is best in all circumstances
? Some algms simple but slow, some faster but more
complex
? Some work on randomly ordered i/ps while others on
sorted i/ps
? For sorting algms - 2 properties
? (i) Stable - preserves the order of any two equal elements
in its i/p
? ie., if i<j in i/p list then i’<j’ in sorted list too
Continued…
? Eg: Name Percentage
A 90
B 75
C 80
D 75
? Sorted in alphabetical order where B comes before D
? Sort in % order - decreasing order
? Sorted by stable algm - Name Percentage
A 90
C 80
B 75
D 75
Continued…

? (ii) In-Place - if it doesn’t require extra m/y

? some are in-place while some others are not
(b) Searching

? Searching pbm - finding a given value called a search

key in a list
? Many searching algms available
? No algm is best in all the situations
? Some faster but requires more m/y
? Some others faster but only applicable to sorted
arrays
(c) String Processing
? String - a sequence of characters
? Text Strings - comprise zeros, letters, no:s and special
characters
? Bit Strings - zeros and ones
? String processing - dealing with the strings
? In String processing - Searching also done
? Searching for a given word in text - String Matching
? Several String Matching algms available
?
(d) Graph Problems
? A Graph - collection of vertices and connected by edges
? Graphs used in a variety of applications
? Such as - transportation, communication, social n/ws,
project scheduling, games etc.,
? Basic Graph pblms - Graph traversal algms
? Such as - Depth-First Search(DFS), Breadth-First
Search(BFS), Shortest-Path algm and Topological
Sorting for graphs with directed edges
? Some Graph algms hard - Eg: Traveling Salesman pbm
and Graph-Coloring pbm
Continued…

? Traveling Salesman Pbm - pbm of finding the

shortest path through n cities that visits every city
exactly once
? Graph Coloring Pbm - assign smallest no: of colors to
the vertices - no 2 adjacent vertices have same color
? Eg: Event Scheduling - events are vertices connected
by an edge only if they aren’t scheduled at same time
(e) Combinatorial Problems
�
Deals with combinatorial objects - permutation, a
combination or a subset
�
Eg: Traveling Salesman Problem(TSP) and Graph-
Coloring problem, Knapsack problem etc.,
�
Combinatorial object - maximum value or minimum
cost
�
are most difficult problems to compute
�
Bcz no: of combinatorial objects - grows fast with
pbm’s size
�
No known algms to solve exactly
(f) Geometric Problems
�
Deal with geometric objects - points, lines and
polygons
�
Used in computer graphics and robotics etc.,
�
Eg: Closest-pair pbm and Convex-Hull pbm
�
Closest-pair pbm -> is self-explanatory
�
-> Gn n points, find the closest pair
�
Convex-Hull pbm -> find smallest convex includes
all points
(g) Numerical Problems
�
Involve mathematical objects
�
Solving equations and s/ms of equations, computing
definite integrals, evaluating fns etc.,
�
Such pbms solved approximately
�
Require manipulating real no:s - represented
approximately
Fundamentals of the Analysis of Algorithm
Efficiency
�
Analysis of algms - investigation of an algm’s
efficiency - time and m/y space
�
3 Notations - O (Big Oh), Ω (Big Omega),
Θ ( Big Theta) - efficiency of algms
The Analysis Framework
� 2 kinds of efficiency
� (a) Time Eff - how fast an algm runs
� (b) Space Eff - amount of m/y units required and
additional space needed
� Factors affect space eff:
� Pgm space - space required for pgm by compilers
and assemblers
� Data space - to store constants, variables etc.,
� Stack space - to store return addresses with
parameters passed to fns
� Space complexity - not a concern - fast main m/y,
slower secondary m/y and cache
Continued…
� Factors affect Time Eff:
� (a) Speed of the computer - used
� (b) Pgmmg Lang - chosen
� (c) Compiler used - run
� (d) Choice of algm - chosen - many algms for same
pbm
� (e) i/p size and o/p
� First 3 factors - not under pgmmer’s control
� Last 2 - under pgmmer’s control
� Among many - choose better algm
� I/p size - major factor - Eg: sort larger arrays,
multiply larger matrices etc.,
Continued…
�
I/p size - expressed as ‘n’ - no: of elements
�
Time Eff ∝ i/p size ‘n’
�
If ‘n’ smaller - less time
�
If ‘n’ larger - more time
�
‘n’ - not same for all pblms
�
Eg: for sorting and searching - ‘n’ is no: of elements
�
for polynomial evaluation - ‘n’ is degree or
coefficients
�
for matrix multiplication of matrices - size or order
of the matrix
Units for Measuring Running Time
� Use std unit of time - sec or millisec
� Drawbacks - dependence on speed of a computer,
quality of pgm and compiler used
� General approach - count the no: of times the
operations is executed
� Identify - most important operation - basic operation
of an algm
� Compute the no: of times the basic operation is
executed
� Basic operation - most time consuming operation -
innermost loop
� Eg: Sorting - key comparison - basic operation
Continued…
�
Algms - mathematical pblms - +ion, _ion, xion and
/ion
�
Most time consuming operation - /, x, _ and +ion
�
Running time T(n) is computed by:
T(n)=Cop.C(n)
�
Cop - execution time of a basic operation
�
C(n) - no: of times basic operation executed ie., a/c
to i/p size
Basic Efficiency Classes
� Different categories of efficiency classes with its running
time
� Class and Name of efficiency
(i) 1: Constant
� Constant value - Eg: 1
(ii) Log n: Logarithmic
� Run time is logarithmic
� Problem size is reduced by constant factor - each
iteration
� Eg: Binary Search
(iii) n : Linear
� Running time is linear
� ie., problem size ∝ i/p size
� Eg: Linear Search
Continued…

(iv) n log n : Linearithmic

�
Running time - pdt of n and log n
�
Eg: Quicksort, Mergesort, Heapsort
(v) n² : Quadratic
�
Running time is quadratic
�
‘n’ is i/p size - algms have 2 for loops
�
Eg: Bubble sort, Matrix +ion, Matrix _ion
(vi) n³ : Cubic
�
Running time is cubic
�
Algms have 3 for loops
�
Eg: Matrix xion
Continued…

(vii) 2ⁿ : Exponential
�
Running time is exponential
�
Run time depends on exponential of ‘n’ (i/p size)
�
Eg: Towers of Hanoi
(viii) n! : Factorial
�
Running time is Factorial of ‘n’
�
Eg: Permutations, Combinations etc.,
❖
2ⁿ and n! - exponential type - computed only for
smaller i/p size
❖
If ‘n’ is larger - generate higher values
Continued…

Table - Order of Growth

Continued…
�
In order of growth table - exponential values grow
faster even for smaller values of ‘n’
�
Algms with linear running time is preferable
�
From lowest to highest is:
1 < log n < n < n log n < n² < n³ < 2ⁿ < n!
�
Lowest running time - 1
�
Highest or exponential running time - 2ⁿ, n!
�
Exponential running time - hard to compute
�
2ⁿ and n! - exponential growth fns
Worst-Case, Best-case and Average-Case
Efficiencies
�
Measure algm’s eff - size of i/p
�
Consider Eg: Sequential Search
�
Search key ’K’ - in list of elements
�
Checking successive elements - until matches or
ends
Pseudocode : Sequential Search
Continued…
�
Sequential search - Linear search
�
Running time is different - same no: of i/ps
�
Eg: An i/p array 10 14 5 8 7 18
�
Key ‘K’ = 8
�
Requires 4 iterations
�
End of 4th iteration - elt found
(i) Worst-Case Eff:
�
Either elt not found or last elt
�
Makes largest no: of comparisons
�
TC - Cworse (n) = n - no: of elts
Continued…
(ii) Best-Case Eff:
�First elt is search key ‘ K’
�TC - Cbest (n) = 1
(iii) Average-Case Eff:
�Neither worst-case nor best-case
�provides eff on random i/p
�2 cases
�(a) Successful Search - elt found at ith position
- No: of comparisons is ‘i’
- Avg no: of key comparisons:
Cavg(n) = (n+1)/2
�(b) Unsuccessful Search - elt not found
-- Avg no: of key comparisons = ‘n’
Asymptotic Notations

� Efficiency analysis - Order of Growth of an algm’s

basic operation count
� Asymptotic notations - mathematical notations - TC
of algms
� To compare and rank order of growth :
(i) O (Big Oh)
(ii) Ω (Big Omega),
(iii) Θ ( Big Theta)
Continued…

(i) O (Big Oh):

�O(g(n)) - Set of all fns with lower or same order of
growth as g(n)
�g(n) - non-negative fn to compare the count with
�Eg:

�First two - linear - lower order of growth - n than g(n)=

n²
�Last one - quadratic – same order of growth as g(n)=n²
�O (Big Oh) notation - f(n) ≤C(g(n))
Continued…

(ii) Ω (Big Omega):

�Ω (n) - Set of all fns with a higher or same order of
growth as g(n)
�Eg: n³ ∈ Ω(n²), ½ n(n-1) ∈ Ω(n²), 100n+5 ∉ Ω(n²)
�First two - Ω notation
�Last one - doesn’t – belongs to O notation
�Ω notation - f(n) ≥ C(g(n))
Continued…

(iii) Θ ( Big Theta):

�Θ (n) - Set of all fns with same order of growth as
g(n)
�Eg: n² ∈ Θ(n²), 200n+5 ∈ Θ(n), n(n-1) ∈ Θ(n²)
�Θ notation - f(n) = C(g(n))
Formal Definitions of All Notations
(i) O (Big Oh) Notation:

?
t(n) – Time Complexity
?
t(n) increases as ‘n’ increases
?
If ‘n’ smaller - less time
?
If ‘n’ larger – more time
?
Difficult to specify TC up to value
?
Only after t(n) increases as n increases
?
c(g(n)) – upper bound – ie., worst case eff.
?
Worst-case eff is expressed as ‘O’ notation
Continued…
(ii) Ω (Big Omega) Notation:

?
t(n) – running time and c(g(n)) – least running time
?
If algm takes least run time – Ω notation
?
Ω notation – lower bound or best-case eff.
Continued…
(iii) Θ ( Big Theta) Notation:

?
As ‘n’ increases –
?
- above, - below
?
If algm’s time between above and below –
avg-case eff.
?
Θ notation – avg-case eff.
Continued…
Useful Properties Involving the Asymptotic
Notations
?
Can prove general properties of asymptotic
notations
?
Following property – 2 consecutive execution parts
?
Assume 4 real no:s -
?
If then
?
Since there exist some
positive constant and some non-negative
integer
?
such that
?

?
Continued…
When adding,

Hence with the constants

required by the definition being
and respectively.
Continued…
?
For eg., Searching a key elt in an array – not sorted
– Binary Search
?
B4 binary search – need to sort
?
Apply search algm – once sorted
?
- TC of sorting operation
?
- TC of searching elt
?
Finally TC of Binary search –
Properties of Asymptotic Notations
(i) General Properties:
?
If f(n) is O(g(n)) – a * f(n) = O(g(n)) – ‘a’ is constant
?
Eg: f(n)=2n²+5 ie., 2n²+5=O(n²)
?
Assume a=7, => 7*f(n) = 7*(2n²+5) = O(n²)
?
|||ly 14n²+35 = O(n²)
?
Satisfies both Ω and Θ notations
?
ie., f(n) is Ω(g(n)) – a * f(n) = Ω(g(n))
f(n) is Θ(g(n)) – a * f(n) = Θ(g(n))
(ii) Reflexive Property:
?
If f(n) given – f(n) = O(f(n))
?
Eg: f(n) = (n²) means – O(f(n)) = O(n²)
?
True for both Ω and Θ notations
?
ie., f(n) gn, f(n) = Ω(f(n)) and f(n) = Θ(f(n))
 (iii) Transitive Property:
?
If f(n) = O(g(n)) and g(n) = O(h(n)) then
f(n) = O(h(n))
?
Eg: f(n) = n, g(n) = n² and h(n) = n³
f(n) = O(g(n)) = O(n²) = n
g(n) = O(h(n)) = O(n³) = n²
f(n) = O(h(n)) = O(n³) = n
?
|||ly This property is true for Ω and Θ notations
?
ie., If f(n) = Θ(g(n)) and g(n) = Θ(h(n)) then
f(n) = Θ(h(n))
?
If f(n) = Ω(g(n)) and g(n) = Ω (h(n)) then f(n) = Ω (h(n))
(iv) Symmetric Property:
?
If f(n) = Θ(g(n)) then g(n) = Θ(f(n))
?
Eg: f(n) = n² and g(n) = n² then
f(n) = Θ(n²) and g(n) = Θ(n²)
?
This property is true only for Θ notation
(v) Transpose Symmetric Property

?
If f(n) = O(g(n)) then g(n) = Ω(f(n))
?
Eg: f(n) = n, g(n) = n² then
f(n) = O(n²), g(n) = Ω(n)
?
This property is only true for O and Ω
notations
 Some More Properties:
(a) If f1(n) = O(g1(n)) and f2(n) = O(g2(n)) then
f1(n) + f2(n) = O(g1(n)) + O(g2(n)) =
O max{g1(n),
g2(n)}

? Eg: g1(n) = n = O(n) => f1(n)

g2(n) = n2 = O(n2) =>f2(n)

Therefore f1(n) + f2(n) = O max{(n), (n2)} = O(n2)

(b) If f1(n) = O(g1(n)) and f2(n) = O(g2(n)) then f1(n) *
f2(n) = O(g1(n) * g2(n))
? Eg: g1(n) = n2 and g2(n) = n3

Therefore f1(n) = O(n2), f2(n) = O(n3)

⇒ f1(n) * f2(n) = O(g1(n) * g2(n))
⇒
= O(n2 * n3)
⇒
= O(n5)

⇒ f1(n) * f2(n) = O(n5)

(c) If f(n) = O(g(n)) and f(n) = Ω(g(n)) then
f(n) = Θ(g(n))
(d) f(n)
=0

f(n) ∈ O(g(n))
f(n) ∉ Θ(g(n))
(e) Any constant C => O(1)
f(n) = C => f(n) = O(1)
(f) Polynomial property => higher order or higher
degree is preferred
Eg: 5n4 + 3n3 + 2n2 + 6n + 7 => O(n4)
Using Limits for Comparing Orders of Growth
?
Use formal definitions of O, Ω, and Θ – compare
orders of growth of two fns (t(n),g(n))
?
Compute the limit of the ratio of 2 fns – 3
cases
?
Continued…
?
First 2 cases -> ≤ -> O notation –>
?
Last 2 cases -> ≥ -> Ω notation –>
?
Second case alone -> = -> Θ notation ->
Mathematical Analysis of Non-Recursive
Algorithms
?
Also called Iterative algm
?
Involves a series of instructions – executed
repeatedly – condition is met
?
Until a soln achieved
?
Use of loops – for or while or do-while
General Plan for Analyzing the Time Efficiency
of Non-Recursive Algorithms
1. Decide on parameter – i/p size
2. Identify basic operation
3. Check – no: of times of basic opn – depends on i/p
size or other property
4. Also depends other property – worst, avg, best case
eff
5. Sum no: of times execution
6. Using std formula – order of growth
Analyze the efficiency of above algm
Continued…
1. i/p size – no: of elts in array
2. Basic operation – comparison, assignment –
comparison
3. No: of times – depends only on i/p size
4. No need – WC, AC and BC eff
5. Sum no: of times execution
6. C(n) – no: of times comparison executed
7. Executed from 1 to (n-1)
8. Sum:
=>
Continued…

1. i/p size – n
2. Basic operation – comparison
3. No: of times – not only on ‘n’ also on equal elts – if
so occupied positions noted
4. WC, BC and AC eff
5. BC eff – if 2 equal elts on 1st 2 positions
6. WC eff – either no equal elts or on last 2 positions
7. WC - one comparison for each j & i value ie., i+1
to n-1 and 0 to n-2
Worst-case eff is as follows:

- - Apply summation formula 1,

Substituting the limits =>

(n -1) + (n – 2) + (n – 3) +……….+1
Apply summation formula 2,
Continued…

= 1 + 2 + 3 + …… + (n – 2) + (n – 1)

= (n – 1) (n – 1 + 1)/2

=
Continued…
?
i/p size – ‘n’ – order of matrix
?
Basic operation - xion and +ion
?
Compute no: of times of xion – only on i/p size
?
Don’t investigate – WC, BC and AC eff
?
Only one xion – each repetition
?
Total no: of xions for each i & j is,

=>

?
Therefore, TC = n3 ∈ O(n3)
Mathematical Analysis of Recursive Algorithms
?
Recursive algm – a fn calls itsef – recursive fn
?
Eg: Towers of Hanoi, tree-traversals –
In-order, Pre-order, Post-order, DFS
General Plan for analysing the Time Efficiency
of Recursive Algorithms
1. Decide on i/p size
2. Identify Basic operation
3. Check – no: of times basic operation executed vary
on diff i/p of same size
4. If so – WC, BC and AC eff
5. Set a Recurrence relation with initial condition
6. Solve the Recurrence relation or order of growth its
soln
A recurrence relation is a mathematical expression
that defines a sequence in terms of its previous terms.
In the context of algorithmic analysis, it is often used
to model the time complexity of recursive algorithms.
Continued…
1. i/p size – n
2. Basic operation – xion – M(n):No of times
3. F(n) is,

1. M(n) satisfy equality,

1. M(n – 1) xions spent to compute F(n – 1)

2. 1 or more xions to xly F(n – 1) by n
3. M(n) eqn – fn of value at (n – 1) - Recurrence relation
4. Recurrence relation or recurrence – vital role – algm’s
analysis
5. Now – solve recurrence : M(n) = M(n – 1) + 1
Solving Recurrence:
? In algm – Initial Condition –

? ie., F(0) = 1, When n = 0, no: of executions =>

Recurrence with
initial condition =>

F(n) by recurrence =>

 on on

We will get =>

on above exp.

Therefore general formula is =>

The Recurrence relation of the above recursive algm is =>

‘n’

Therefore TC of Factorial using recursion=C(n)=O(n)

The Tower of Hanoi Puzzle
?
‘n’ no: of disks of diff sizes
?
All placed on 1st peg – in order of size
?
Largest – bottom, smallest – top
?
Goal – move all to 3rd peg
?
Use 2nd peg – auxiliary – if necessary
?
At a time – move 1 disk
?
Don’t place – largest on smallest
?
2 cases – n=1 and n>1
?
Case1: n=1 => move 1 disk directly from source
peg(1st) to destination peg(3rd)
Continued…
?
Case2: n>1 => follow 3 steps
(i) recursively move (n-1) disks from peg1 to
peg2 (peg3 – auxiliary)
(ii) move largest from peg1 to peg3 directly
(iii) recursively move (n-1) disks from peg2 to
peg3 (peg1 – auxiliary)
Apply General Steps

1. i/p size – ‘n’ – no: of disks

2. Basic operation – moving 1 disk
3. No: of moves – M(n) – depends only on ‘n’
4. Recurrence relation =>

1. Initial condition => M(1) =1

2. Recurrence relation =>
Solving Recurrence Relation
? Solve using backward substitution =>
--------------------- (1)

Substitute M(n-1) on eqn (1)=>

--------------(2)
|||ly on eqn (2)

----------------(3)
Sub M(n-3) on eqn (3), we get,
After i substitutions we get,

Since initial condition is n=1, achieved for i=(n-1)

We get,

When recursive algm - more than 1 time – construct a tree

of recursive calls
Nodes – recursive calls
Label with value of parameters of calls
Tree for Tower of Hanoi =>

No: of calls – no: of nodes in tree => ie.,

Therefore TC of Tower of
Hanoi=C(n)=O(2n)
Example:3 Recursive algorithm to find the no: of binary digits
in the representation of a positive decimal integer
1. Set up recurrence relation with initial condition - no: of
+ions, A(n)
2. No: of +ions in => + 1 to

∴ recurrence relation -
↑se returned value by 1
3.

1. When n=1, initial condition => A(1) = 0

2. Std approach - only for n = 2 ⁱ - correct order of growth

for all values of n

3. Apply on

Sub n = 2ⁱ,
for k = 0,
Now go for backward substitutions,

—----------------(1)
Sub k=(k-1) =>
Sub on eqn (1)

—----------(2)
To compute , Sub k = k - 2 on eqn (1)

Sub on eqn (2)

We end up with,
Example:4 Compute Fibonacci series of ‘n’
Consider the Fibonacci series,

defined by recurrence =>

with 2 initial conditions =>

1. Basic operation - +ion
2. A(n) - no: of times - to compute F(n)
3. A(n - 1) and A(n - 2) - no: of times - to compute F(n - 1)
and F(n - 2)
4. Recurrence for A(n) =>

Eg: Compute F(4) where n=4 ie., n >1

∴ F(n) = F(n-1) + F(n-2)

F(4) = F(3) + F(2)
As we know, F(0) = 0, F(1) = 1
F(4) = 0, 1, 1, 2
❏
∴ 5 fn calls made for F(4)
❏
Compute value - every new F(n) - ie., F(0), F(1),
F(2),.....
❏
If recursive algm not used - need 9 fn calls to compute
F(4)
❏
ie., F(0) - 2 times
F(1) - 3 times
F(2) - 2 times
F(3) - 1 time and
F(4) - 1 time => 9 fn calls
❏
We get TC as, A(n) ∈ O(n)
Figure: Tree of Recursive calls for computing 4th
Fibonacci number
F(n)=F(4)
Empirical Analysis of Algorithms
❏
Empirical Analysis - alternative to mathematical
analysis of algorithms
❏
Mathematical analysis - recursive and non-recursive
algms
❏
Some simple algms - analyzed mathematically
General Steps for the Empirical Analysis of
Algorithms
1. Understand the experiment’s purpose
2. Decide on efficiency metric (M) - operation count
and time unit
3. Decide on i/p size, range etc.,
4. Prepare a program implementing algm
5. Generate sample of i/ps
6. Run on sample i/ps and observe data
7. Analyze data obtained

❖Completed 7 steps - represent data - table or

scatterplot (graphical representation)
❖Scatterplot form - ensure algm’s eff class
(a) Logarithmic Algorithm
❏
Scatterplot -
Concave shape -
logarithmic algm
(b) Linear
❏
Linear algm - points
around straight or
linear line
(c) One of the Convex functions
❏
Scatterplot -

- Convex shape
Algorithm Visualization
❏
Third way - represent algms - algm visualization
❏
Use images - informations about algms
❏
Visual Illustration - operation, performance,
execution speed
❏
Uses graphic elements - points, lines, 1D & 2D bars
❏
2 variations
(a) Static algm visualization
(b) Dynamic algm visualization
(algm animation)
(a) Static Algorithm Visualization
❏
Algm’s progress - series of still images

(b) Dynamic Algorithm Visualization

❏
Algm animation - continuous movie
❏
difficult to implement
❏
Animation s/ms used - BALSA, TANGO and ZEUS
❏
Allow interaction with algms - must
Applications of Algorithm Visualization

1. Research: helps researchers to uncover unknown

features of algms
2. Education: helps students to learn algms with its
proofs effectively
UNIT - I
OVER