2/26/2020

Design and Analysis of Algorithm

Department of Computer Science &

Software Engineering
University of Swat

CS Course : Design and Analysis of Algorithm

Course Instructor : Muzammil Khan

Chapter 2

Complexity Analysis

1
 2/26/2020

Analysis of Algorithms
 Analyzing an algorithm has come to mean predicting the
resources that it requires

 Algorithm analysis use to determine

 Time efficiency (Complexity)
 Performance in terms of running times for different input
sizes
 Space utilization
 Requirement of storage to run the algorithm
 Correctness (accuracy) of algorithm
 Results are trustworthy, and algorithm is robust
Design and Analysis of Algorithms

Algorithm Efficiency
 Time efficiency remains an important consideration when
developing algorithms
 Algorithms designed to solve the same problem may differ
dramatically in efficiency
 These differences can be much more significant than
differences due to hardware and software
 Example
 Sequential search vs. Binary search
 Next slide …

Design and Analysis of Algorithms

2
 2/26/2020

Algorithm Efficiency (Cont...)

 The number of comparisons done by
 Sequential search and Binary search
 When x (value being searched) is larger than all array items

Array Size Number of comparisons Number of comparisons

- Sequential search - Binary search
128 128 8
1,024 1,024 11
1,048,576 1,048,576 21
4,294,967,296 4,294,967,296 33

Design and Analysis of Algorithms

Algorithm Efficiency (Cont...)

 Time taken by comparisons
 In terms of algorithm execution time
Execution time of Execution time of
Algorithm 1 Algorithm 2
41 ns 1048 µs
61 ns 1s
81 ns 18 min
101 ns 13 days
121 ns 36 years
161 ns 3.8 * 107 years
201 ns 4 * 1013 years
Design and Analysis of Algorithms

3
 2/26/2020

Analysis Approaches
 Basically three approaches can be adopted to analyze
algorithm running time in terms of input size
 Empirical Approach
 Running time measured experimentally

 Analytical Approach
 Running time estimated using mathematical modeling

 Visualization
 Performance is studied through animation for different data
sets

Design and Analysis of Algorithms

Empirical Approach
 The running time of algorithm is measured for different
data sizes and time estimates are plotted against the input
 The graph shows the trend

Design and Analysis of Algorithms

4
 2/26/2020

Empirical Approach (Cont...)

 Limitations of Running time

 Running time critically depends on

 Hardware resources used
 CPU speed, IO throughput, RAM size

 Software environment
 Compiler, Programming Language

 Program design approach

 Structured programming, Object Oriented programming

Design and Analysis of Algorithms

Analytical Approach
 Is a measure that is independent of
 The computer, programming language, and complex details
of the algorithm
 Usually this measure is in terms of
 How many times a basic operation is carried out for each
value of the input size
 Strategy
 Running time is estimated by analyzing the primitive
operations which make significant contributions to the
overall algorithm time
 These may broadly include
 Comparing data items
Design and Analysis of Algorithms

5
 2/26/2020

Analytical Approach (Cont...)

 Computing a value
 Moving a data item
 Calling a procedure

 Algorithm Specifications

 Algorithm is written in

 Plain text or natural language

 High level description

 Pseudo Code
 Low level to facilitate analysis and implementation
Design and Analysis of Algorithms

Example
 Specification Using Natural Language

 Preorder Tree Traversal Algorithm

Step 1. Push tree root to stack
Step 2. Pop the stack. If stack is empty exit, else process the node
Step 3. Travel down the tree following the left most path, and
pushing each right child onto the stack
Step 4. When leaf node is reached, go back to step 2.

Design and Analysis of Algorithms

6
 2/26/2020

Pseudo Code Convention

Design and Analysis of Algorithms

Example

Design and Analysis of Algorithms

7
 2/26/2020

Algorithm Design
 There are many approaches to design an algorithms
 Divide-and-Conquer
 Greedy
 Dynamic Programming
 Brute Force
 Approximation

 Each has certain

 Advantages &
 Limitations

Design and Analysis of Algorithms

Complexity Analysis
 Traditionally
 The running time of a program described as a function of the
size of the input
 For example,
 In a sorting problem, the input size is the number of items
n in array
 Input size can also contain more than one parameter e.g. for a
graph
 The running time of an algorithm on a particular input is the
number of primitive operations or steps executed
 The notion (conception) of step should be taken in a machine
independent way
Design and Analysis of Algorithm

8
 2/26/2020

Example
 As an example,
 Take the pseudo code for an algorithm that finds the
maximum element in an array of size n

 Problem
 Develop an algorithm to find maximum element in an array
of size n
 Analyze the algorithm for
 Time efficiency
 Space efficiency
 Correctness
Design and Analysis of Algorithm

Algorithm in Plain Text

 Steps
 Step #1
 First element of the array Store in variable max
 Step #2
 Scan array by comparing max with other elements
 Step #3
 Replace max with a larger element
 If the max is not last value, then repeat step 2 & 3
 Step #4
 Return value held by max
Design and Analysis of Algorithm

9
 2/26/2020

Pseudocode
 The function FIND-MAX finds the maximum element in an
array.
 The array A, of size n, is passed as argument to the function

Design and Analysis of Algorithm

Primitive Operations

Design and Analysis of Algorithm

10
 2/26/2020

Primitive Operations (Cont...)

Design and Analysis of Algorithm

Time Complexity
 Time complexity
 Three type of time complexity
 Best running time (best case running time)
 Average running time
 Worst running time

 Running time of the algorithm for

 Finding maximum value in the array of length n is

 Hence k is the number of times the statement

executed
Design and Analysis of Algorithm

11
 2/26/2020

Best Case Running Time

 Hence k is the number of times the statement

executed

 Best Case
 Best case occurs when the statement is not executed at all
 This happen when maximum value is at first position
 In this case k = 0
 Best (minimum) running time will be

Design and Analysis of Algorithm

Average Case Running Time

 Hence k is the number of times the statement

executed
 Average Case
 Average case occurs when the statement is executed on
average n / 2 times
 This happen when maximum value lies in the middle of the
array
 In this case k = n / 2

 The better average case time can be analysis by probabilistic

analysis
Design and Analysis of Algorithm

12
 2/26/2020

Worst Case Running Time

 Hence k is the number of times the statement

executed

 Worst Case
 In this case the statement is executed n times
 This happen when maximum value lies at the last position
of the array
 In this case k = n
 The worst case (maximum) time will be

Design and Analysis of Algorithm

Comparison of Running Times (rough figure)

Design and Analysis of Algorithm

13
 2/26/2020

Space Efficiency
 Space analysis of algorithm that find maximum element in
the array is simple and straight
 Determining the space as a function of Array Size
 Space required by the Program Instructions
 The amount of storage requirement of the array
 Depends on nature of the data
 Integers, Floating point, String etc…
 Space increases in direct proportion to array size
 So

Design and Analysis of Algorithm

Correctness - Loop Invariant Technique

 Several standard algorithms are based on
 One or More iterative computations using loop structures
 The correctness of such algorithm is established by proving
the correctness of loops
 Loop Invariant Techniques is use to ensure loop correctness

 Loop Invariant
 Is set of conditions or relationship that is either true or false
during loop execution
 Depends on nature of the problem being analyze

Design and Analysis of Algorithm

14
 2/26/2020

Correctness (Cont...)
 Loop Invariant Method
 The algorithm/loop is correct if we establish the following
 Initialization
 Loop invariant is true prior to execution of first iteration
 Maintenance
 Loop invariant is true prior to some iteration, will remain
true before next iteration, till last iteration
 Termination
 After termination of the loop, the post condition can be
evaluated as true

Design and Analysis of Algorithm

Correctness (Cont...)
 The correctness of FIND-MAX algorithm, will establish by
loop invariant techniques

 Define loop invariant S

 S = After kth iteration, the variable MAX holds the largest
value of the first k element of the array
 Now
 Consider the loop invariant method
Design and Analysis of Algorithm

15
 2/26/2020

Correctness (Cont...)
 Initialization
 Condition requires that the prior the first iteration the statement
S should be true
 So; this is trivially (vacuously) true
 Because
 The max contain 1st element of the array &
 Loop start with index 2
 Maintenance
 Condition requires that if S is true before an iteration of loop, it
should remain true before the next iteration
 Can be verified as if max holds the largest value of k element,
then it holds the largest of k+1 element, and so on
Design and Analysis of Algorithm

Correctness (Cont...)
 Termination
 Condition requires that the post condition should be true
 max should return the maximum value of the array
 At the end of the array
 The loop terminates when j exceeds n

Design and Analysis of Algorithm

16
 2/26/2020

Complexity Analysis
 Problem
 Adding array element
 Algorithm
ARRAY-ADD (A, n) Cost Times
1. result  0 c1 1
2. for ( i 1; i < = n; i++) c2 n+1
3. result  result + A[i] c3 n

 To compute T(n) the running time of the algorithm,

 Sum up, product of cost and time column
 T(n) = c1+c2(n+1)+c3(n) = A + Bn
 Where a and b are constants that depend on ci )
Design and Analysis of Algorithm

Complexity Analysis (Cont...)

 In the previous example
 There are always n passes through the for loop
 No matter what the value of the numbers/elements in the
array

 This is known as the Every-case running time

 Means
 Best = Average = Worst case running time

 Other examples include ….

 Consider the example of search in an array
Design and Analysis of Algorithm

17
 2/26/2020

Complexity Analysis (Cont...)

 Problem
 Search Array Members
 Algorithm
ARRAY-SEARCH (A, n, key) cost Times
1. For ( i 1; i <= n; i++) c1 ?
2. if A[i]=key c2 ?
3. return i c3 ?
4. return i c4 ?
 Computing Running Time
 Worst Case Analysis
 The loop will execute maximum number of time i.e. n + 1
 So T(n) = A + Bn or W(n) = A + Bn or T(n)w = A + Bn
Design and Analysis of Algorithm

Complexity Analysis (Cont...)

 Average Case Analysis
 It is more difficult to analyze the average case than the worst
case
 To compute average case complexity
 We need to assign probabilities
 In case of linear search
 Equal probabilities are assigned to all array slots i.e. the key
is equally likely in any array slot
 Assuming that key has equal probability 1/n of being in
any position &
 Unit cost is c
 So 1c.1/n + 2c.1/n + 3c.1/n +….+ nc.1/n
= c/n (1+2+….+n) = c/n (n(n+1)/2)
= cn (n+1)/2n
Design and Analysis of
= c(n+1)/2 Algorithm

18
 2/26/2020

Complexity Analysis (Cont...)

 So, average case analysis
T(n)=c(n+1)/2 or A(n)=c(n+1)/2

 Best Case Analysis

 When value is found at first location
 So
T(n) = 1 or B(n) = 1

 Worst-case and average-case analysis are done much more

often than best-case analysis

Design and Analysis of Algorithm

End
 End of chapter

 You may have quiz next weak

Design and Analysis of Algorithm

19