Unit 1 - Analysis and Design of Algorithm - WWW - Rgpvnotes.in
Unit 1 - Analysis and Design of Algorithm - WWW - Rgpvnotes.in
Tech
Subject Name: Analysis and Design of Algorithm
Subject Code: IT-403
Semester: 4th
Downloaded from be.rgpvnotes.in
Algorithm
The o d Algorithm o es f o the Pe sia autho A dullah Jafa Muha ad i Musa Al-
An Algorithm is a set of rules for carrying out calculation either by hand or on a machine.
khowarizmi in ninth century, who has given the definition of algorithm as follows:
An Algorithm is a well defined computational procedure that takes input and produces
4. Finiteness. If we trace out the instruction of an algorithm, then for all cases, the algorithm
terminates after a finite number of steps.
5. Effectiveness. Every instruction must be very basic so that it can be carried out, in principal, by a
person using only pencil and paper. It is not enough that each operation be definite as in criterion
3; it also must be feasible.
Designing Algorithms
Decide on the appropriate data structure:Some algorithms do not demand any ingenuity in
representing their inputs.Someothers are in fact are predicted on ingenious data
structures.A data type is a well-defined collection of data with a well-defined set of
operations on it.A data structure is an actual implementation of a particular abstract data
type. The Elementary
Algorithm design techniques: Creating an algorithm is an art which may never be fully
automated. By mastering these design strategies, it will become easier for you to devise
new anduseful algorithms. Divide & Conquer, Dynamic programming , Greedy strategy ,
such technique.
Methods of specifying an algorithm: There are mainly two options for specifying an
algorithm: use of natural language or pseudocode & Flowcharts.A Pseudo code is a mixture
of natural language & programming language like constructs. A flowchart is a method of
expressing an algorithm by a collection of connected geometric shapes.
Analysis of algorithms:
There are any criteria upon which we can judge an algorithm for instance:
1. Does it do what we want to do?
2. Does it work correctly according to the original specifications to the task?
3. is there documentation that describes how to use it and how it works?
4. Are procedures created in such a way that they perform logical sub functions?
5. is the code readable?
The space complexity of an algorithm is the amount of memory it needs to run to completion.
The time complexity of an algorithm is the amount of computer time it needs to run to
completion.
Algorithm Complexity
The complexity of an algorithm f(n) gives the running time and / or storage space required by the
algorithm in terms of n as the size of input data.
Suppose X is an algorithm and n is the size of input data, the time and space used by the Algorithm
Space complexity
Space complexity of an algorithm represents the amount of memory space required by the
algorithm in its life cycle. Space required by an algorithm is equal to the sum of the following two
o po e ts −
A fixed part that is a space required to store certain data and variables, that are
independent of the size of the problem. For example simple variables & constant used,
A variable part is a space required by variables, whose size depends on the size of the
program size etc.
problem. For example dynamic memory allocation, recursion stack space etc.
Space complexity S(P) of any algorithm P is S(P) = C + SP(I) Where C is the fixed part and S(I) is the
variable part of the algorithm which depends on instance characteristic I.
Time complexity
Time Complexity of an algorithm represents the amount of time required by the algorithm to run
to completion. Time requirements can be defined as a numerical function T(n), where T(n) can be
measured as the number of steps, provided each step consumes constant time.
For example, addition of two n-bit integers takes n steps. Consequently, the total computational
time is T(n) = c*n, where c is the time taken for addition of two bits. Here, we observe that T(n)
grows linearly as input size increases.The time T (P) taken by a program P is sum of compile time
and run time. The compile time does not depend on the instance characteristics. Also, we may
assume that a compiled program will be run several time of a program. This run time is denoted by
tp. Because of many of the factor tp depends on are not known at the time of a program is
conceived, it is reasonable to attempt only to estimate tp . T(p)=c+tp
Best-case efficiency: Efficiency (number of times the basic operation will be executed) for the best
case input of size n. i.e. The algorithm runs the fastest among all possible inputs of size n.
Average-case efficiency: Average time taken (number of times the basic operation will be
executed) to solve all the possible instances (random) of the input. NOTE: NOT the average of
worst and best case
Algorithm Analysis
Efficiency of an algorithm can be analyzed at two different stages, before implementation and after
i ple e tatio , as e tio ed elo −
A priori analysis − This is theo eti al a al sis of a algo ith . Effi ie of algorithm is
measured by assuming that all other factors e.g. processor speed, are constant and have no
A posterior analysis − This is e pi i al a al sis of a algo ith . The sele ted algo ith is
effect on implementation.
We learn a priori algorithm analysis. Algorithm analysis deals with the execution or running time of
various operations involved. Running time of an operation can be defined as no. of computer
instructions executed per operation.
Asymptotic Notations
Asymptotic analysis of an algorithm, refers to defining the mathematical boundation/framing of its
run-time performance. Using asymptotic analysis, we can very well conclude the best case, average
case and worst case scenario of an algorithm.
Asymptotic analysis are input bound i.e., if there's no input to the algorithm it is concluded to work
in a constant time. Other than the "input" all other factors are considered constant.
Asymptotic analysis refers to computing the running time of any operation in mathematical units
of computation. For example, running time of one operation is computed as f(n) and may be for
another operation it is computed as g(n2). Which means first operation running time will increase
linearly with the increase in n and running time of second operation will increase exponentially
when n increases. Similarly the running time of both operations will be nearly same if n is
significantly small.
Usuall , ti e e ui ed a algo ith falls u de th ee t pes −
Best Case − Mi i u ti e e ui ed fo p og a e e utio .
Average Case − A e age ti e e ui ed fo p og a e e utio .
Worst Case − Ma i u time required for program execution.
Following are commonly used asymptotic notations used in calculating running time complexity of
Ο Notatio
an algorithm.
Ω Notatio
θ Notatio
Big Oh Notatio , Ο
The Ο is the fo al a to e p ess the uppe ou d of an algorithm's running time. It measures
the worst case time complexity or longest amount of time an algorithm can possibly take to
complete.
Definition:
A function t(n) is said to be in O(g(n)), denoted t(n) ∈ O(g(n)), if t(n) is bounded above
by some constant multiple of g(n) for all large n, i.e., if there exist some positive constant
c and some nonnegative integer n0 such that
t g fo all
O ega Notatio , Ω
The Ω is the fo al a to e p ess the lo e ou d of a algo ith 's u i g ti e. It easu es
the best case time complexity or best amount of time an algorithm can possibly take to complete.
Definition:
A functio t is said to e i Ω g , de oted t ∈ Ω g , if t is ou ded
below by some constant multiple of g (n) for all large n, i.e., if there exist some positive
constant c and some nonnegative integer n0 such that
t g fo all 0
Theta Notatio , θ
The θ is the fo al a to e p ess oth the lo e ou d a d uppe ou d of a algo ith 's
running time. Definition:
θ f(n)) = { g(n) if and only if g = Ο f(n)) and g = Ω f(n)) for all n > n0. }
Efficiency Class
Fast 1 constant High time efficiency
log n logarithmic
n linear
n log n n log n
n2 quadratic
n3 cubic
2n exponential
slow n! Factorial low time efficiency
constant − Ο
logarithmic − Ο log
linear − Ο
n log n − Ο log
quadratic − Ο 2)
cubic − Ο 3)
polynomial − nΟ
exponential − 2Ο
Recurrence relation:
Recurrence relations often arise in calculating the time and space complexity of algorithms. Any
problem can be solved either by writing recursive algorithm or by writing non-recursive algorithm.
A recursive algorithm is one which makes a recursive call to itself with smaller inputs. We often use
a recurrence relation to describe the running time of a recursive algorithm. A recurrence relation is
an equation or inequality that describes a function in terms of its value on smaller inputs or as a
function of preceding (or lower) terms.
1) Substitution Method: We make a guess for the solution and then we use mathematical
induction to prove the the guess is correct or incorrect.
We need to prove that T(n) <= cnLogn. We can assume that it is true
for values smaller than n.
T(n) = 2T(n/2) + n
<= cn/2Log(n/2) + n
= cnLogn - cnLog2 + n
= cnLogn - cn + n
<= cnLogn
2) Recurrence Tree Method: In this method, we draw a recurrence tree and calculate the time
taken by every level of tree. Finally, we sum the work done at all levels. To draw the recurrence
tree, we start from the given recurrence and keep drawing till we find a pattern among levels. The
pattern is typically a arithmetic or geometric series.
cn2
/ \
T(n/4) T(n/2)
cn2
/ \
c(n2)/16 c(n2)/4
/ \ / \
T(n/16) T(n/8) T(n/8) T(n/4)
Breaking down further gives us following
cn2
/ \
c(n2)/16 c(n2)/4
/ \ / \
c(n )/256 c(n )/64 c(n2)/64 c(n2)/16
2 2
/ \ / \ / \ / \
3) Master Method:
Master Method is a direct way to get the solution. The master method works only for following
type of recurrences or for recurrences that can be transformed to following type.
In recurrence tree method, we calculate total work done. If the work done at leaves is polynomially
more, then leaves are the dominant part, and our result becomes the work done at leaves (Case 1).
If work done at leaves and root is asymptotically same, then our result becomes height multiplied
by work done at any level (Case 2). If work done at root is asymptotically more, then our result
becomes work done at root (Case 3).
Examples of some standard algorithms whose time complexity can be evaluated using Master
Method
Me ge “o t: T = T / + Θ . It falls i ase as c is 1 and Log ba] is also 1. So the solution is
Θ Log
Notes:
1) It is not necessary that a recurrence of the form T(n) = aT(n/b) + f(n) can be solved using Master
Theorem. The given three cases have some gaps between them. For example, the recurrence T(n)
= 2T(n/2) + n/Logn cannot be solved using master method.
In divide and conquer approach, the problem in hand, is divided into smaller sub-problems and
then each problem is solved independently. When we keep on dividing the sub-problems into even
smaller sub-problems, we may eventually reach at a stage where no more dividation is possible.
Those "atomic" smallest possible sub-problem (fractions) are solved. The solution of all sub-
problems is finally merged in order to obtain the solution of original problem.
This step involves breaking the problem into smaller sub-problems. Sub-problems should
Divide/Break
represent as a part of original problem. This step generally takes recursive approach to
divide the problem until no sub-problem is further dividable. At this stage, sub-problems
become atomic in nature but still represents some part of actual problem.
This step receives lot of smaller sub-problem to be solved. Generally at this level, problems
Conquer/Solve
When the smaller sub-problems are solved, this stage recursively combines them until they
Merge/Combine
Some Examples
The following computer algorithms are based on divide-and-conquer p og a i g app oa h −
Binary Search
Merge Sort
Quick Sort
Strassen's Matrix Multiplication
Binary search
Binary Search is applied on the sorted array or list. In binary search, we first compare the key value
with the element in the middle position of the array. If the value is matched, then we return the
position. If the value is less than the middle element, then it must lie in the lower half of the array
and if it's greater than the element then it must lie in the upper half of the array. We repeat this
procedure on the lower (or upper) half of the array. Binary Search is useful when there are large
The below given is our sorted array and assume that we need to search location of key value 31
using binary search.
First, we shall determine the mid of the array by using this formula −
Now we compare the value stored at location 4, with the value being searched i.e. 31. We find that
value at location 4 is 27, which is not a match. Because key value is greater than 27 and we have a
sorted array so target value must be in upper portion of the array.
We change our low to mid + 1 and find the new mid value again.
low = mid + 1
mid = (LB + UB) / 2 = 5+9/2 =7
Our new mid is 7 now. We compare the value stored at location 7 with our target value 31.
The value stored at location 7 is not a match, rather it is greater that what we are looking for. So
the value must be in lower part from this location.
We compare the value stored ad location 5 with our target value. We find that it is a match.
Description: Here A is a sorted array having N elements. ITEM is the value to be searched. BEG
denotes first element and END denotes last element in the array. MID denotes the middle value.
6.Else
11.Else
13.Exit
Merge sort
Merge sort is a sorting technique based on divide and conquer technique. With worst-case time
o ple it ei g Ο log .Me ge so t fi st di ides the a a i to e ual hal es a d the o i es
them in a sorted manner.
We know that merge sort first divides the whole array iteratively into equal halves unless the
atomic values are achieved. We see here that an array of 8 items is divided into two arrays of size
4.
This does not change the sequence of appearance of items in the original. Now we divide these
two arrays into halves.
We further divide these arrays and we achieve atomic value which can no more be divided.
Now, we combine them in exactly same manner they were broken down. Please note the color
codes given to these lists.
We first compare the element for each list and then combine them into another list in sorted
manner. We see that 14 and 33 are in sorted positions. We compare 27 and 10 and in the target list
of 2 values we put 10 first, followed by 27. We change the order 19 and 35. 42 and 44 are placed
sequentially.
In next iteration of combining phase, we compare lists of two data values, and merge them into a
list of foud data values placing all in sorted order.
Algorithm
Merge sort keeps on dividing the list into equal halves until it can no more be divided. By
definition, if it is only one element in the list, it is sorted. Then merge sort combines smaller sorted
Quick sort
Quick sort is a highly efficient sorting algorithm and is based on partitioning of array of data into
smaller arrays. A large array is partitioned into two arrays one of which holds values smaller than
specified value say pivot based on which the partition is made and another array holds values
greater than pivot value. The quick sort partitions an array and then calls itself recursively twice to
sort the resulting two subarray. This algorithm is quite efficient for large sized data sets as its
average and worst case complexity are of O(nlogn) where n are no. of items.
Quicksort is a divide and conquer algorithm. Quicksort first divides a large list into two smaller
sub-lists: the low elements and the high elements. Quicksort can then recursively sort the sub-lists.
QuickSort Algorithm
Using pivot algorithm recursively we end-up with smaller possible partitions. Each partition then
p o essed fo ui k so t. We defi e e u si e algo ith fo ui kso t as elo −
Step 1 − Make the ight-most index value pivot
Step 2 − pa titio the a a usi g pi ot alue
Step 3 − uicksort left partition recursively
Step 4 − ui kso t ight pa titio e u si el
Given two square matrices A and B of size n x n each, find their multiplication matrix.
Following is a simple way to multiply two matrices.
void multiply(int A[][N], int B[][N], int C[][N])
{
for (int i = 0; i < N; i++)
{
for (int j = 0; j < N; j++)
{
C[i][j] = 0;
for (int k = 0; k < N; k++)
{
C[i][j] += A[i][k]*B[k][j];
}
}
}
}
In the above method, we do 8 multiplications for matrices of size N/2 x N/2 and 4 additions. So the
time complexity can be written as T(N) = 8T(N/2) + O(N 2).Time complexity of above method is
O(N3).
“t asse ’s ethod:
In the above divide and conquer method, the main component for high time complexity is 8
recursive calls. The idea of Strasse ’s ethod is to reduce the number of recursive calls to 7.
“t asse ’s ethod is si ila to a o e si ple di ide a d o ue ethod i the se se that this
method also divide matrices to sub-matrices of size N/2 x N/2 as shown in the above diagram, but
i “t asse ’s ethod, the fou su -matrices of result are calculated using following formulae.