DATA STRUCUTERE

INTRODUCTION TO DATA
STRUCTURE

Lecture 6
Analysis of Algorithm
Lecture 6
 What is Algorithm
 An Algorithm is a sequence of steps to solve a
problem. Design and Analysis of Algorithm is very
important for designing algorithm to solve different
types of problems in the branch of computer
science and information technology.
 What is Algorithm?
4

 Algorithm
 is any well-defined computational procedure that takes
some value, or set of values, as input and produces
some value, or set of values, as output.
 is thus a sequence of computational steps that
transform the input into the output.
 is a tool for solving a well - specified computational
problem.
 Any special method of solving a certain kind of
problem .
 Cont..
 What is an algorithm?
An algorithm is a finite set of precise

instructions for performing a computation or for

solving a problem.
 Algorithms and their analysis
 An algorithm is a step-by-step procedure
for solving a problem in a finite amount
of time.

Input Algorithm Output

Algorithm Descriptions
 Nature languages: Arabic,
English, etc.
 Flowcharts.
 Pseudo-code: codes very close
to computer languages, e.g., C
programming language.
 Programs: C programs, C++
programs, Java programs.
 Algorithms
 Properties of algorithms:

• Input from a specified set,

• Output from a specified set (solution),
• Definiteness of every step in the computation,
• Correctness of output for every possible input,
• Finiteness of the number of calculation steps,
• Effectiveness of each calculation step and
• Generality for a class of problems.
Performing a Task on the
Computer
 Determine Output
 Identify Input
 Determine process necessary to turn given Input
into desired Output

Chapter 2 9
 Example
How fast is a car traveling if it goes 50 miles in 2 hours?

1. Output:
a number giving the speed in miles
per hour
2. Input:
the distance and time the car has
traveled

3. Process:
speed = distance / time
Program Planning

 Always have a plan before trying to write

a program
 The more complicated the problem, the
more complex the plan must be
 Planning and testing before coding saves
time coding

Chapter 2 11
Program Planning Example
- A recipe

 Ingredients and amounts are determined by

what you want to bake
 Ingredients are input
 The way you combine them is the
processing
 What is baked is the output

Chapter 2 12
 What is a program?
13

 A program is the expression of an algorithm in a

programming language
 a set of instructions which the computer will follow
to solve a problem
 Introduction
14

 Why need algorithm analysis ?

 writing a working program is not good enough
 The program may be inefficient!
 If the program is run on a large data set, then the
running time becomes an issue
 Algorithm Design
 The important aspects of algorithm design include creating
an efficient algorithm to solve a problem in an efficient
way using minimum time and space.
 To solve a problem, different approaches can be followed.
Some of them can be efficient with respect to time
consumption, whereas other approaches may be memory
efficient. However, one has to keep in mind that both time
consumption and memory usage cannot be optimized
simultaneously. If we require an algorithm to run in lesser
time, we have to invest in more memory and if we require
an algorithm to run with lesser memory, we need to have
more
Problem Development Steps
 computational problems.
 Problem definition
 Development of a model
 Specification of an Algorithm
 Designing an Algorithm
 Checking the correctness of an Algorithm
 Analysis of an Algorithm
 Implementation of an Algorithm
 Program testing
 Documentation
Other Algorithmic Characteristics
 “Algorithms Should Be General” and be applicable
to several cases
 “Algorithms Should Use Resources Efficiently”:
 Fast Speed
 Minimal RAM or Disk Space
 Algorithms Should Be Understandable” so that the
operations are clear to readers
Other Algorithmic Characteristics
 “Algorithms Should Be Clear and Precise” despite
the language used
 Natural languages are ambiguous in that words or
phrases may have multiple meanings
 Programming languages are formal languages with
clear, unambiguous rules
Characteristics of Algorithms
 Algorithms must have a unique name
 Algorithms should have explicitly defined set of
inputs and outputs
 Algorithms are well-ordered with unambiguous
operations
 Algorithms halt in a finite amount of time.
Algorithms should not run for infinity, i.e., an
algorithm must end at some point
Pseudocode
 Pseudocode gives a high-level description of an
algorithm without the ambiguity associated with
plain text but also without the need to know the
syntax of a particular programming language. The
running time can be estimated in a more general
manner by using Pseudocode to represent the
algorithm as a set of fundamental operations which
can then be counted.
Algorithm Analysis
 Space complexity
 How much space is required

 Time complexity
 How much time does it take to run the
algorithm
Space Complexity
 Space complexity = The amount of memory required
by an algorithm to run to completion
 [The most often encountered cause is “memory leaks” – the
amount of memory required larger than the memory
available on a given system]
 Some algorithms may be more efficient if data
completely loaded into memory
 Need to look also at system limitations
 E.g. Classify 2GB of text in various categories [politics,
tourism, sport, natural disasters, etc.] – can I afford to load
the entire collection?
Space Complexity (cont’d)
1. Fixed part: The size required to store certain
data/variables, that is independent of the size of the
problem:
- e.g. name of the data collection
- same size for classifying 2GB or 1MB of texts

2. Variable part: Space needed by variables, whose size

is dependent on the size of the problem:
- e.g. actual text
- load 2GB of text VS. load 1MB of text
Time Complexity
 Often more important than space complexity
 space available (for computer programs!) tends to be larger
and larger
 time is still a problem for all of us

 3-4GHz processors on the market

 still …
 researchers estimate that the computation of various
transformations for 1 single DNA chain for one single
protein on 1 TerraHZ computer would take about 1 year to
run to completion
 Algorithms running time is an important issue
 Analysis of Algorithms
25

 Estimate the running time

 Estimate the memory space required.

Depends on the input size.

Running Time
 The running time of an algorithm typically
grows with the input size.
 Big O Notation
 What is the Big O Notation? Big O is the way we
describe the performance efficiency or complexity of
algorithms.
1. O(1): Executes in the same time regardless of the
size of the input
2. O(n): Executes linearly and proportionally to the
size of the input
3. O(n²): Performance is directly proportional to the
square of the size of the input (ex: nested iterations,
loops)
Rate of Growth =Asymptotic Analysis

 Using rate of growth as a measure to compare

different functions implies comparing them
asymptotically.
 If f(x) is faster growing than g(x), then f(x) always
eventually becomes larger than g(x) in the limit
(for large enough values of x).
 Example

 Suppose you are designing a web site to process

user data (e.g., financial records).
 Suppose program A takes fA(n)=30n+8
microseconds to process any n records, while
program B takes fB(n)=n2+1 microseconds to
process the n records.
 Which program would you choose, knowing
you’ll want to support millions of users?
Visualizing Orders of Growth
 On a graph, as
you go to the
right, a faster
fA(n)=30n+8
growing

Value of function 
function
eventually
becomes fB(n)=n2+1

larger...
Increasing n 
Big-O Notation

 We say fA(n)=30n+8 is order n, or O(n).

It is, at most, roughly proportional to n.
 fB(n)=n2+1 is order n2, or O(n2). It is, at most,
roughly proportional to n2.
 In general, an O(n2) algorithm will be slower
than O(n) algorithm.
 Warning: an O(n2) function will grow faster
than an O(n) function.
 More Examples …

 We say that n4 + 100n2 + 10n + 50 is of the order

of n4 or O(n4)
 We say that 10n3 + 2n2 is O(n3)
 We say that n3 - n2 is O(n3)
 We say that 10 is O(1),
 We say that 1273 is O(1)
Big-O Visualization
Computing running time (cont.)

Cost
sum = 0; c1
for(i=0; i<N; i++) c2
for(j=0; j<N; j++) c2
sum += arr[i][j]; c3
------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N x N

O(n2)
 Examples

i = 0;
while (i<N) {
X=X+Y; // O(1)
result = mystery(X); // O(N), just an example...
i++; // O(1)
}
 The body of the while loop: O(N)

 Loop is executed: N times

N x O(N) = O(N2)
 Examples (cont..’d)
if (i<j)
for ( i=0; i<N; i++ ) O(N)
X = X+i;
else
X=0; O(1)

Max ( O(N), O(1) ) = O (N)

 Constant time complexity
 A(a,b) Time
Cost
 {
 Temp=a; 1
C1 1
 A=b; C2 1
C3
 B=temp; 3 constamt order of 1
O(1)
 }
 Constant time
 Constant time means, there is some constant k such
that any operation always take k unit. Regardless of
input size.
 A programming statement takes constant time if:
o It does not include loop.
o It does not include calling a method whose time is
unknown or not content.
o Assignment operation, arithmetic operation, if-else
constriction switch case they take constant time
 Analysis loops
 For(i=1;i<=n;i++) Cost
time
 { N+1
 Cout<<I; C1
C2
 } N

cn
N+1+n
Order of n O(n)
 Linear time complexity
 Informally this means that the time taken by
algorithm increases at most linear with the size of
the input.
 In other words larger the input size greater the time
taken.
 1<n<n2<n3<--------

Task is to design algorithm which takes lesser time

 Constant or linear time! Which is better?

 Suppose we have two algorithm to solve a task.:

 Algorithm a takes 2000 time unit(constant)
 Algorithm B takes 100*n time unit (linear).

 Which is better?
 Clearly algorithm B is better if our problem size small,
that is , if n<20.
 Algorithm A is better for large problems, with n>20
 So b is better for small problems.
 But A is better for large problems.
 We usually care the most about large problems
 Analysis loops
 For(i=1;i<=n;i++)
 {
 Cout<<I;
 }
 For(j=1;j<=n;j++)
 {
 Cout<<I;
 }
 Eg n+n---n 3n orfer of n O(n)
 Loop analysis
 For(i=1;i<=n;i++) Cost time

 {
 For(j=1;j<=n;j++) C1 N+1=n
C2 N+1=n
 {
N
 Cout<<i<<j; Cn (n)*(n)+n
=n^2
 } Order of square O(n^2)
 }
Loop analysis
 For(i=1;i<=n;i++) Cost time
 {
 For(j=1;j<=n;j++)
C1 N+1=n
 { C2 N+1=n
 For(j=1;j<=n;j++)
N
 { Cn (n)*(n) )*(n) +n
=n^3
Order of square O(n^2)
 Cout<<i<<j;
 }
 }
 }
 Worst Case , Average Case and best case
45

Worst case complexity: provides absolute

guarantees for time a program will run. The worst
case complexity as a function of n is longest
possible time for any input of size n.
Average case complexity: suitable if small function
is repeated often or okay to take a long time –very
rarely. The average case as a function of n is the
avg. complexity over all possible inputs of that
length.
Avg. case complexity analysis usually requires
probability theory.
L8
Worst Case , Average Case and best case

 The best case is when the number of steps is less as

possible. If the target value is found in a sequential
search array of the first position (i.e., we need to
compare the target value with only one element
from the array)—we have found the element by
executing only one iteration (or by least possible
statements)
END
?