Data Structures and Algorithm Analysis
12
Dr. Syed Asim Jalal
Department of Computer Science
University of Peshawar
In this lecture ….
• Asymptotic Performance
– How does the algorithm behave as the problem size
gets very large…. ?
• Asymptotic Notations
–O
–Θ
–Ω
–o
–ω
2
What is asymptotic analysis ?
• Asymptotic analysis deals with analyzing the
properties of the running time when the input
size goes to infinity (this means a very large
input size)
• The differences between orders of growths are
more significant for larger input size. Analyzing the
running times on small inputs does not allow us to
distinguish between efficient and inefficient
algorithms
– The objective of asymptotic analysis is to describe the
behavior of a function T(N) as it goes to infinity.
– Asymptotic notations are used to describe the
asymptotic analysis
3
Function Bounds..
• Lets understand with the help of example. Suppose we
have a function 10N2
• Can we say it is bounded by 11N2 and 9N2 for all N
≥ 1?
– i.e 10N2 cannot go above 11N2 and doesn’t come down
below 9N2 for all values of N. 10N2 is sandwiched
between 9N2 and 11N2
– Now if f(n) is 10N2 and g(n) is N2
– Then we say that f(n) is Θ (g(n))
• (explanation later..)
4
Asymptotic Notations
big-Theta Θ(g(n))
Big-Oh O(g(n))
big-Omega Ω(g(n))
little-oh o(g(n))
little-omega ω(g(n))
5
Big-Theta Θ
• A function f(n) is Θ (g(n)) if there exist a positive
constants c1, c2, and n0 such that
0 ≤ c1 g(n) ≤ f(n) ≤ c2 g(n), for all n ≥ n0
– We define Θ(g(n)) to be a set of functions that are
asymptotically equivalent to g(n)
– A function f(n) belongs to the set Θ(g(n)), if there
exist positive constants c1 and c2, such that g(n) can
be "sandwiched" between c1g(n) and c2g(n), for
sufficiently large n.
6
• Representation:
– “f(n) = Θ(g(n))” or
– “f(n) ∈ Θ(g(n))”
• We say this as:
– f(n) and g(n) are asymptotically equivalent.
Or
– g(n) is asymptotically tight bound for f(n)
7
f(n) = Θ(g(n))
0 ≤ c1g(n) ≤ f(n) ≤ c2g(n)
8
• The following equations are asymptotically equivalent
• 5n2
• 2n2 - 5n + 10
• (8n2 + 2n – 3)
• (n2/5 + √n – 10 log n)
• n(n – 3)
As ‘n’ becomes large, the dominant term is some constant
times n2
9
Lower and upper bounds (Example1)
• f(n) = 8n2 + 2n – 3
– To show that f(n) ∈ Θ (n2)
– We need to find the following three values.
– c1, c2 and no
• To find Lower bound we need c1 and no
• To find Upper bound we need c2 and no
– We will have two no, select the maximum no
10
Finding c1 and no (Example1)
Lower bound: 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n)
f(n) = 8n2 + 2n – 3, f(n) ∈ Θ (n2)
• c1n2 ≤ 8n2 + 2n – 3 ?? c1 can be anything lesser
than the constant with n2 of
• 7n2 ≤ 8n2 + 2n – 3 the expression
• c1= 7
• No = 1 no =1
7(1) 2 ≤ 8(1) 2 +2(1)-3
7 ≤ 8+2-3
7≤7
11
Finding c2 and no (Example1)
Upper Bound: 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n)
f(n) = 8n2 + 2n – 3, f(n) ∈ Θ (n2)
8n2 + 2n – 3 ≤ c2n2
8n2 + 2n – 3 ≤ 9n2
c2 can be anything greater
=>8n2 + 2n – 3 ≤ 9n2 than the constant with n2 of
the expression
c2 = 9
No = 1
12
f(n) = Θ(g(n))
f(n) = 8n2 + 2n – 3
g(n) = n2,
c1= 7, c2= 10, n0= 1
1e + 11
8e + 10
6e + 10
f(n)
4e + 10
2e + 10
0
0 20000 40000 60000 80000 100000
n
13
OR
C1 = 1 / 4
n0 = 13
14
15
Example 2
f(n) = 2n2 - 5n +10
f(n) ∈ Θ (n2)??
• 1n2 ≤ 2n2 - 5n +10 … for lower bound
– C1= 1, No = 1
• 2n2 - 5n +10 ≤ 2 n2 … for upper bound
– C1= 2, No = 2
16
Practice
f(n) = (3n2 / 2 ) + (5n/2) – 3
f(n) ∈ Θ (n2)??
– C 1= 1
– C2= 2,
– No = ???
17
Example 3
• f(n) = 3n+3
• g(n) = n
– f(n) ∈ Θ (n)
C1=2, No = 1
C2=4, No = 3
No = 3
18
Suppose …
• f(n) = 2n3 - 5n2 +10n+1
• g(n) = n2
• Is f(n) ∈ Θ (g(n)) ???
• No.
19
Lower bound: 5n2 ≤ 6n3
Upper Bound: 6n3 ≤ ?n2 … always false
20
O-notation (Big-O)
• Sometimes we are only interested in proving
one bound or the other
• We use O-notation, when we have only an
asymptotic upper bound
• O(g(n)) = {f(n) | there exist positive constants ‘c’
and ‘n0’ such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0}
• We write it as
21
22
• If any quadratic function is in
is also a
• Example:
– f(n) = 2n2
– g(n) = n2
–f(n) = O (g(n))
for c = 5/2, n0 = 1
23
Practice examples
24
• 2n2 = O(n3):
2n2 ≤ cn3 ⇒ c = 1 and n0= 2
• n2 = O(n2)
n2 ≤ cn2 ⇒ c = 1 and n0= 1
• 1000n2+1000n = O(n2)
1000n2+1000n ≤ cn2 ⇒ c=1001 and n0 = 1000
25
Ω - notation
• Just as O-notation provides an asymptotic upper bound on a
function, omega notation provides an asymptotic lower bound.
– Ω(g(n)) = the set of functions with a larger or same order of
growth as g(n)
26
• 5n2 = Ω(n)
– 0 ≤ cn ≤ 5n2, c = 1 and n0 = 1
• 100n + 5 ≠ Ω(n2)
– 0 ≤ cn2 ≤ 100n + 5
• n = Ω(n)
• n3 = Ω(n2)
27
28
Theorem
•
if and only if
29
