Data Structures and Algorithm Analysis

12
Dr. Syed Asim Jalal
Department of Computer Science
University of Peshawar
 In this lecture ….
• Asymptotic Performance
– How does the algorithm behave as the problem size
gets very large…. ?

• Asymptotic Notations
–O
–Θ
–Ω
–o
–ω
2
 What is asymptotic analysis ?
• Asymptotic analysis deals with analyzing the
properties of the running time when the input
size goes to infinity (this means a very large
input size)
• The differences between orders of growths are
more significant for larger input size. Analyzing the
running times on small inputs does not allow us to
distinguish between efficient and inefficient
algorithms
– The objective of asymptotic analysis is to describe the
behavior of a function T(N) as it goes to infinity.
– Asymptotic notations are used to describe the
asymptotic analysis
3
 Function Bounds..
• Lets understand with the help of example. Suppose we
have a function 10N2

• Can we say it is bounded by 11N2 and 9N2 for all N

≥ 1?

– i.e 10N2 cannot go above 11N2 and doesn’t come down

below 9N2 for all values of N. 10N2 is sandwiched
between 9N2 and 11N2
– Now if f(n) is 10N2 and g(n) is N2
– Then we say that f(n) is Θ (g(n))
• (explanation later..)

4
 Asymptotic Notations

big-Theta Θ(g(n))
Big-Oh O(g(n))
big-Omega Ω(g(n))
little-oh o(g(n))
little-omega ω(g(n))

5
 Big-Theta Θ
• A function f(n) is Θ (g(n)) if there exist a positive
constants c1, c2, and n0 such that

0 ≤ c1 g(n) ≤ f(n) ≤ c2 g(n), for all n ≥ n0

– We define Θ(g(n)) to be a set of functions that are

asymptotically equivalent to g(n)
– A function f(n) belongs to the set Θ(g(n)), if there
exist positive constants c1 and c2, such that g(n) can
be "sandwiched" between c1g(n) and c2g(n), for
sufficiently large n.
6
• Representation:
– “f(n) = Θ(g(n))” or

– “f(n) ∈ Θ(g(n))”

• We say this as:

– f(n) and g(n) are asymptotically equivalent.
Or
– g(n) is asymptotically tight bound for f(n)

7
 f(n) = Θ(g(n))
0 ≤ c1g(n) ≤ f(n) ≤ c2g(n)

8
• The following equations are asymptotically equivalent
• 5n2
• 2n2 - 5n + 10
• (8n2 + 2n – 3)
• (n2/5 + √n – 10 log n)
• n(n – 3)

As ‘n’ becomes large, the dominant term is some constant

times n2

9
 Lower and upper bounds (Example1)
• f(n) = 8n2 + 2n – 3
– To show that f(n) ∈ Θ (n2)
– We need to find the following three values.
– c1, c2 and no

• To find Lower bound we need c1 and no

• To find Upper bound we need c2 and no
– We will have two no, select the maximum no

10
 Finding c1 and no (Example1)
Lower bound: 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n)
f(n) = 8n2 + 2n – 3, f(n) ∈ Θ (n2)

• c1n2 ≤ 8n2 + 2n – 3 ?? c1 can be anything lesser

than the constant with n2 of
• 7n2 ≤ 8n2 + 2n – 3 the expression

• c1= 7
• No = 1 no =1
7(1) 2 ≤ 8(1) 2 +2(1)-3
7 ≤ 8+2-3
7≤7

11
 Finding c2 and no (Example1)
Upper Bound: 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n)
f(n) = 8n2 + 2n – 3, f(n) ∈ Θ (n2)
8n2 + 2n – 3 ≤ c2n2
8n2 + 2n – 3 ≤ 9n2
c2 can be anything greater
=>8n2 + 2n – 3 ≤ 9n2 than the constant with n2 of
the expression

c2 = 9
No = 1

12
 f(n) = Θ(g(n))
f(n) = 8n2 + 2n – 3
g(n) = n2,
c1= 7, c2= 10, n0= 1
1e + 11

8e + 10

6e + 10
f(n)
4e + 10

2e + 10

0
0 20000 40000 60000 80000 100000
n

13
OR
C1 = 1 / 4
n0 = 13
14
15
 Example 2

f(n) = 2n2 - 5n +10

f(n) ∈ Θ (n2)??

• 1n2 ≤ 2n2 - 5n +10 … for lower bound

– C1= 1, No = 1

• 2n2 - 5n +10 ≤ 2 n2 … for upper bound

– C1= 2, No = 2
16
 Practice

f(n) = (3n2 / 2 ) + (5n/2) – 3

f(n) ∈ Θ (n2)??

– C 1= 1
– C2= 2,
– No = ???

17
 Example 3

• f(n) = 3n+3
• g(n) = n
– f(n) ∈ Θ (n)

C1=2, No = 1
C2=4, No = 3

No = 3
18
Suppose …

• f(n) = 2n3 - 5n2 +10n+1

• g(n) = n2

• Is f(n) ∈ Θ (g(n)) ???

• No.

19
Lower bound: 5n2 ≤ 6n3
Upper Bound: 6n3 ≤ ?n2 … always false

20
 O-notation (Big-O)
• Sometimes we are only interested in proving
one bound or the other
• We use O-notation, when we have only an
asymptotic upper bound
• O(g(n)) = {f(n) | there exist positive constants ‘c’
and ‘n0’ such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0}
• We write it as

21
22
• If any quadratic function is in
is also a

• Example:
– f(n) = 2n2
– g(n) = n2
–f(n) = O (g(n))
for c = 5/2, n0 = 1

23
Practice examples

24
• 2n2 = O(n3):
2n2 ≤ cn3 ⇒ c = 1 and n0= 2

• n2 = O(n2)

n2 ≤ cn2 ⇒ c = 1 and n0= 1

• 1000n2+1000n = O(n2)

1000n2+1000n ≤ cn2 ⇒ c=1001 and n0 = 1000

25
 Ω - notation
• Just as O-notation provides an asymptotic upper bound on a
function, omega notation provides an asymptotic lower bound.
– Ω(g(n)) = the set of functions with a larger or same order of
growth as g(n)

26
• 5n2 = Ω(n)
– 0 ≤ cn ≤ 5n2, c = 1 and n0 = 1

• 100n + 5 ≠ Ω(n2)
– 0 ≤ cn2 ≤ 100n + 5

• n = Ω(n)

• n3 = Ω(n2)

27
28
 Theorem
•

if and only if

29