Scribd
Upload
37 views

Basic of Algorithms Analysis: Computational Tractability

algorithm analysis

Uploaded by

Mengistu Ketema
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
37 views

Basic of Algorithms Analysis: Computational Tractability

algorithm analysis

Uploaded by

Mengistu Ketema
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 4

Computational Tractability

2. Basic of Algorithms Analysis

As soon as an Analytic Engine exists, it will necessarily guide the future course of the science. Whenever any result is sought by its aid, the question will arise - By what course of calculation can these results be arrived at by the machine in the shortest time? - Charles Babbage

"For me, great algorithms are the poetry of computation. Just like verse, they can be terse, allusive, dense, and even mysterious. But once unlocked, they cast a brilliant new light on some aspect of computing." - Francis Sullivan

Charles Babbage (1864)


Algorithm Design by va Tardos and Jon Kleinberg
Copyright 2005 Addison Wesley Slides by Kevin Wayne

Analytic Engine (schematic)


2

Computational Tractability
Worst case running time. Obtain bound on largest possible running time of algorithm on input of a given size N, and see how this scales with N. Generally captures efficiency in practice. Draconian view, but hard to find effective alternative.
! !

Why It Matters

Desirable scaling property. When the input size increases by a factor of 2, the algorithm should only slow down by some constant factor C.
There exists constants c > 0 and d > 0 such that on every input of size N, its running time is bounded by c Nd steps.

Def. An algorithm is efficient if it has polynomial running time. Justification. It really works in practice!
3 4

Asymptotic Order of Growth


Upper bounds. T(n) is O(f(n)) if there exist constants c > 0 and n0 # 0 such that for all n # n0 we have T(n) $ c f(n). Lower bounds. T(n) is "(f(n)) if there exist constants c > 0 and n0 # 0 such that for all n # n0 we have T(n) # c f(n). Tight bounds. T(n) is !(f(n)) if T(n) is both O(f(n)) and "(f(n)). Ex: T(n) = 32n2 + 17n + 32. T(n) is O(n2), O(n3), "(n2), "(n), and !(n2) . T(n) is not O(n), "(n3), !(n), or !(n3).
! !

Notation
Slight abuse of notation. T(n) = O(f(n)). Vacuous statement. Any comparison-based sorting algorithm requires at least O(n log n) comparisons.

Properties
Transitivity. If f = O(g) and g = O(h) then f = O(h). If f = "(g) and g = "(h) then f = "(h). If f = !(g) and g = !(h) then f = !(h).
! ! !

Asymptotic Bounds for Some Common Functions


Polynomials. a0 + a1n + + adnd is !(nd) if ad > 0. Polynomial time. Running time is O(nd) for some constant d independent of the input size n.

Additivity. If f = O(h) and g = O(h) then f + g = O(h). If f = "(h) and g = "(h) then f + g = "(h). If f = !(h) and g = O(h) then f + g = !(h).
! ! !

Logarithms. O(log a n) = O(log b n) for any constants a, b > 0.


can avoid specifying the base, assuming it is a constant

Logarithms. For every x > 0, log n = O(nx).


log grows slower than every polynomial

Exponentials. For every r > 1 and every d > 0, nd = O(rn).


every exponential grows faster than every polynomial

Linear Time: O(n)


Linear time. Running time is at most a constant factor times the size of the input. Computing the maximum. Compute maximum of n numbers a1, , an.
max % a1 for i = 2 to n { if (ai > max) max % ai }

Linearithmic Time: O(n log n)


Linearathmic time. Arises in divide-and-conquer algorithms. Sorting. Mergesort and heapsort are sorting algorithms that perform O(n log n) comparisons. Largest empty interval. Given n time-stamps x1, , xn on which copies of a file arrive at a server, what is largest interval of time when no copies of the file arrive? O(n log n) solution. Sort the time-stamps. Scan the sorted list in order, identifying the maximum gap between successive time-stamps.

10

Quadratic Time: O(n2)


Quadratic time. Enumerate all pairs of elements. Closest pair of points. Given a list of n points in the plane (x1, y1), , (xn, yn), find the pair that is closest. O(n2) solution. Try all pairs of points.
min % (x1 - x2)2 + (y1 - y2)2 for i = 1 to n { for j = i+1 to n { d % (xi - xj)2 + (yi - yj)2 if (d < min) min % d } }

Cubic Time: O(n3)


Cubic time. Enumerate all triples of elements. Set disjointness. Given n sets S1, , Sn each of which is a subset of 1, 2, , n, is there some pair of these which are disjoint? O(n3) solution. For each pairs of sets, determine if they are disjoint.

don't need to take square roots

foreach set Si { foreach other set Sj { foreach element p of Si { determine whether p also belongs to Sj } if (no element of Si belongs to Sj) report that Si and Sj are disjoint } }

Remark. "(n2) seems inevitable, but this is just an illusion.

Chapter 5

11

12

Polynomial Time: O(nk) Time


Independent set of size k. Given a graph, are there k nodes such that no two are joined by an edge? O(nk) solution. Enumerate all subsets of k nodes.
foreach subset S of k nodes { check whether S in an independent set if (S is an independent set) report S is an independent set } }

Exponential Time
Independent set. Given a graph, what is maximum size of an independent set? O(n2 2n) solution. Enumerate all subsets.
S* % & foreach subset S of nodes { check whether S in an independent set if (S is largest independent set seen so far) update S* % S } }

Check whether S is an independent set = O(k2). Number of k element subsets = " n % n (n ( 1) (n ( 2) L (n ( k + 1) nk = ) $ ' 2 k k O(k n / k!) = O(n ). k ( k ( 1) ( k ( 2) L (2) (1) k! #k &
assuming k is a constant

!
13 14

You might also like