Fundamentals
Data Structures and Algorithm Analysis in C, by Mark Allen Weiss, 2nd
edition, 1997, Addison-Wesley, ISBN 0-201-49840-5
Danang University of Science and Technology
Dang Thien Binh
dtbinh@[Link]
Readings and References
Reading
Chapters 1-2, Data Structures and Algorithm Analysis in C, Weiss
Other References
Data Structures 2
Mathematical Background
Today, we will review:
Logs and exponents
Series
Recursion
Motivation for Algorithm Analysis
Data Structures 3
Powers of 2
Many of the numbers we use will be powers of 2
Binary numbers (base 2) are easily represented in digital
computers
each "bit" is a 0 or a 1
20=1, 21=2, 22=4, 23=8, 24=16, 28=256, …
an n-bit wide field can hold 2n positive integers:
0 k 2n-1
Data Structures 4
Unsigned binary numbers
Each bit position represents a power of 2
For unsigned numbers in a fixed width field
the minimum value is 0
the maximum value is 2n-1, where n is the number of bits in
the field
Fixed field widths determine many limits
5 bits = 32 possible values (25 = 32)
10 bits = 1024 possible values (210 = 1024)
Data Structures 5
Binary, Hex, and Decimal
28=256
27=128
26=64
25=32
24=16
21=2
22=4
23=8
20=1
Hex16 Decimal10
1 1 0x3 3
1 0 0 1 0x9 9
1 0 1 0 0xA 10
1 1 1 1 0xF 15
1 0 0 0 0 0x10 16
1 1 1 1 1 0x1F 31
1 1 1 1 1 1 1 0x7F 127
1 1 1 1 1 1 1 1 0xFF 255
Data Structures 6
Logs and exponents
Definition: log2 x = y means x = 2y
the log of x, base 2, is the value y that gives x = 2y
8 = 23, so log28 = 3
65536= 216, so log265536 = 16
Notice that log2x tells you how many bits are needed to
hold x values
8 bits holds 256 numbers: 0 to 28-1 = 0 to 255
log2256 = 8
Data Structures 7
x = 0:.1:4
y = 2.^x
plot(x,y,'r')
hold on
plot(y,x,'g')
plot(y,y,'b')
2x and log2x
Data Structures
x = 0:10
y = 2.^x
plot(x,y,'r')
hold on
plot(y,x,'g')
plot(y,y,'b')
2x and log2x
Data Structures
Example: log2x and tree depth
7 items in a binary tree, 3 = log27+1 levels
2 6
1 3 5 7
Data Structures 10
Properties of logs (of the mathematical
kind)
We will assume logs to base 2 unless specified
otherwise
log AB = log A + log B
A=2log2A and B=2log2B
AB = 2log2A • 2log2B = 2log2A+log2B
so log2AB = log2A + log2B
note: log AB log A•log B
Data Structures 11
Other log properties
log A/B = log A – log B
log (AB) = B log A
log log X < log X < X for all X > 0
log log X = Y means
log X grows slower than X
2Y
2 X
called a “sub-linear” function
Data Structures 12
A log is a log is a log
Any base x log is equivalent to base 2 log within a con-
stant factor
log x B log x B
x log x B B
B 2log 2 B
(2log 2 x ) log x B 2log 2 B
x 2log 2 x
2log 2 x log x B 2log 2 B
log 2 x log x B log 2 B
log 2 B
log x B
log 2 x
Data Structures 13
Arithmetic Series
N
S (N ) 1 2 N i
i 1
The sum is
S(1) = 1
S(2) = 1+2 = 3
S(3) = 1+2+3 = 6
N
N ( N 1)
i 1
i
2
Why is this formula useful?
Data Structures 14
Quicky Algorithm Analysis
Consider the following program segment:
for (i = 1; i <= N; i++)
for (j = 1; j <= i; j++)
printf(“Hello\n”);
How many times is “printf” executed?
Or, How many Hello’s will you see?
Data Structures 15
What is actually being executed?
The program segment being analyzed:
for (i = 1; i <= N; i++)
for (j = 1; j <= i; j++)
printf(“Hello\n”);
Inner loop executes “printf” i times in the ith iteration
j goes from 1 to i
There are N iterations in the outer loop
i goes from 1 to N
Data Structures 16
Lots of hellos
Total number of times “printf” is executed =
N
N ( N 1)
1 2 3 ... i
2
Congratulations
i 1
- You’ve just analyzed your first pro-
gram!
Running time of the program is proportional to N(N+1)/2 for
all N
Proportional to N2
Data Structures 17
Recursion
Classic (bad) example: Fibonacci numbers Fn
1, 1, 2, 3, 5, 8, 13, 21, 34, …
First two are defined to be 1
Rest are sum of preceding two
Leonardo Pisano
F =F +F (n > 1) Fibonacci (1170-1250)
n n-1 n-2
Data Structures 18
Recursive Procedure for Fibonacci
Numbers
int fib(int i) {
if (i < 0) return 0;
if (i == 0 || i == 1)
return 1;
else
return fib(i-1)+fib(i-2);
}
Easy to write: looks like the definition of Fn
But, can you spot the big problem?
Data Structures 19
Recursive Calls of Fibonacci Procedure
Re-computes fib(N-i) multiple times!
Data Structures 20
Iterative Procedure for Fibonacci Num-
bers
int fib_iter(int i) {
int fib0 = 1, fib1 = 1, fibj = 1;
if (i < 0) return 0;
for (int j = 2; j <= i; j++) {
fibj = fib0 + fib1;
fib0 = fib1;
fib1 = fibj;
}
return fibj;
}
More variables and more bookkeeping but avoids
repetitive calculations and saves time.
Data Structures 21
Recursion Summary
Recursion may simplify programming, but beware
of generating large numbers of calls
Function calls can be expensive in terms of time and
space
Be sure to get the base case(s) correct!
Each step must get you closer to the base case
Data Structures 22
Motivation for Algorithm Analysis
Suppose you are
given two algos A and
B for solving a prob- TA
Run Time
lem
The running times
TA(N) and TB(N) of A
and B as a function of TB
input size N are given
Input Size N
Which is better?
Data Structures 23
More Motivation
For large N, the running time of A and B is:
Now which
algorithm would
Run Time
TA(N) = 50N
you choose?
TB(N) = N2
Input Size N
Data Structures 24
Asymptotic Behavior
The “asymptotic” performance as N , regard-
less of what happens for small input sizes N, is
generally most important
Performance for small input sizes may matter in
practice, if you are sure that small N will be com-
mon forever
We will compare algorithms based on how they
scale for large values of N
Data Structures 25
