Department of Computer Engineering, Faculty of Engineering,
Kasetsart University, THAILAND
1st Semester 2014
(Aug – Dec)
Assoc. Prof. Anan Phonphoem, Ph.D.
http://www.cpe.ku.ac.th/~anan
Department of Computer Engineering, Faculty of Engineering,
Kasetsart University, THAILAND
Definition:
FX(x) = P[X x]
• Contain complete information about
the probability model of the random variable
• PMF CDF
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 4
FFXX(x
(x21)) == P[X
P[X xx21]] FX() = P[X ]
PX(x)
- x1 x2 x
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 5
Theorem: For a discrete random variable X
with SX={x1,x2,…} & x1 x2 …
1) Fx(-) = 0 and Fx() = 1
2) x’ x, Fx(x’) Fx(x)
3) For xi Sx and = +small number
Fx(xi) – Fx(xi - ) = Px(xi)
4) Fx(x) = Fx(xi) x, xi x < xi+1
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 6
FX (-) PX(x) FX ()
-
x
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 7
PX(x) FX (x) FX (x’)
-
x
x x’
Monotonic Increasing
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 8
PX(x) FX (xi)
FX (xi - )
-
xi x
Discontinuity = PX(x)
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 9
PX(x) FX (xi)
-
x
xi xi+1
Horizon line
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 10
Theorem: For a discrete random variable X
with SX={x1,x2,…} & x1 x2 …
1) Fx(-) = 0 and Fx() = 1 From 0 to 1
2) x’ x, Fx(x’) Fx(x) Monotonic Increasing
3) For xi Sx and = +small number
Fx(xi) – Fx(xi - ) = Px(xi) Discontinuity = PX(x)
4) Fx(x) = Fx(xi) x, xi x < xi+1 Horizon line
Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 11
• For a Binomial RV, # of fail programs in 5
tests with p = 0.2
PK(k) = () 5
k
(0.2)k(0.8)5-k k = 0,1,2,…,5
0 Otherwise
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 12
k PK(k) k PK(k)
0 0.33 3 0.05
1 0.41 4 0.01
2 0.20 5 0
. . .
PK(k) 0.5 FK(k) 1
.
0.25
0
0.5
0
.
0 1 2 3 4 5 0 1 2 3 4 5
k k
Note: FK(k) is continuous from right
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 13
b a,
FX(b) - FX(a) = P[a < X b]
Difference of the CDF is the probability that RV takes on the
value between two points
PX(x)
x
a b
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 14
• Study RV Average
• What is the average of an RV?
• A single number that describes the RV
• An example of statistic
• What is Statistic?
• Numbers that collect all information of things
under our interesting
• Averages: mean, mode, and median
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 16
Picture from http://rchsbowman.wordpress.com/2008/09/15/statistics-notes-%E2%80%94-measures-of-variation/
Can you identify the Mean, Mode, and Median for each team?
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 17
• Mean:
• Sum / #terms
• Mode:
• Most common value
• PX(xmod) PX(x) x
• Median:
• The middle of the data set
• P[X < xmed] = P[X > xmed]
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 18
• Adding all measurements / #terms
Definition:
E[X] = X = xPX(x)
xSX
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 19
• Experiment Random Variable X
• Perform n independent trials
• The value X takes on ith trial x(i)
The average
x(3)
x(2) x(1) n
mn = 1 x(i)
x(5) n i=1
x(4)
x(n)
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 20
• Each x(i) takes value in the set Sx
• Out of n trials, assume xS occurs N X X times
X(x(1)) = x
x(3)
x(2) x(1)
x(4) x(5) -2.0 -1.5 -1.0 -0.5 0 0.5
x
x(n)
SX
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 21
n
mn = 1 x(i)
n i=1
1 NX
mn = n xSxNxx = xSxn x
NA NX
P[A] = lim PX(x) = lim
n→ n n→ n
limm n = xPX(x)
n→ xS x
E[X] = xPX(x)
xSX
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 22
E[X] = xPX(x)
xSX
• Example:
1/4 t=0
PT(t) = 3/4 t=2
0 Otherwise
• E[T] = ?
= 0(1/4) + 2(3/4) = 3/2
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 23
PX(x)
PT(t) Probability MASS Function
1
0.5
0 t
0 1 2
E[X]
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 24
1–p x=0
PX(x) = p x=1
0 Otherwise
E[X] = xPX(x)
xSX
Sx = {0,1}
E[X] = 0(1-p) + 1(p)
=p
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 25
p(1 – p)x-1 x = 1,2,3,…
PX(x) = 0 Otherwise
E[X] = xPX(x)
xSX
= xp(1-p)x-1 = xpqx-1
x=1 x=1
= p
xqx-1
? 1
E[X] = p
x=1
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 26
n
Sn = xqx-1
x=1
Sn = 1 + 2q + 3q2 + 4q3 + … + nqn-1
q Sn= q + 2q2 + 3q3 + … + (n-1)qn-1 + nqn
(1-q) Sn= 1 + q + q2 + q3 + … + qn-1 - nqn
q(1-q) Sn= q + q 2 + q3 + … +qn – nqn+1
(1-q)2 Sn= 1 – (n+1)qn + nqn+1
n
0 n+1 0
(n+1)q nq
Sn = 1 2 - 2
+
(1-q) (1-q) (1-q)2
1
Sn = E[X]= p xqx-1 = 1
p2 x=1 p
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 27
From example:
• Find the number of tests until find a fail
program
• We have p = 0.2 2/10 1/5
• E[X] = 1/p = 5
• Intuitively, on average, we will find the fail
program after 5 tests.
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 28
xe- x = 0,1,2…
PX(x) = x!
0 Otherwise
E[X] = xPX(x) = x x -
e
x!
e
x=0 x=0
=
e- x-1
=
k
e-
(x-1)! k!
(x=0) 0 x=1 k=0
= e e- E[X] =
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 29
Uniform
1/(j-k+1) x = k,k+1,k+2,..,j
Equiprobable E[X] =(j+k)
outcomes 2
0 Otherwise
Bernoulli 1–p x=0
p x=1 E[X] = p
Pass/Fail 0 Otherwise
Geometric
p(1 – p)x-1 x = 1,2,3,…
E[X] = 1/p
# tests until fail 0 Otherwise
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 30
Binomial n
( ) p (1 – p)
x
x n-x x=1,2,…,n
E[X] = np
# fails in n tests 0 Otherwise
Pascal x-1
( ) p (1 – p) k x-k x = k,k+1,…
k-1
# tests until k E[X] = k/p
fails 0 Otherwise
Poisson (T)xe-(T) x = 0,1,2…
E[X] =
occurrence in a x! = T
period 0 Otherwise
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 31
Theorem: Let p = /n ( > 0 and n > )
Binomial PMF Poisson PMF (parameter )
PX(x) =
( ) p (1 – p)
x
x n-x x=0,1,2,…,n
0 Otherwise
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 32
p = /n
n x
PX(x) =
() x
( n
) (1 – n
)n-x x=0,1,2,…,n
0 Otherwise
n(n-1)…(n-x+1) x (1 – )n-x
= n
e-
nx x!
(1 – )n
n
lim (n-j) = 1
n→ n (1 – )x
n 1
xe- x = 0,1,2…
lim PX(x) = x!
n→
0 Otherwise
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 33
s1 s2 s3 X(s)
s4 s5 Y=g(X)
sn
-2.0 -1.5 -1.0 -0.5
x0
SX
0 1 2 3 4
y5 6 7 8
SY
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 35
• From sample values of the random variable,
use these values to compute other quantities.
• Y = g(X)
• Example:
• Find a decibel value form signal-to-noise ratio
• Watt dB
Calculate 0.1W relative to 1 mW (milliwatt)
100 mW
GdB = 10 log10( ) = 20 dB 20 dBm
1 mW
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 36
• Random Variable X = # pages in one fax
• PX(x) = number of pages in each fax
• Charging plan
• 1st page = 10 Baht
• 2nd page = 9 Baht
• …
• 5th page = 6 Baht
• 6 – 10 pages = 50 Baht
• Find the charge in Baht for sending one fax
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 37
• Random Variable Y = the charge in Baht
for sending one fax
10.5X – 0.5X2 1X 5
Y = g(X) = 50 6 X 10
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 38
Theorem:
PY(y) = PX(x)
x:g(x)=y
P[Y=y] = of all outcomes X = x for which Y = y
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 39
• Suppose all faxes contain 1,2,3, or 4 pages
with equal probability
• Find PMF and E[Y]
¼ x = 1,2,3,4
PX(x) = 0 Otherwise
• SX = {1,2,3,4}
• SY = {10,19,27,34}
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 40
¼ X=1 Y=10
¼ X=2 Y=19
¼ X=3 Y=27
¼
X=4 Y=34
¼ y = 10,19,27,34
PY(y) = 0 Otherwise
E[Y] = (1/4)(10 + 19 + 27 + 34) = 22.5 Baht
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 41
0.15 x = 1,2,3,4
PX(x) = 0.1 x = 5,6,7,8
0 Otherwise
• Find PMF and E[Y]
0.2
PX(x)
0.1
0
0 2 4 6 8
x
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 42
X=1 Y=10
0.15
0.15 X=2 Y=19
0.15 X=3 Y=27
0.15
X=4 Y=34
0.1
X=5 Y=40
0.1
0.1 X=6
0.1
X=7 Y=50
X=8
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 43
From Theorem
0.15 y = 10,19,27,34
PY(5) = 0.1 y = 40
PY(y) = PY(6)+PY(7)+PY(8) = 0.3 y = 50
0 Otherwise
0.4
PY(y)
E[Y] = 0.15(10+19+27+34)
0.2
+ 0.1(40) + 0.3(50)
0 = 32.5
10 19 27 34 40 50
y
3 July 2013 Anan Phonphoem, Dept.of Computer Engineering, Kasetsart Universty 44