Random Variables
Chapter 2
1
• Random variable
– A numerical value to each outcome of a
particular experiment
-3 -2 -1 0 1 2 3
2
Definition of random variable
A random variable is a function that assigns a number to
each outcome in a sample space.
• If the set of all possible values of a random variable X is
countable, then X is discrete. The distribution of X is
described by a probability mass function:
p a P s S : X s a P X a
• Otherwise, X is a continuous random variable if there is a
nonnegative function f(x), defined for all real numbers x,
such that for any set B,
P s S : X s B P X B f x dx
B
f (x) is called the probability density function of X.
3
Example
• Example 1 : Machine Breakdowns
– Sample space : {electrical, mechanical, misuse}
– Their chances are 50%, 45% and 5% respectively
– Each of these failures may be associated with a repair
cost: 50, 200 and 350 respectively.
Cost is a random variable : 50, 200, and 350
can you provide an example of continuous RV? What is
the state space?
4
pmf’s and cdf’s
• The probability mass function (pmf) for a discrete
random variable is positive for a number of values
of X: x1, x2, …, and
p x 1
i 1
i
• The cumulative distribution function (cdf) for any
random variable X is F x P X x
F(x) is a monotonically increasing (nondecreasing)
function with
lim F x 0 and lim F x 1
x x
• For a discrete random variable X, F a p xi 5
x a i
Cumulative Distribution Function
-Example-
f(x) F(x)
Cumulative
Density Distribution
Function Function
0.5
0 x 2 0 2 6
Exercise
• Let X be the random variable for the daily revenue
(in thousand dollars). Suppose that X has the
following probability function.
f ( x)
0.25 for 0 x 2
0.5 for 2 x 3
0 otherwise
a. Plot f(x), b. Find F(2.5),
c. Find P(1.5<X<3) d. Plot F(x)
7
Some probability functions
Mass functions: Density functions
•Bernoulli •Uniform
•Binomial •Exponential
•Geometric •Normal
•Poisson
8
Bernoulli Random Variable
• An experiment has two possible outcomes, called “success”
and “failure”: sometimes called a Bernoulli trial
• The probability of success is p
• X = 1 if success occurs, X = 0 if failure occurs
Then p (0) = P{X = 0} = 1 – p and p (1) = P{X = 1} = p
X is a Bernoulli random variable with parameter p.
9
Binomial Random Variable
• A sequence of n independent Bernoulli trials are performed,
where the probability of success on each trial is p
• X is the number of successes
Then for i = 0, 1, …, n,
n i
p i P X i p 1 p
n i
i
where n
n!
i i !n i !
X is a binomial random variable with parameters n and p.
10
Geometric Random Variable
• A sequence of independent Bernoulli trials is performed
with p = P(success)
• X is the number of trials until (including) the first success.
Then X may equal 1, 2, … and
p i P X i 1 p
i 1
p, i 1, 2,...
X is named after the geometric
series:
Use this to verify that p i 1
i 1
11
Exercise
• An urn contains 3 green balls, 3 white balls and 4 black balls. Find out the following:
a. If 5 balls are withdrawn with replacement, what is the chance to get two of them white?
b. If balls are withdrawn with replacement, what is the chance that the first green ball will
appear at the 7th trial?
12
Multinomial example
if you are randomly choosing 8 people from an audience that
contains 50% democrats, 30% republicans, and 20% green
party, what’s the probability of choosing exactly 4 democrats,
?3 republicans, and 1 green party member
8!
P ( D 4, R 3, G 1) (.5) 4 (.3) 3 (.2)1
4! 3!1!
the multinomial distribution is a generalization of the Binomial PMF.
For n independent trials each of which leads to a success for exactly one
of k categories, with each category having a given fixed success
probability
Poisson Random Variable
X is a Poisson random variable with parameter > 0 if
e i
p i P X i , i 0,1,...
i!
note: i 0 i 0 i i !
p i 1 follows from e
X can represent the number of “events” that occur during
an interval of specified length
A Poisson random variable can also approximate a
binomial random variable with large n and small p if =
np: split the interval into n subintervals, and label the
occurrence of an event during a subinterval as “success”.
14
Example
For example, if new cases of West Nile Virus
in New England are occurring at a rate of
about 2 per month, then these are the
probabilities that: 0,1, 2, 3, 4, 5, 6, to 1000 to
1 million to… cases will occur in New
England in the next month:
Poisson Probability table
X P(X)
20 e 2
0 =.135
0!
1 21 e 2 =.27
1!
2 22 e 2 =.27
2!
3 23 e 2 =.18
3!
4 =.09
… …
?
3 cases in next two months? “re-
write the pmf”
17
Continuous random variables
A probability density function (pdf) is a mathematical
function which must satisfy:
f x 0
f x dx 1
b
P a X b f x dx (note P X a 0)
a
The cdf is:
18
Uniform random variable
X is uniformly distributed over an interval (a, b) if its pdf is
1
, a x b all we know about
f x b a
X is that it takes a
0, otherwise
value between a and b
Then its cdf is:
0, x a
x a
F x , a x b
b a
1, x b
19
Exponential random variable
X has an exponential distribution with parameter > 0 if its pdf is
e x , x 0
f x
Then its cdf is: 0, otherwise
0, x 0
F x x
1
- Exponential is derived
efrom 0
, xPoisson
- It is memoryless
20
Example
For example, if new cases of West Nile Virus
in New England are occurring at a rate of
about 2 per month, what is the probability that
the time to next case is <= 1 week?
Keep the units consistent
Gamma random variable
X has an gamma distribution with parameters > 0 and > 0
if its pdf is e x x 1
, x 0
f x
0, otherwise
It gets its name from the gamma function 0 e x x 1dx
If is an integer, then 1!
22
Normal random variable
X has a normal distribution with parameters and if its pdf
is
This is the classic “bell-shaped” distribution widely used
in statistics. It has the useful characteristic that a linear
function Y = aX+b is normally distributed with parameters
ab and (a . In particular, Z = (X – )/ has the
standard normal distribution with parameters 0 and 1.
23
Practice problem
If birth weights in a population are
normally distributed with a mean of 109
oz and a standard deviation of 13 oz,
a. What is the chance of obtaining a birth
weight of 141 oz or heavier when
sampling birth records at random?
b. What is the chance of obtaining a birth
weight of 120 or lighter?
Expectation
Expected value (mean) of a random variable is
i xi p xi , discrete
E X
xf x dx, continuous
-
Also called first moment
If the experiment is repeated and random variable
observed many times, it represents the long run average
value of the r.v.
25
Expectations of Discrete Random
Variables
• Bernoulli: E[X] = 1(p) + 0(1-p) = p
• Binomial: E[X] = np
• Geometric: E[X] = 1/p
• Poisson: E[X] = the parameter is the expected or
average number of “events” per interval; the random
variable is the number of events in a particular interval
chosen at random
26
Expectations of Continuous Random
Variables
• Uniform: E[X ] = (a + b)/2
• Exponential: E[X ] = 1/
• Gamma: E[X ] =
• Normal: E[X ] =
1 x 2
f x
2 2
e , x
2
27
Expectation of a function of a r.v.
• First way: If X is a r.v., then Y = g(X) is a r.v.. Find the
distribution of Y, then find E Y i yi p yi
• Second way: If X is a random variable, then for any real-
valued function g,
i g xi p xi , X discrete
E g X
g x f x dx, X continuous
-
If g(X) is a linear function of X:
E aX b aE X b
28
Higher-order moments
The nth moment of X is E[Xn]:
xi n p xi , discrete
i
E X
n
x n f x dx, continuous
-
The variance is
Var X E X E X
2
It is sometimes easier to calculate as:
Var X E X E X
2
2 “verify!”
29
Variances of Discrete Random Variables
• Bernoulli: E[X 2] = 1(p) + 0(1-p) = p; Var(X) = p – p2 =
p(1-p)
• Binomial: Var(X ) = np(1-p)
• Geometric: Var(X) = 1/p2 (similar trick as for E[X])
• Poisson: Var(X) = the parameter is also the variance
of the number of “rare events” per interval!
30
Variances of Continuous Random
Variables
• Uniform: Var(X ) = (b - a)2/2
• Exponential: Var(X ) = 1/
• Gamma: Var(X ) = 2
• Normal: Var(X ) = 2
1 x 2
f x
2 2
e , x
2
31
Joint Probability Mass Function
The function p(x, y) is a joint probability mass function of
the discrete random variables X and Y if
1. p(x, y) 0 for all (x, y)
2.
x y
p ( x , y ) 1
3. P(X = x, Y = y) = p(x, y)
For any region A in the xy plane,
P[(X, Y) A] = p (x, y)
A
32
Example
Two refills for a ballpoint pen are selected at random
from a box that contains 3 blue refills, 2 red refills
and 3 green refills. If X is the number of blue refills
and Y is the number of red refills selected, find
a. The joint probability mass function p(x, y), and
b. P[(X, Y) A], where A is the region
{(x, y)|x + y 1}
33
Example - Solution
The possible pairs of values (x, y) are (0, 0), (0, 1),
(1, 0), (1, 1), (0, 2), and (2, 0), where p(0, 1), for
example represents the probability that a red and a
green refill are selected. The total number of equally
likely ways of selecting any 2 refills from the 8 is:
8 8!
28
2 2!6!
34
Example - Solution
Hence, p(0, 1) = 6/28 = 3/14. Similar calculations yield the
probabilities for the other cases, which are presented in the
following table. Note that the probabilities sum to 1.
X
0 1 2 Row Totals
3 9 3 15
0
28 28 28 28
3 3 3
y 1 14 14 7
1 1
2 28 28
Column Totals 5 15 3
14 28 28
35
Example - Solution
3 2 3
x y 2 x y
p x, y
8
2
for x = 0, 1, 2; y = 0, 1, 2; 0 x + y 2.
b. P[(X, Y) A] = P(X + Y 1)
= p(0, 0) + p(0, 1) + p(1, 0)
3 3 9
28 14 28
9
14 36
Find
f(x), f(y), E(x)
37
Joint Density Functions
The function f(x, y) is a joint probability density
function of the continuous random variables X and Y if
1. f(x, y) 0 for all (x, y)
2. f(x, y)dxdy 1
3. P[(X, Y) A] =
f ( x, y )dx dy
A
For any region A in the xy plane.
38
Marginal Distributions
The marginal probability mass functions
of x alone and of Y alone are
g x p ( x , y ) and h y p ( x , y )
y x
for the discrete case.
39
Marginal Distributions
The marginal probability density functions of x
alone and of Y alone are
g x f ( x , y ) dy and h y f ( x , y ) dx
for the continuous case.
40
Example
52 (2 x 3 y ), 0 x 1,0 y 1
f ( x, y )
0, elsewhere
a)Verify whether f(x, y)dxdy 1
b) Find P[(X,Y) A], where A is the region {(x,y) | 0<x<½,
¼<y<½}.
44
Example – Solution
1 1
a) 2
f ( x, y )dxdy (2 x 3 y )dxdy
0 0
5
1 2 x 1
2x 6 xy
dy
0
5 5 x 0
1 2 1
2 6y 2 y 3y
dy
0
5 5 5 5 0
2 3
1
5 5
45
Example – Solution
b) P( X , Y ) A P(0 X 12 , 14 Y 12 )
1 1
2
2 2
(2 x 3 y )dxdy
1 0 5
4
1
2 x 12
2x
2
6 xy
dy
1 5 5 x 0
4
1 1
2
1 3y y 3y
2 2
dy
1 10 5 10 10 1
4 4
1 1 3 1 3 13
10 2 4 4 16 160
46
Example
Find g(x) and h(y) for the joint density function of
the previous example :
52 (2 x 3 y ), 0 x 1, 0 y 1
f ( x, y )
0, elsewhere
47
Example – Solution
By definition,
1 2 y 1
2 4 xy 6 y 4x 3
g ( x) f ( x, y )dy (2 x 3 y )dy
0
5 5 10 y 0
5
For 0x 1, and g(x)=0 elsewhere. Similarly,
1
2 4(1 3 y )
h( y ) f ( x, y )dx (2 x 3 y )dx
0
5 5
For 0 y 1, and h(y)=0 elsewhere
48
Example
verify that it is joint pdf and find f(x) and f(x)
Are f(x) and f(y) independent?
49
Statistical Independence
Let X and Y be two continuous random variables, with joint
probability density function f(x, y) and marginal probability
density functions g(x) and h(y), respectively. The random
variables X and Y are said to be statistically independent if
and only if
f x , y g x h y
for all (x, y) within their range.
50
Example
Two refills for a ballpoint pen are selected at random
from a box that contains 3 blue refills, 2 red refills
and 3 green refills. If X is the number of blue refills
and Y is the number of red refills selected, Show that
the random variables are not statistically independent.
51
Example – Solution
Let us consider the point (0,1). From the
following table:
X
0 1 2 Row Totals
3 9 3 15
0 28 28 28 28
3 3 3
y 1 14 14 7
1 1
2 28 28
5 15 3
Column Totals
14 28 28
52
Example – Solution
We find the three probabilities f(0,1), g(0),
and h(1) to be
3
f (0,1) ,
14
2
3 3 1 5
g (0) f (0, y ) ,
y 0 28 14 28 14
2
3 3 3
h(1) f ( x,1) 0
y 0 14 14 17
Clearly, f (0,1) g (0)h(1) and therefore X and Y are
not statistically independent.
53
Jointly Distributed Random Variables
E X Y E X E Y
E aX bY aE X bE Y
E a1 X 1 a2 X 2 ... an X n a1E X 1 ... an E X n
especially useful with indicator r.v.’s: IA = 1 if A occurs, 0
otherwise
54
Independent Random Variables
X and Y are independent if P X a, Y b P X a P Y b
This implies that: p x, y p X x pY y (discrete)
f x, y f X x fY y (continuous)
Also, if X and Y are independent, then for any functions h
and g, E g X h Y E g X E h Y
55
Covariance
The covariance of X and Y is:
Cov X , Y E X E X Y E Y E XY E X E Y
Show the same result starting from first principle?
If X and Y are independent then Cov(X,Y) = 0.
Properties (use the result above to prove them!):
56
Variance of a sum of r.v.’s
n n n n n
Var X i Cov X i , X j Var X i 2 Cov X i , X j
i=1 i=1 j=1 i=1 i=1 j<i
If X1, X2, …, Xn are independent, then
n n
Var X i Var X i
i=1 i=1
Show the above result by induction!
Examples 2.34 and 2.34 are applications on this important
result
57
practice
58
Practice
Ch2: 36, 37, 50, 53, 55, 63, 76, 79
59
Moment generating function
60
