3.

SPECIAL UNIVARIATE DISTRIBUTIONS

3.1 The Bernoulli distribution

The distribution is named after James Bernoulli who investigated independent

trials at the end of the 17th century. It is a discrete distribution with two possible
outcomes labeled as x=0 and x=1; in which x=1 (“success”) occurs with
probability p and x=0 (“failure”) occurs with probability q = 1-p, where 0 < p < 1.
It therefore has a probability density function given by

 p, for x=1

p ( X  x)  1  p, for x=0
0,
 elsewhere

Which can also be written as

 p x (1  p)1 x , x=0, 1
f ( x)  
0, elsewhere

The corresponding probability distribution is given by

 p, for x=1
F ( x)  
1  p, for x=0

The performance of a fixed number of trials with fixed probability of success on

each trial in known as a Bernoulli trial. The distribution of heads and tails in
coin tossing is an example of a Bernoulli distribution with p=q= ½. The Bernoulli
distribution is the simplest discrete distribution and it is the building block for
other more complex discrete distributions e.g. the binomial distribution, the
hypergeometric distribution, etc.

3.1.1 Some distributional quantiles.

The r-th moment about the origin is given by

1
 1
E ( X r )   x r f ( x)
x 0
1
  x r p x q1 x
x 0

 0r p 0 q  1r pq 0
p

Therefore E ( X )  p and E ( X 2 )  p .

Hence the mean of a Bernoulli distributed random variable X (say) is p and the
variance is pq .

The moment generating function is also given by

M (t )  E (etx )
1
  etx f ( x)
x 0
1
  etx p x q1 x
x 0

 e0 p 0 q  et pq 0
 pet

Therefore

d
E( X )  M (t ) |t  0
dt
d
 pet |t  0 and
dt
 pet |t  0
p

2
 d2
E( X 2 )  M (t ) |t 0
dt 2
d2
 2 pet |t 0
dt
d
 pet |t 0
dt
 pet |t  0
p

Hence

Var ( X )  E ( X 2 )   E ( X ) 
2

 p  p2
 pq

3.2 Binomial Distribution

If p is the probability that an event will occur in any single Bernoulli trial (i.e.
the probability of success) and q = 1 - p is the probability that it will fail to occur
in a single Bernoulli trial (i.e. the probability of failure), then the probability that
the event will occur exactly x times in n trials ( i.e. x successes and n-x failures)
is given by

 n  x
  p 1  p  ,
n x
x=0, 1, 2, ..., n
p( X  x)   x 
0,
 elsewhere

The random variable X denotes the number of successes in n trials and x=0, 1,
2, …, n.

Note that n! = n(n-1) (n-2)….1 and 0!=1

Example 1

Find the probability of getting exactly 2 heads in 6 tosses of a fair coin.

Solution.

3
n=6, x=2, p=q= ½

n
p(X=x) =   p x 1  p 
n x

 x

Thus

62
 6  1   1 
2

P(x=2) =     
 2 2   2 

2 4
6!  1   1 
   
= 4!2!  2   2  =15/64.

Example 2

Find the probability of getting at least 4 heads in 6 tosses of a fair coin.

Solution

p(x≥4) = p(x=4) + p(x=5) +p(x=6)

 6  1   1   6 1   1   6 1   1 
4 2 5 1 6 0

=                 
 4 2   2   5 2   2   6 2   2 

=15/64 +6/64 +1/64 =11/32

 n  x n  x
  p q , x=0,1,2,....,n
f(x) = p(X=x) =  x  is often called the Binominal distribution.
0,
 elsewhere

Because for x=0, 1, 2, …, n, it corresponds to successive terms of the binomial

expansion

n n n

q  p  q n    q n 1 p    q n  2 p 2  ...    p n
n

1  2 n

n
n
=    p x q n x
x 0  x 

4
The special case of a binomial distribution with n=1 is also called the Bernoulli
distribution.

3.2.1 Some distributional quantiles

The mean of X is given by

μ = E(x) = ∑ xf(x)

n
n
=  x.  p x q n  x
x 0  x

n
n!
=  x. p x q n x
x 0 x ! n  x !

n
n!
= p x q n x
x 1  x  1! n  x !

= np 
n
 n  1! p x1q n x
x 1  x  1! n  x !

n
 n  1 x 1 n  x
= np   p q
x 1  x  1 

m
m
= np    p y q m  y , where x=y+1, n=m+1
y 0  y 

=np× 1=np.

And the variance of X is given by

var(x) = E(x2) – [E(x)]2

= E(x2) – E(x) +E(x) -[E(x)]2

=E[x(x-1)] +np – n2p2

Now,

5
E[x(x-1)]=∑ x(x-1) f(x)

n
n
=  x  x  1   p x q n  x
x 0  x

n
n!
 x  x  1. x! n  x ! p qx n x

= x 0

n
n!
  x  2 ! n  x ! p q
x n x

= x2

n
n  n  1 p 2 
 n  2 ! p x  2 q n  x
x  2  x  2 ! n  x !
=

n
 n  2  x2 n x
n  n  1 p 2   p q
x2  x  2 
=

m
m
n  n  1 p 2    p y q m  y
y 0  y 
= , where x=y+2, n=m+2

=n(n-1)p2× 1 ==n(n-1)p2

Hence,

Var(x) = n(n-1)p2 +np – n2p2

= n2p2 - n2p + np - n2p2

=np - np2

=np(1-p) = npq

6
Note.

We can also use the r-th moment about the origin to obtain both the mean and

variance of a binomially distributed random variable.

Furthermore, the moment generating function can be similarly used.

3.3 The Poisson distribution

Let X be a discrete random variable that can take on the values 0, 1, 2,… such
e  x
f ( x)  P( X  x) 
that the probability function of X is given by x! where x is a
given positive constant. This distribution is called the Poisson distribution (after
S. D. Poisson, who discovered it in the early part of the nineteenth century), and
a random variable having this distribution is said to be Poisson distributed.

3.3.1 Some distributional quantiles

The first moment about the origin is given by


E ( X )   x f ( x)
x 0

e  x
x
x 0 x!

x
e 
 ( x  1)!
x 1

  2 3 4 5 
 e       ...
 0! 1! 2! 3! 4! 
 2 3 4 5 

 e  1        ...
 2! 3! 4! 5! 
 
 e e


Which is the mean of a Poisson distributed random variable.

Similarly,

7
 
E ( X 2 )   x 2 f ( x)
x 0

e  x
  x2
x 0 x!

x
 e   x 2
x 1 x!

x
 e    x 2  x  x 
x 1 x!

 x 
x
 e    x  x  1  e  x
x 1 x! x 1 x!

x
 e    x  x  1 
x 1 x!

x
 e  
x2 ( x  2)!
2 3 4 5 6 
 e       ...  
 0! 1! 2! 3! 4! 
 2 3 4 
 e    2 1       ...  
 2! 3! 4! 
 2 
e  e 
 2  

Therefore

Var ( X )  E ( X 2 )   E ( X ) 
2

 2    2


Note

The mean and variance of a Poisson distributed random variable are equal i.e.,
 2   .

And its moment generating function is given by

8
M (t )  e  ( e 1) ,
t

3.4 The Hypergeometric Distribution

Suppose that a box contains b black marbles and r red marbles. Let us perform
n trials of an experiment in which a marble is chosen at random, its color noted,
and then the marble is put back in the box. This type of experiment is often
referred to as sampling with replacement. In such a case, if X is the random
variable denoting the number of black marbles chosen (successes) in n trials,
then using the binomial distribution:

 n  x n  x

p ( X  x)   x 
pq , x=0,1,2,....,n
0,
 elsewhere

we see that the probability of exactly x successes is

n
p( X  x)    p x q n  x
 x

n x
 n  b   r 
x

     
 x  b  r   b  r 

 n  b x .r n  x
  , x= 0,1,2,...,n
 x  b  r 
n

Since

b r
p ,q  1 p 
br br

If we modify the above experiment so that sampling is without replacement, i.e.

the marbles are not replaced after being chosen, then

9
  b  r 
  
x n  x
p( X  x)   
b  r 
 
 n  , x = max(0, n-r), …. min(n,b) …………………………………..1

This is the Hypergeometric distribution. The mean and variance for this

distribution are:

nb

br
and

nbr  b  r  n 
2 
 b  r   b  r  1
2

If we let the total number of black and red marbles be N, while the probability of

black and red marbles are p and q respectively, then equation 1 above becomes

 Np  Nq 
  
x n  x
p( X  x)   
N
  , where Since
n

b b r r
p  , q 
br N br N

or b=Np and r=Nq.

In this case, the mean and variance become

10
  np

and

npq  N  n 
2 
N 1

Example 3

A box contains 6 blue marbles and 4 red marbles. An experiment is performed

in which a marble is chosen at random and its color noted, but the the marble
is not replaced. Find the probability that after 5 trials of the experiment, 3 blue
marbles would have been chosen.

Solution

N=6 +4 = 10, n=5, x=3, p=6/10=3/5,

q=4/10=2/5

Therefore

10  3 5 10  2 5   6  4 
     
p( x  3)   3  2   3  2  10
 
10  10  21
   
5 5

11