Theory of Distribution (MSTA021/STT201M)

CHAPTER 3- Discrete Random Variables and their Probability Distributions

3.1. Basic Definition

3.2. The Probability Distribution for a Discrete Random Variable

3.3. The Expected Value of a Random Variable or a Function of a Random Variable

3.4. The Binomial Probability Distribution

3.5. The Geometric Probability Distribution

3.6. Negative Binomial Probability Distribution (Self Study)

3.7. Hypergeometric Probability Distribution (Self Study)

3.8. The Poisson Probability Distribution

3.9. Moments and Moment-Generating Functions

3.11. Tchebysheff’s Theorem

 Recap of Chapter 1

In statistics the word “Distribution” describes the way in which a characteristic is spread over members of a class.
E.G.

• It describes the number of times each possible outcome occurs in a sample (Frequency distribution).

OR

• It assigns a probability to each measurable subset of the possible outcomes of a random experiment, survey,
or procedure of statistical inference (Probability distribution).
 Basic Definition

Definition 3.1

A random variable 𝑌𝑌 is said to be discrete if it can assume only a finite or countably infinite number of distinct
values (counted items). E.g.

• number of children in a household

• number of magazines

• number of student in a class

 The Probability and Distribution for a Discrete Random Variable

𝑌𝑌 = random variable
𝑦𝑦 = value for a random variable
𝑃𝑃 𝑌𝑌 = 𝑦𝑦 = the probability that 𝑌𝑌 takes on the value of 𝑦𝑦

Definition 3.2

The probability that 𝑌𝑌 takes on the value 𝑦𝑦, 𝑃𝑃(𝑌𝑌 = 𝑦𝑦), is defined as the sum of the probabilities of all sample
points in S that are assigned the value 𝑦𝑦. We will sometimes denote 𝑷𝑷 (𝒀𝒀 = 𝒚𝒚) by 𝒑𝒑(𝒚𝒚).

Definition 3.3

The probability distribution for a discrete variable 𝒀𝒀 can be represented by a formula, a table or a graph that
provides 𝒑𝒑(𝒚𝒚) = 𝑷𝑷 (𝒀𝒀 = 𝒚𝒚) for all 𝒚𝒚.

Example 3.1
 The Probability and Distribution for a Discrete Random Variable

Theorem 3.1

For any discrete probability distribution, the following must hold:

1. 0 ≤ 𝑝𝑝 𝑦𝑦 ≤ 1 for all y.

2. ∑𝑦𝑦 𝑝𝑝(𝑦𝑦) = 1, where the summation is over all values of y with nonzero probability.

Note: The probability distribution we derived are models, not exact representations, for the frequency
distribution of populations of real data that occur (or would be generated) in nature.
 The Expected Value of a Random Variable

Definition 3.4

Let Y be a discrete random variable with probability function p(y). Then the expected value of Y, E(Y), is defined
to be:

𝐸𝐸 𝑌𝑌 = � 𝑦𝑦𝑦𝑦(𝑦𝑦)
𝑦𝑦

And for 𝑋𝑋

𝐸𝐸 𝑋𝑋 = � 𝑥𝑥𝑥𝑥(𝑥𝑥)
𝑥𝑥

Definition 3.5

If 𝑌𝑌 is a random variable with mean 𝐸𝐸(𝑌𝑌) = µ, the variance of a random variable 𝑌𝑌 is defined to be the expected
value of (𝑌𝑌 − µ)2 . That is, 𝑉𝑉 𝑌𝑌 = 𝐸𝐸 [(𝑌𝑌 − µ)2 ]. The standard deviation of 𝑌𝑌 is the positive root of 𝑉𝑉 𝑌𝑌 .
 The Expected Value of a Random Variable

Example 3.2

Consider a discrete random variable 𝒀𝒀 with probability distribution as follows:

𝒀𝒀 𝒑𝒑(𝒚𝒚)

0 1/8
1 1/4
2 3/8
3 1/4

Find the 𝐸𝐸(𝑌𝑌) and 𝑉𝑉(𝑌𝑌)

Theorem 3.3
Let 𝑐𝑐 be a constant. Then 𝐸𝐸(𝑐𝑐) = 𝑐𝑐.

Theorem 3.6
Let 𝑌𝑌 be a discrete random variable with probability distribution 𝑝𝑝(𝑦𝑦) and mean 𝐸𝐸(𝑌𝑌) = µ; then
𝑉𝑉 𝑌𝑌 = 𝜎𝜎 2 = 𝐸𝐸 𝑌𝑌 − 𝜇𝜇 2 = 𝐸𝐸 𝑌𝑌 2 − 𝜇𝜇2
 The Binomial Probability Distribution

Definition 3.6
A binomial experiment possesses the following properties:
1) The experiment consists of a fixed number,𝑛𝑛, of identical trials.

2) Each trial results in one of two outcomes: success, 𝑆𝑆, or failure, 𝐹𝐹.

3) The probability of success on a single trial to some value 𝑝𝑝 and remain the same from trial to trial. The
probability of failure is equal to 𝑞𝑞 = (1 – 𝑝𝑝).

4) The trials are independent.

5) The random variable of interest is 𝑌𝑌, the number of successes observed during the 𝑛𝑛 trials.

Definition 3.7
A random variable Y is said to have a binomial distribution based on 𝑛𝑛 trials with success probability 𝑝𝑝 if and only if

𝑛𝑛
𝑝𝑝 𝑦𝑦 = 𝑦𝑦
𝑝𝑝 𝑦𝑦 𝑞𝑞𝑛𝑛−𝑦𝑦 𝑦𝑦 = 0, 1, 2, … , 𝑛𝑛 and 0 ≤ 𝑝𝑝 ≤ 1.

NOTATION:

𝑌𝑌~ 𝐵𝐵 (𝑛𝑛; 𝑝𝑝) or 𝑌𝑌~ 𝐵𝐵𝐵𝐵𝐵𝐵 (𝑛𝑛; 𝑝𝑝)

 The Binomial Probability Distribution

Example 3.7

Theorem 3.7
Let Y~ Bin (n; p), then
i. µ = 𝐸𝐸 𝑌𝑌 = 𝑛𝑛𝑛𝑛
2
ii. 𝜎𝜎 = 𝑉𝑉 𝑌𝑌 = 𝑛𝑛𝑛𝑛𝑛𝑛

NB: KNOW THE PROOF FOR THE ABOVE THEOREM

HINT: USE DEFINITIONS 3.4 AND 3.7

 The Geometric Probability Distribution

- Very similar to the binomial experiment

• Each trial results in a success or a failure
• Trials are independent

- The probability of success is = 𝑝𝑝 and is consistent from trial to trial.

- The geometric random variable 𝑌𝑌 is the number of the trial on which the first success occurs.

- The experiment consist of a series of trails that concludes with the 1st success.

Geometric distribution is a probability model and statistical data that is used to find out the number of failures which
occurs before single success.

Definition 3.8

A random variable 𝑌𝑌 is said to have a geometric probability distribution if and only if

𝑝𝑝 𝑦𝑦 = 𝑞𝑞 𝑦𝑦−1 𝑝𝑝 𝑦𝑦 = 1, 2, 3, … , 0 ≤ 𝑝𝑝 ≤ 1
 The Geometric Probability Distribution

Theorem 3.8

If 𝑌𝑌~ 𝐺𝐺𝐺𝐺𝐺𝐺 (𝑝𝑝) then

1
i. µ = 𝐸𝐸 𝑌𝑌 = and
𝑝𝑝

2 1−𝑝𝑝 𝑞𝑞
ii. 𝜎𝜎 = 𝑉𝑉 𝑌𝑌 =
𝑝𝑝2
=
𝑝𝑝2

NOTATION:
𝑌𝑌~ 𝐺𝐺𝐺𝐺𝐺𝐺 (𝑝𝑝)

Example 3.11
 The Poisson Probability Distribution

This can be applied in situations where the total number of outcomes is difficult or impossible to determine.

A Poisson experiment is a statistical experiment that has the following properties:

• Very similar to the binomial experiment

 Each trial results in a success or a failure

 Trials are independent

• The average number of successes (μ) that occurs in a specified region is known.

• The probability that a success will occur is proportional to the size of the region.

The Poisson probability distribution is an approximation of the binomial distribution for large n and small p,
using a single parameter 𝛌𝛌, which can be interpreted as the mean number of successes or 𝛌𝛌 = 𝐧𝐧𝐧𝐧.

Definition 3.11

A random variable Y is said to have a Poisson probability distribution,𝑌𝑌~ 𝑃𝑃𝑃𝑃𝑃𝑃 (𝜆𝜆) if and only if

𝜆𝜆𝑦𝑦 −𝜆𝜆
𝑃𝑃 𝑦𝑦 = 𝑒𝑒 , 𝑦𝑦 = 0, 1, 2, … , 𝜆𝜆 > 0.
𝑦𝑦!
 The Poisson Probability Distribution

Theorem 3.11

If 𝑌𝑌~ 𝑃𝑃𝑃𝑃𝑃𝑃 (𝜆𝜆), then

i. µ = 𝐸𝐸 𝑌𝑌 = 𝜆𝜆

ii. 𝜎𝜎 2 = 𝑉𝑉 𝑌𝑌 = 𝜆𝜆

NOTATION:
𝑌𝑌~ 𝑃𝑃𝑃𝑃𝑃𝑃 (𝜆𝜆)

Example 3.20
 Moments and Moment-Generating Functions

The probability distribution of a random variable may be uniquely described in terms of mgf. The mean and
variance, parameters, may then be obtained directly from this mgf (also called a “special expected value” – see
definition 3.12 & 3.13).

Definition 3.14

The moment-generating function m(t) for a random variable Y is defined to be:

𝑚𝑚 𝑡𝑡 = 𝐸𝐸(𝑒𝑒 𝑡𝑡𝑡𝑡 )

We say that a moment-generating function for 𝑌𝑌 exists if there exists a positive constant 𝑏𝑏 such that 𝑚𝑚(𝑡𝑡) is finite
for 𝑡𝑡 ≤ 𝑏𝑏.

Important Results-NB-Know them!!!!

𝑥𝑥 𝑟𝑟
𝑥𝑥
(a)𝑒𝑒 = ∑∞
𝑟𝑟=0 𝑟𝑟!

𝑥𝑥
(b)∑∞
𝑟𝑟=0 𝑥𝑥 𝑟𝑟
=
1−𝑥𝑥
if 𝑥𝑥 < 1
∞ 𝑛𝑛
(c)∑𝑥𝑥=0 𝑥𝑥 𝑎𝑎 𝑥𝑥 𝑏𝑏 𝑛𝑛−𝑥𝑥 = 𝑎𝑎 + 𝑏𝑏 𝑛𝑛

𝑥𝑥
(d) lim 1 + = 𝑒𝑒 𝑥𝑥
𝑛𝑛→∞ 𝑛𝑛

Example 3.23
 Tchebysheff’s Theorem

We use the above theorem in cases where the probability distribution is unknown and we want to find the
probability that a random variable Y of interest falls in an interval 𝜇𝜇 ± 𝑘𝑘𝑘𝑘.

Theorem 3.14

2
Let 𝑌𝑌 be a random variable with mean µ and infinite variable 𝜎𝜎 . Then, for any constant k > 0,

1 1
𝑃𝑃 𝑌𝑌 − µ < 𝑘𝑘𝑘𝑘 ≥ 1 − or P 𝑌𝑌 − µ ≥ 𝑘𝑘𝑘𝑘 ≤
𝑘𝑘 2 𝑘𝑘 2

NB: The results apply to any probability distribution.

Example 3.28