Prepared by: Mohammad Saifuddin

Random variables and their probability distributions

Random Variable: A variable whose values are determined by the outcomes of a random experiment. In other
word, if a variable has a probability distribution then it is called a random variable. Random variables can be either
discrete or continuous.

Example (discrete case): Suppose that two coins are tossed so that the sample space is S = {HH, HT, TH, TT}.
Suppose X represent the number of heads which can come up, with each sample point we can associate a number
for X as shown in the table below:
Sample point HH HT TH TT
X 2 1 1 0

Thus the random variable X takes the values 0, 1, 2 for this random experiment. The above example takes only a finite
number of values and for each random value we can associate a probability as shown in the table.

Usually, for each value of random variable X, the probability of respective probability is denoted by f(x).

X 0 1 2
P[X=x]=f(x) 1/4 2/4 ¼

Observe that the sum of the probabilities of all the random variable is equal to one. i.e.,
f(0) + f(1) + f(2) =1/4+2/4+1/4=1.
Thus the probability distribution for a random variable provides a probability for each possible value and that these
probabilities must sum to 1.

Attention!!!!!!!!!!!!!
Be careful about following NOTATION in all over the lectures about probability distribution of a random
variable!!!
✓ X (Capital)= Random variable
✓ x (Little)= specified value of random variable X

Discrete random variable

A random variable is a discrete random variable that can take on only certain/isolated values along an interval, with
the possible values having gaps between them. In most statistical situations, discrete random variables produce values
that are nonnegative whole numbers.
Example:
1. Determining the number of defects in a batch of 50 items
2. Counting the number of people who arrive at a store during a five-minute period
3. Sampling 100 registered voters and determining how many voted for the president in the last election

Continuous random variable

Continuous random variables take on values at every point over a given interval.
Example:
1. Measuring the time between customer arrivals at a retail outlet
2. Measuring the lengths of newly designed automobiles
3. Measuring the weight of grain in a grain elevator at different points of time

N.B: Discrete r.v is “Counted” and continuous r.v is “Measured”

Page | 1
 Prepared by: Mohammad Saifuddin

Probability Distributions

A. Discrete probability distribution

The set of ordered pairs (x, f(x)) is a probability function, probability mass function, or probability distribution
of the discrete random variable X if, for each possible outcome x,

1. For any value of x, 𝑓(𝑥) ≥ 0.

2. ∑ 𝑓(𝑥) = 1 ; ∀ 𝑥
3. 𝑃(𝑋 = 𝑥) = 𝑓(𝑥)

This function f (x) is called the probability function (pf) or probability mass function(pmf) of the discrete random
variable X.

Example-1: A shipment of 20 similar laptop computers to a retail outlet contains 3 that are defective. If a
school makes a random purchase of 2 of these computers, find the probability distribution for the number of
defectives. [Walpole 9th ed. Example 3.8]

Example-2: The following table gives the probabilities that a certain computer will malfunction 0, 1, 2,3,4,5
or 6 times on any day:

Number of malfunctions: x 0 1 2 3 4 5 6
Probability: f(x) 0.17 f(1)= 0.27 0.16 0.07 0.03 0.01
0.29

𝑃(𝑋 ≤ 2) = ∑ 𝑓(𝑥) = 𝑓(0) + 𝑓(1) + 𝑓(2) = 0.73

0
1

𝑃(𝑋 < 2) = ∑ 𝑓(𝑥)

𝑥=0

𝑃(𝑋 ≥ 2) = ∑ 𝑓(𝑥) = 𝑓(2) + ⋯ + 𝑓(6)

𝑥=2
Calculate the following probabilities:
(𝑖)𝑃(𝑋 = 1) (𝑖𝑖)𝑃(𝑋 ≤ 2) (𝑖𝑖𝑖) 𝑃(𝑋 < 2)(𝑖𝑣) 𝑃(𝑋 ≥ 2)(𝑣)𝑃(𝑋 > 2)(𝑣𝑖)𝑃(4 ≤ 𝑋 ≤ 6)
Solution: Try yourself

Cumulative distribution functions (cdf) of Discrete r.v:

If a random variable is discrete in general, its distribution will also be discrete. For a discrete random variable X, the
distribution function or cumulative distribution is given by F(x) and is written as
(i) 𝐹(𝑥) = 𝑃(𝑋 ≤ 𝑥) = ∑𝑡≤𝑥 𝑓(𝑡)
(ii) 0 ≤ 𝐹(𝑥) ≤ 1
Example-3: The following table gives the probabilities that a certain computer will malfunction 0, 1, 2,3,4,5
or 6 times on any day:

Number of malfunctions: X 0 1 2 3 4 5 6
Probability: f(x) 0.17 0.29 0.27 0.16 0.07 0.03 0.01
F(x) F(0)=f(0)=0.17 F(1)= ? ? ? ? F(6)=1

Page | 2
 Prepared by: Mohammad Saifuddin

𝐹(1) = ∑ 𝑓(𝑥)
𝑥=0
N.B: (𝑖) 𝑓(𝑥) = 𝐹(𝑥) − 𝐹(𝑥 − 1)
(𝑖𝑖)𝑃(𝑋 > 𝑥) = 1 − 𝑃(𝑋 ≤ 𝑥) = 1 − 𝐹(𝑥)

B. Continuous Probability distribution

Probability Density Function (pdf) of Continuous r.v: A function f(x) is said to be the probability density function
(pdf) of a continuous random variable X if it satisfies the following properties.
(i) 𝑓(𝑥) ≥ 0
∞
(ii) ∫−∞ 𝑓(𝑥)𝑑𝑥 = 1
𝑏
(iii) 𝑃(𝑎 ≤ 𝑋 ≤ 𝑏) = 𝑃(𝑎 < 𝑋 < 𝑏) = ∫𝑎 𝑓(𝑥)𝑑𝑥 = 𝑎𝑟𝑒𝑎 𝑢𝑛𝑑𝑒𝑟 𝒇(𝒙)𝑓𝑟𝑜𝑚 𝒂 𝑡𝑜 𝒃

N.B:𝑃(𝑋 = 𝑥) = 0 𝑓𝑜𝑟 𝑐𝑜𝑛𝑡𝑖𝑛𝑢𝑜𝑢𝑠 𝑑𝑖𝑠𝑡𝑟𝑖𝑏𝑢𝑡𝑖𝑜𝑛 (𝑊ℎ𝑦? )

Cumulative Distribution Function (cdf) Continuous r.v: The cumulative distribution function of a continuous
random variable X is:
𝑥

𝐹(𝑥) = 𝑃(𝑋 ≤ 𝑥) = ∫ 𝑓(𝑥)𝑑𝑥

−∞
N.B: (𝑖) 𝑃(𝑎 ≤ 𝑋 ≤ 𝑏) = 𝐹(𝑏) − 𝐹(𝑎)
(𝑖𝑖)𝑃(𝑋 > 𝑥) = 1 − 𝑃(𝑋 ≤ 𝑥) = 1 − 𝐹(𝑥)

Example-1: Suppose that the error in the reaction temperature, in ◦C, for a controlled laboratory experiment is a
continuous random variable X having the probability density function
𝑥2
𝑓(𝑥) = { 3 , −1 < 𝑥 < 2
0, 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒
(a) Verify that f(x) is a density function.
1
(b) Find P (0 < X ≤ 1)=∫0 𝑓(𝑥)𝑑𝑥 =? ? ?.

Example-2: For the density function of Example-1, find F (x), and use it to evaluate P (0 < X ≤ 1).

Self-reading: What Are Important Characteristics of Probability Distributions and Where Do They Come From?
[Walpole, 9th edition, pg-104]

1
𝐹(𝑥) = (𝑥 3 + 1); −1 < 𝑥 < 2
9
1
𝐹(2) = (8 + 1) = 1
9

Page | 3
 Prepared by: Mohammad Saifuddin

𝟐 𝟏 𝟏
𝑷 (𝟎 < 𝑿 ≤ 𝟏) = 𝑭(𝟏) − 𝑭(𝟎) = − =
𝟗 𝟗 𝟗

Exercise

1. Check whether the following can define probability distributions and explain your answers:
𝑥
(i) 𝑓(𝑥) = ; 𝑥 = 0,1,2,3,4,5
15

𝑓(𝑥) ≥ 0
𝑥
∑5𝑥=0 𝑓(𝑥) = 1 , so 𝑓(𝑥) = is a pmf.
15

5−𝑥 2
(ii) 𝑓(𝑥) = ; 𝑥 = 0,1,2,3
6
(iii)
2. Determine the value k so that the following function can serve as a probability
distribution of the discrete random variable X:
𝑓(𝑥) = 𝑘(𝑥 2 + 3), 𝑓𝑜𝑟 𝑥 = 0, 1, 2, 3.
Solution: If f(x) is a pmf then, ∑3𝑥=0 𝑓(𝑥) = 1, 𝑜𝑟, ∑30 𝑘(𝑥 2 + 3) = 1

3. [Walpole] Consider the density function

𝑘 𝑥 ; 𝑓𝑜𝑟 0 < 𝑥 < 1
𝑓(𝑥) = { √
0; 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒
(a) Evaluate k.
(b) Find F(x) and use it to evaluate P (0.3 < X < 0.6).
1 1
Solution : Since , f(x) is a pdf, so, ∫0 𝑓(𝑥)𝑑𝑥 = 1, 𝑜𝑟, ∫0 𝑘√𝑥𝑑𝑥 = 1

3
𝑘=
2
𝑥 13 𝑥3
b) 𝐹(𝑥) = 𝑃(𝑋 ≤ 𝑥) = ∫0 𝑓(𝑥) 𝑑𝑥 = ∫0 √𝑥 𝑑𝑥 =
2 2

P (0.3 < X < 0.6)=F(0.6)-F(0.3)=do…

4. [Walpole] Measurements of scientific systems are always subject to variation, some more than others. There
are many structures for measurement error, and statisticians spend a great deal of time modeling these errors.
Suppose the measurement error X of a certain physical quantity is decided by the density function
𝑘(3 − 𝑥 2 ) ; 𝑓𝑜𝑟 − 1 < 𝑥 < 1
𝑓(𝑥) = {
0; 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒
(a) Determine k that renders f(x) a valid density function.
(b) Find the probability that a random error in measurement is less than 1/2.
1 1/2
Solution:𝑃 (𝑋 < ) = ∫−1 𝑓(𝑥)𝑑𝑥
2
(c) For this particular measurement, it is undesirable if the magnitude of the error (i.e., |x|) exceeds 0.8.
What is the probability that this occurs?

𝑥
Solution-c) First, find 𝐹(𝑥) = ∫−1 𝑓(𝑥) 𝑑𝑥 =

Page | 4
 Prepared by: Mohammad Saifuddin

Now, |X|>0.8

⇒ 𝑿 > 𝟎. 𝟖 𝒐𝒓, −𝑿 > 𝟎. 𝟖

⇒ 𝑿 > 𝟎. 𝟖 𝒐𝒓, 𝑿 < −𝟎. 𝟖

Hence, P(|X|>0.8)=P(X>0.8) +P(X<-0.8)=1-F(0.8)+F(-0.8)=do your self

Mathematical Expectation

The expected value or mathematical expectation of a random variable X is the weighted average of the values that X
can assume with probabilities of its various values as weights.
Thus, the expected value of a random variable is obtained by considering the various values that the variable can take
multiplying these by their corresponding probabilities and summing these products. Expectation of X is denoted by
E(X).

Expectation of a discrete random variable :

Let X= {x1, x2, ...xn} be a discrete random variable with probability mass function f(x). Then the

(a) Population Mean (𝝁) or Expected Value or mathematical expectation of X is given by

𝜇 = 𝐸(𝑋) = ∑ 𝑥. 𝑓(𝑥)
𝑎𝑙𝑙 𝑥

(b)The variance of X, denoted as σ2.

𝝈𝟐 = 𝑉(𝑋) = 𝐸[𝑋 − 𝜇]2 = ∑ (𝑥 − 𝜇)2 . 𝑓(𝑥)

𝑎𝑙𝑙 𝑥

Variance of X can be also written as,

𝝈𝟐 = 𝐸(𝑋 2 ) − 𝜇 2 =

∑ 𝑥 2 . 𝑓(𝑥) − 𝜇2
𝑎𝑙𝑙 𝑥

The standard deviation of X is 𝜎 = √𝜎 2

Example: The expected and variance number of ears affected by ear infection during the first
two years of life is computed as follows:
X 𝑃(𝑋 = 𝑥) 𝑥. 𝑓(𝑥) (𝑥 − 𝜇)2 . 𝑓(𝑥)
= 𝑓(𝑥)
0 0.13 0 (0 − 1.26)2 (0.13)
1 0.48 0.48 (1 − 1.26)2 (0.48)
2 0.39 0.78 (2 − 1.26)2 (0.39)
Total 𝐸(𝑥) = 1.26 𝑉𝑎𝑟(𝑥) = 0.452
Interpretation:
• The mean number of ears affected by otitis media during the first two years of life is
1.26.
• The population standard deviation of X is 𝜎 = √𝜎 2 = √0.152 = 0.673

Page | 5
 Prepared by: Mohammad Saifuddin

Expectation of a continuous random variable :

Let X be a continuous random variable with probability density function f(x), then the mathematical expectation of x
is defined as
∞
𝝁 =E(X) = ∫−∞ 𝑥𝑓 (𝑥)𝑑𝑥 ; provided the integral exists.

The variance of X, denoted as σ2 = V(X) is

∞
𝑉(𝑋) = 𝜎 2 = 𝐸[𝑋 − 𝜇]2 = ∫−∞(𝑥 − 𝜇)2 𝑓 (𝑥)𝑑𝑥 … … … … … … … . . (5). This is equivalent to,

∞
𝑉(𝑋) = 𝜎 2 = 𝐸[𝑋 2 ] − 𝜇 2 = ∫ 𝑥 2 . 𝑓 (𝑥)𝑑𝑥 − 𝜇 2 … … … … … … … . (6)
−∞
Remark:
If g(x) is function of a random variable and E [g(x)] exists, then E[(g(x)] =∫ 𝑔(𝑥). 𝑓(𝑥)𝑑𝑥

Properties of E(.): If 𝛼, 𝛽 are constants, then:

• 𝐸(𝛼) = 𝛼
• 𝐸(𝛼𝑋) = 𝛼. 𝐸(𝑋)
• 𝐸(𝛼𝑋 + 𝛽) = 𝛼. 𝐸(𝑋) + 𝛽
• 𝐸(𝑎𝑋 + 𝑏𝑌) = 𝑎𝐸(𝑋) + 𝑏𝐸(𝑌)

Properties of V(.):
• 𝑉𝑎𝑟(𝑋) ≥ 0
• 𝑽(𝜶𝑿 + 𝜷) = 𝜶𝟐 𝑽(𝑿)

𝑽𝒂𝒓(𝟓𝑿 + 𝟐𝟑𝟏) = 𝟓𝟐 𝑽𝒂𝒓(𝑿) + 𝟎 = 𝟐𝟓 ∗ 𝟑 = 𝟕𝟓

• 𝐼𝑓 𝑋 𝑎𝑛𝑑 𝑌 𝑎𝑟𝑒 𝒊𝒏𝒅𝒆𝒑𝒆𝒏𝒅𝒆𝒏𝒕, 𝑡ℎ𝑒𝑛

𝑉𝑎𝑟(𝑎𝑋 + 𝑏𝑌) = 𝑎2 𝑉𝑎𝑟(𝑋) + 𝑏 2 𝑉𝑎𝑟(𝑌)

Covariance and correlation

Expectation, variance, and standard deviation characterize the distribution of a single random variable. Now
we introduce measures of association of two random variables.

Covariance between two random variables X and Y is:

𝐶𝑜𝑣(𝑋, 𝑌) = 𝜎𝑋𝑌 = 𝐸[(𝑋 − 𝜇𝑋 )(𝑌 − 𝜇𝑌 )]
It summarizes interrelation of two random variables.

Correlation coefficient between variables X and Y is defined as:

𝐶𝑜𝑣(𝑋, 𝑌) 𝜎𝑋𝑌
𝜌= = ; −1 ≤ 𝜌 ≤ 1
√𝑉𝑎𝑟(𝑋)𝑉𝑎𝑟(𝑌) 𝜎𝑋 𝜎𝑌
For details see book [Baron, 2nd ed., page 51].

Exercise on Mathematical Expectation

1. The following table gives the probabilities that a certain computer will malfunction 0,1,2,3,4,5 or 6
times on any day:
Number of malfunctions: x 0 1 2 3 4 5 6
Probability: f(x) 0.17 0.29 0.27 0.16 0.07 0.03 0.01

Page | 6
 Prepared by: Mohammad Saifuddin

(a) Calculate the mean of this probability distribution;

(b) Calculate the standard deviation of this probability distribution

2. Consider the following probability distribution for discrete r.v X:

𝒙 0 1 2 3 4
𝒇(𝒙) 0.33 0.25 k 0.05 0.01

Find the value of k. Then find E(X) and SD(X)

3. Find the mean, variance and standard deviation of the uniform probability distribution given by: 𝑓(𝑥) =
1
; 𝑥 = 1,2,3, … … … … , 𝑛.
𝑛

4. If the probability density of a r.v is given by

2𝑥 ; 𝑓𝑜𝑟 0 < 𝑥 < 1
𝑓(𝑥) = {
0; 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒
Then evaluate E(X), Var(X) and Sd(X).
5. [Application-Walpole 9th ed. Exercise 4.35] The random variable X, representing the number of errors
per 100 lines of software code, has the following probability distribution:

𝒙 2 3 4 5 6
𝒇(𝒙) 0.01 0.25 0.40 0.30 0.04
Find average number of errors and standard deviation of error.

6. [Application-Walpole 9th ed. Exercise 4.85] Suppose it is known that the life X of a particular
compressor, in hours, has the density function
1 −𝑥
𝑓(𝑥) = {900 𝑒
900 ; 𝑓𝑜𝑟 𝑥 > 0

0; 𝑒𝑙𝑠𝑒𝑤ℎ𝑒𝑟𝑒
(a) Find the mean life of the compressor.
(b) Find E(X2).
(c) Find the variance and standard deviation of the random variable X. [Hint: Use Gamma function]

N.B: Gamma function:

∞
(𝑖) Γ(𝑛) = ∫ 𝑒 −𝑥 𝑥 𝑛−1 𝑑𝑥
0
(𝑖𝑖)Γ(𝑛) = (𝑛 − 1)!

Page | 7