2.

Random Variables and Probability Distributions

2.1 The Concept of a Random Variable

De…nition (2.1): A random variable (r.v.) is a real valued function de…ned on the elements of a sample
space; i.e., if S is a sample space with probability measure and X is a real valued function de…ned over the
elements of S, then X is called a random variable.
Alternatively,

A random variable is a real valid function X : S ! R such that Ax = f! : X(!) xg is a member of

F (means P (Ax ) is de…ned).

Example (2.1): Toss a fair coin, S = fH; T g. De…ne X by

X(!) = 0; !=T

X(!) = 1; !=H

1
Consider, x = 2 and Ax = A1=2 , then A1=2 = f! : X(!) 1=2g = fT g

Example (2.2): Consider the experiment of tossing a single coin. Let the random variable X denote the
number of heads. Then S = fH; T g and X(!) = 1 if ! = H, and X(!) = 0 if ! = T .

Example (2.3): Consider an experiment of tossing two coins. Let the random variable X denote the number
of heads, and let the random variable Y denote the number of tails. Then S = fHH; HT; T H; T T g, and
X(!) = 2 if ! = HH, X(!) = 1 if ! = HT or T H, X(!) = 0 if ! = T T .

Similarly, Y (!) = 2 if ! = T T , Y (!) = 1 if ! = HT or T H, Y (!) = 0 if ! = HH

! X Y X +Y
HH 2 0 2
HT 1 1 2
TH 1 1 2
TT 0 2 2

We say that the space of the r.v. X = f0; 1; 2g.

Example (2.4): The sample space for tossing a die once is S = f1; 2; 3; 4; 5; 6g.

Let the r.v. X denote a number on the face that turns up in a sample point, then we can write

X(1) = 1; X(2) = 2; X(3) = 3; X(4) = 4; X(5) = 5; X(6) = 6:

Note that in this case X(!) = !: we call such a function an identity function.

1
Example (2.5): The sample space for tossing a coin until a head turns up is

S = fH; T H; T T H; T T T H; :::g.

Let the r.v X be the number of trials required to produce the …rst head, then we can write

X = f1; 2; 3; 4; :::g X(H) = 1; X(T H) = 2; X(T T H) = 3; :::::

In this case, the space or the range of the r.v. X = f1; 2; 3 : : :g

2.2 Discrete Random Variables and Their Probability Distributions

De…nition (2.2): The Cumulative distribution function (distribution function) of a random variable is

F : R ! [0; 1] such that

FX (x) = P (f! : X (!) xg)

= F (Ax )

= P (X x) informal usage but standard

Properties of Cumulative Distribution Function (CDF) or Distribution Function (DF):

i ). As x ! 1; FX (x) ! 0

Proof: As x ! 1; Ax ! ? but P (?) = 0

As x ! 1; FX (x) ! 0.

ii ). As x ! 1; FX (x) ! 1

Proof: As x ! 1; Ax ! S but P (S) = 1

As x ! 1; FX (x) ! 1.

iii). If x1 x2 ; then FX (x1 ) FX (x2 ).

Proof: x1 x2 ! Ax1 Ax2 ; so P (Ax1 ) P (Ax2 ) ; i.e. FX (x1 ) FX (x2 ).

iv ). lim FX (x + h) = FX (x) ; i.e., F is continuous from the right.

h!0
(h>0)

Discrete random variable: a random variable is discrete if its range is either …nite or countably in…nite (for
this example 5 is an example), i.e., 8! 2 S; X (!) 2 fx1 ; x2 ; x3 ; : : :g (possibly …nite or countably in…nite).
Probability distribution: if X is a discrete random variable, the function given by

fX (x) = P (X = x)

for each x within the range of X is called the probability mass function (or the probability distribution) of
X, i.e. 8
< P (X = x) ; x = x ; i = 1; 2; : : :
i
fX (x) =
: 0; x=6 xi ; i = 1; 2; : : :

2
 P
Note: In this case: FX (x) = fX (xi ) = P (X x).
fi:xi xg
Example (2.6): In the experiment of tossing an unbiased coin once we have the sample space, S = fH; T g
with P (H) = P (T ) = 0:5.

Let the r.v. X = number of heads in a sample point, then

8
< 1 ; x = 0; 1
2
fX (x) =
: 0; otherwise

Note: the common properties to all probability functions are

1). fX (x) 0
P
2). fX (x) = 1. This can also be presented as
x

x fX (x) = P (X = x)
0 0:5
1 0:5

Example (2.7): In an experiment of tossing an unbiased coin twice we have the sample space,

S = fHH; HT; T H; T T g

Let the r.v. X = number of heads in a sample point, s, then letting P (H) = p and P (T ) = 1 p ;we
can present this as follows:

x fX (x) = P (X = x)
2
0 (1 p) 0:25
1 2p (1 p) 0:50
2 p2 0:25

Example (2.8): In the experiment of ‡ipping a coin and generating the number of tosses required to …nd a
head, the sample space,
S = fH; T H; T T H; T T T H:::g

Let the r.v. X = the number of tosses required to produce the …rst head and let P (H) = p, P (T ) = 1 p,
then
x fX (x) = P (X = x)
1 p
2 (1 p) p
2
3 (1 p) p
3
4 (1 p) p
.. ..
. .

3
 The sum of the probability of this process must be equal to 1.

Proof: The probability function (distribution) for the above experiment is given by:
8
< p (1 p)x 1 ; x = 1; 2; 3; : : :
fX (x) = P (X = x) =
: 0; elsewhere

X 1
X 1
X
)
x 1
fX (x) = P (X = x) = p (1 p) = 1.
x x=1 x=1

Example (2.9): In the experiment of rolling a die we have the sample space,

S = f1; 2; 3; 4; 5; 6g.

Here, X(!) = !; P (!) = 1=6. Such distributions are known as uniform distributions.

x fX (x) = P (X = x) P (X x) = FX (x)
1 1
1 6 6
1 2
2 6 6
1 3
3 6 6
1 4
4 6 6
1 5
5 6 6
1
6 6 1

Probability mass function of the above uniform distribution is given by:

8
< 1 ; x = 1; 2; : : : ; 6
6
fX (x) =
: 0; elsewhere

FX (1:5) = 1=6, FX (1:9) = 1=6, implying that the probability of X between 1 and 2 , i.e., P (1 < X < 2),
is equal zero
F(x)

5/6

4/6

3/6

2/6

1/6

x
0 1 2 3 4 5 6

Graphically FX (x) is a step function with the height of the step at xi equal to fX (xi ).

Formally
X
FX (x) = P (X x) = fX (y)
y x

Let R = the space of the random variable X, then

R = fx1 ; x2 ; x3 ; :::; xk g; x1 < x2 < x3 < ::: < xk ;

4
 Then

FX (x) = 0 for x < x1

FX (x1 ) = fX (x1 )

FX (x) = fX (x1 ) for x1 x < x2

FX (x2 ) = fX (x1 ) + fX (x2 )

FX (x) = fX (x1 ) + fX (x2 ) for x2 x < x3

FX (x3 ) = fX (x1 ) + fX (x2 ) + fX (x3 ),

..
.

FX (x) = 1 when x xk

Properties of FX (x): The following …ve properties hold for the CDF

a). 0 FX (x) 1

b). FX (x) is non-decreasing

c). FX (x) = 0 for x < x1 , x1 being the minimum of the values of the random variable X.

d). FX (x) = 1 for x xn , xn being the largest value of X.

e). P (x < X x0 ) = P (X x0 ) P (X x) = FX (x0 ) FX (x)

Thus we can go from fX (x) to FX (x) or vice versa; i.e., given fX (x) we can derive FX (x) or given
FX (x) we can derive fX (x).

2.3 Continuous Random Variables and Their Probability Density Functions (pdf )

Continuous random variable: a random variable is continuous if the sample space of X is an interval or a
union of intervals (e.g. height, weight, and the time elapsing between two telephone calls)

Let x1 ; x2 ; :::; xn be the observations on a random variable, and

no of observation of xi x
Fn (x) = ¯
n

or the relative frequency for the event xi x.

For any given value of x as n in Fn increases to the limit, which we denote by FX (x), is referred to as the
probability that (X x). Thus, is called the distribution function of the random variable X.

De…nition (2.3): A random variable X is called continuous if there exists a function fX ( ) such that
Rx
P (X x) =FX (x) = 1 fX (u)du for every real number x, and FX (x) is called the distribution function
of the random variable X.

5
 Rx
De…nition (2.4): If X is a continuous random variable, the function fX ( ) in FX (x) = 1
fX (u)du is
called the probability density function of X.

Properties of continuous random variables

i ). FX (+1) = lim FX (x) = 1

x!+1

ii ). FX ( 1) = lim FX (x) = 0
x! 1

iii ). Properties 1 and 2 imply that 0 FX (x) 1.

iv ). FX (x) is non decreasing, i.e., if a > b then FX (a) FX (b) 0 which is P (x a) P (x b).

v). If FX (x) is continuous and di¤erentiable then

dFX (x)
= fX (x)
dx

Proof:

P (a < X < b) = P (X b) P (X a)

= FX (b) FX (a)

Consider the interval (x; x + x)

P (x < X < x + x) = FX (x + x) FX (x)

Supposing that x is very small so that fX (x) is constant over the range (x; x + x), then
Z x+ x
) P (x < X < x + x) = fX (u) du = fX (x) x
x
) FX (x + x) FX (x) = fX (x) x
FX (x + x) FX (x)
) lim = fX (x)
x!0 x
dFX (x)
) = fX (x)
dx

The function fX (x) is known as the probability density function of the random variable X and measures
the density of the function at a point. As x ! 0; x + x ! x; and P (x < X < x + x) ! 0.

dFX (x) Rx
vi ). dx = fX (x) ) FX (x) FX ( 1) = 1
fX (u) du

vii ). Since FX (x) is non-decreasing it follows that

a). fX (x) 0
Z +1
b). fX (x) dx = 1
1

viii ). Note, however, that fX (a) 6= P (X = a), and fX (a) could actually be greater than 1. For a continuous
random variable X, P (X = a) = 0; P (X = a) = lim a!0 P (a X <a+ a).

6
 2.4 The Expected Value of a Random Variable and Moments

Mathematical Expectation:

1). Let X be a discrete random variable taking values x1 ; x2 ; x3 ; : : :with fX (x) as its P D, then the expected
value of X, denoted by E(X), is de…ned as
X
E(X) = xi fX (xi )
i

2). Let X be a continuous random variable with probability density function fX (x), then the expected
value of X, denoted by E(X), is de…ned as .
Z 1
E(X) = xfX (x) dx
1

Suppose we have an empirical frequency distribution given by

Mid value Frequency

x1 f1
x2 f2
x3 f3
.. ..
. .
X
Sample mean = x = x1 f1 =N + x2 f2 =N + :::; N = fi .
i

Probability = lim of the relative frequency, e.g. fX (x1 ) = lim f1 =N , etc.

X
E(X) = xi fX (xi )
i

E(X) is also denoted by .

Bernoulli Random Variable: A random variable with only two outcomes (0 and 1) is known as a Bernoulli
r.v.
Example (2.10): Let X be a random variable with probability p of success and (1 p) of failure:

x fX (x)
Failure 0 1 p
Success 1 p

E(X) = 0(1 p) + 1(p) = p:

The above tabular expression for the probability of a Bernoulli r.v. can be written as

8
< px (1 p)
1 x
; if x = 0; 1
fX (x) =
: 0; otherwise

7
Example (2.11): Let X be the number of trials required to produce the 1st success, say a head in a toss of
a fair coin. This is easily described by a geometric random variable and is given as

x fX (x)
1 p
2 (1 p) p
2
3 (1 p) p
3
4 (1 p) p
.. ..
. .

2 3
E (X) = 1 p+2 (1 p) p + 3 (1 p) p + 4 (1 p) p +
h i
2 3
= p 1 + 2 (1 p) + 3 (1 p) + 4 (1 p) +

Let

2 3
S = 1 + 2 (1 p) + 3 (1 p) + 4 (1 p) + ; then
2 3 4
(1 p) S = (1 p) + 2 (1 p) + 3 (1 p) + 4 (1 p) + ;

2 3
S (1 p) S = 1 + (1 p) + (1 p) + (1 p) +
1
pS =
p
1
S =
p2
1 1
) E (X) = p =
p2 p

Alternatively, 8
< p (1 p)x 1
; if x = 1; 2; 3; : : :
fX (x) = P (X = x) =
: 0; otherwise

1
X 1
X 1
X
x 1 x 1
E (X) = xfX (x) = xp (1 p) =p x (1 p)
x=1 x=1 x=1
X1
d x d x x 1
= p [(1 p) ] ; as [(1 p) ] = x (1 p)
x=1
dp dp
" 1
# " 1
#
d X x d X x
= p (1 p) =p (1 p)
dp x=1 dp x=0
d 1 1 1
= p =p 2
=
dp p p p

1
Note: if p = 2 then E(x) = 2; if p = 1=3 then E(x) = 3; if p = 1=4 then E(x) = 4 if p = 1=5 then E(x) = 5;
if p = 1=10 then E(x) = 10.

8
 As p becomes smaller the number of trials required to get a success increases.

Expectation of a Function of a Random Variable

Let g(x) be a function of X where X is a discrete random variable then

X
E(g(x)) = g(x)fX (x)
x

Example (2.12): Let g(X) = X 2 , then

X
E(g(X)) = E X 2 = x2 fX (x)
x

2
Variance of random variable X, denoted by , is de…ned as

2
X
2
= E (x ) = (x )2 fX (x) where = E(X)

Let g(x) be a function of X where X is a continuous random variable then

Z 1
E [g(X)] = g(x)fX (x)dx
1

Properties of mathematical expectations

a). if c is a constant E(c) = c; i.e. E(c) = (c)1 = c

b). E(aX + b) = aE(X) + b, where a and b are constants in R

X X
c). E(cg(X)) = cE(g(X)) i.e. E(cg(X)) = cg(x)fX (x) = c g(x)fX (x) = cE(g(X)).

Moments of a probability distribution

The mean of a distribution is the expected value of the random variable X. A generalization of this is to raise
X to any power greater than 1, i.e., set m = 2; 3; ::: and compute the E(X m ). This is known as the moment
of order m about the origin. The mth moment about the origin, denoted by 0
m, of a random variable X is
given by:
X
0
m = E(X m ) = xm fX (x); m = 1; 2; 3; : : : (Discrete)
Z 1
0
m = E(X m ) = xm fX (x); m = 1; 2; 3; : : : (Continuous)
1

Moments can also be generated around the mean, which are known as central moments or moments around
the mean. They are de…ned as:

m
X m
m = E [(X ) ]= (x ) fX (x); m = 1; 2; 3; : : : or
Z 1
m m
m = E [(X ) ]= (x ) fX (x); m = 1; 2; 3; : : :
1

Propositions:

9
 a). 0 = 1.

b). 1 = 0.

Proof:

Proof of proposition (a) is trivial, while proof of (b) goes as follows:

h i
1
1 = E (X ) = E (X ) = E (X) =0

De…nition (2.5):
h i
2 2
a). 2 = E (X ) = is de…ned as the variance of the random variable, and is also denoted by
V ar(X) or V (X).
h i
2
2 = E (X ) = E X2 2X + 2

= E X2 2E (X) + 2
= E X2 2 2
+ 2

= E X2 2

0 0 2
2 = 2 ( 1) = E X2 2

Thus, i.e., the variance of a random variable is expected value of the square of the random variable less
the square of the expected value of the random variable.
h i
3
b). 3 = E (X )
h i
3
E (X ) = E X3 3X 2 + 3X 2 3

= E X3 3E X 2 + 3E (X) 2 3

0 0 3 3
= 3 3 2 +3
0 0 3
= 3 3 2 +2

0 0 3
is the third moment about the mean and is equal to : 3 = 3 3 2 +2 .
h i
4
c). 4 = E (X )
h i
4
E (X ) = E X4 4X 3 + 6X 2 2
4X 3
+ 4

= E X4 4E X 3 + 6E X 2 2
4E (X) 3
+ 4

0 0 0 2 4 4
= 4 4 3 +6 2 4 +
0 0 0 2 4
= 4 4 3 +6 2 3

0 0 0 2 4
is the fourth moment about the mean and is equal to : 4 = 4 4 3 +6 2 3 .

Interpretations

10
 0
1). 1 = is a measure of central tendency

2
2). 2 which is denoted by , V ar(X), or V (X), is the known as the variance, and is a measure of
2
dispersion of the random variable. If X is a random variable given in centimeters, ’s dimension is
centimeters squared cm 2 .

3). , or, the positive root of the variance, is called the standard deviation of the random variable and its
dimension is given in centimeters. Thus, to convert these measures of dispersion into a dimensionless
measure, we divide it by the mean to get the coe¢ cient of variation given as cv = . This is a
dimensionless measure of dispersion of the random variable.

4). 3 is the third moment about the mean and is used to calculate the measure of skewness which is
given as 3 = 3
3 and known as the Pearson’s measure of skewness. If 3 = 0 then the distribution is
symmetric. If 3 > 0 then the distribution is positively skewed, and there is a spread to the right–few
observations on the right-hand of the mean pull the mean to the right. If 3 < 0 then the distribution
is negatively skewed, and there is a spread to the left–few observations on the left hand of the mean
pull the mean to the left.

5). 4, the fourth moment about the mean and is used to calculate the measure of peakedness or ‡atness
(which is known as kurtosis) and is given as . 4 = 3 for a normal distribution. 4 > 3 if the
distribution is narrower and thinner at its tails than the normal distribution (it is known as leptokurtic).
4 < 3 if the distribution is ‡atter and thicker at its tails than the normal distribution (it is known as
platykurtic).

Example (2.13): We have already obtained the expected value of the Bernoulli random variable to be equal
to E(x) = 0(1 p) + 1(p) = p. To obtain the variance of the Bernoulli random variable we …rst get

E(X 2 ) = 02 (1 p) + 12 (p) = p:

Thus
2 2
= E(X 2 ) [E(X)] = p p2 = p(1 p)

Thus p
p p(1 p)
= p(1 p) and cv =
p
Exercise (2.1): Let the random variable Y be the number of failures preceding a success in an experiment
2
of tossing a fair coin, where success is obtaining a head. Find E(Y ) and for this random variable.

Moment Generating Function (MGF.)

De…nition (2.6): Moment generating function of a random variable X, written as MX ( ), is de…ned as

x
MX ( ) = E e , it generates moments.

11
 Taylor series expansion about a point x0 of a continuously di¤erentiable function f (x) is given by:
1
X f (k) (x0 ) k dk f (x)
f (x) = f (x0 ) + (x x0 ) ; where f (k) (x0 ) =
k! dxk x=x0
k=1

Taylor series expansion about the origin or zero of a continuously di¤erentiable function f (x) is given
by:
1
X f (k) (0)
f (x) = f (0) + xk
k!
k=1

Example (2.14): Let f (x) = ex , then

x2 x3 x4
f (x) = 1 + x + + + +
2! 3! 4!

Similarly,
2 3 4
x ( x) ( x) ( x)
e =1+ x+ + + +
2! 3! 4!
Hence,
2 3 4 5
X
MX ( ) = E e = 1 + E (X) + E X2 + E X3 + E X4 + E X5 +
2! 3! 4! 5!
2 3 4
d
MX ( ) = E (X) + E X 2 + E X 3 + E X 4 + E X 5 +
d 2! 3! 4!
d d
) MX ( ) = MX (0) = E (X) ; the …rst moment about the origin.
d =0 d
Similarly,
2 3
d2 2
2 MX ( ) = E X + E X3 + E X4 + E X5 +
d 2! 3!
d2 d2
) MX ( ) = MX (0) = E X 2 ; the second moment about the origin.
d 2 =0 d 2
Note:

1). Taking higher order derivative of the moment generating function and then evaluating the resulting
function at the origin ( = 0) can also generate all higher order moments about the origin of a random
variable.

2). There is a one-to-one correspondence between the probability density function of a random variable
and its moment generating function, it exists.

12