NPTEL- Probability and Distributions

MODULE 7
LIMITING DISTRIBUTIONS
LECTURE 38
Topics
7.1 CONVERGENCE IN DISTRIBUTION AND
PROBABILITY
Theorem 1.1
𝑑
Let 𝑋𝑛 𝑛≥1 be a sequence of random variables such that 𝑋𝑛 𝑋, as 𝑛 ∞, for some
random variable 𝑋. Let 𝐹𝑛 and 𝐹 denote the d.f.s of 𝑋𝑛 𝑛 = 1, 2, … and 𝑋, respectively.
Then

lim 𝐹𝑛 𝑥 − = 𝐹 𝑥 − = 𝐹 𝑥 = lim 𝐹𝑛 𝑥 , ∀𝑥 ∈ 𝐶𝐹 ,
𝑛 ∞ 𝑛 ∞

where 𝐶𝐹 is the set of continuity points of 𝐹.

Proof. We are given that

𝑑
lim 𝐹𝑛 𝑥 = 𝐹 𝑥 , ∀𝑥 ∈ 𝐶𝐹 since 𝑋𝑛 𝑋, as 𝑛 ∞ .
𝑛 ∞

Moreover 𝐹 𝑥 − = 𝐹 𝑥 , ∀𝑥 ∈ 𝐶𝐹 .Thus it suffices to show that lim𝑛 ∞ 𝐹𝑛 𝑥 − =

𝐹 𝑥 − , ∀𝑥 ∈ 𝐶𝐹 . Let 𝑑 ∈ 𝐶𝐹 so that 𝐹 𝑑 − = 𝐹 𝑑 . Fix 𝑚 ∈ ℕ = 1, 2, … . Since the
1
set 𝐶𝐹𝑐 = ℝ − 𝐶𝐹 of discontinuity points of 𝐹 is countable and the interval 𝑑 − 𝑚 , 𝑑 is
1
uncountable there exists a 𝑑𝑚 ∈ 𝑑 − 𝑚 , 𝑑 ⋂𝐶𝐹 . Then we have lim𝑛 ∞ 𝐹𝑛 𝑑𝑚 =
𝐹 𝑑𝑚 and lim𝑛 ∞ 𝐹𝑛 𝑑 = 𝐹 𝑑 . Moreover

𝐹𝑛 𝑑𝑚 ≤ 𝐹𝑛 𝑑 − ≤ 𝐹𝑛 𝑑 , 𝑛 = 1, 2, …

⇒ lim 𝐹𝑛 𝑑𝑚 ≤ lim 𝐹𝑛 𝑑 − ≤ lim 𝐹𝑛 𝑑

𝑛 ∞ 𝑛 ∞ 𝑛 ∞

⇒ 𝐹 𝑑𝑚 ≤ lim 𝐹𝑛 𝑑 − ≤ 𝐹 𝑑 = 𝐹 𝑑 − . (1.1)
𝑛 ∞
1
Since 𝑑𝑚 ∈ 𝑑 − 𝑚 , 𝑑 , we have

lim 𝐹 𝑑𝑚 = 𝐹 𝑑 − = 𝐹 𝑑 . (1.2)
𝑚 ∞

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 1

NPTEL- Probability and Distributions

On taking 𝑚 ∞ in (1.1) we get

lim 𝐹 𝑑𝑚 ≤ lim 𝐹𝑛 𝑑 − ≤ 𝐹 𝑑 −
𝑚 ∞ 𝑚 ∞

⇒ 𝐹 𝑑 − ≤ lim 𝐹𝑛 𝑑 − ≤ 𝐹 𝑑 − (using (1.2))

𝑛 ∞

⇒ lim 𝐹𝑛 𝑑 − = 𝐹 𝑑 − ∙ ▄
𝑛 ∞
Corollary 1.1

Let 𝑋𝑛 𝑛≥1 be a sequence of random variables with corresponding sequence of d.f.s as

𝐹𝑛 𝑛≥1 . Further let 𝑋 be another random variable having the d.f. 𝐹.
𝑑
(i) If 𝑋𝑛 𝑋, as 𝑛 ∞ , and 𝑋 is of continuous type then lim𝑛 ∞ 𝐹𝑛 𝑥 =
𝐹 𝑥 , ∀𝑥 ∈ ℝ and lim𝑛 ∞ 𝐹𝑛 𝑥 − = 𝐹 𝑥 − , ∀𝑥 ∈ ℝ.
𝑑
(ii) Suppose that 𝑃 𝑋𝑛 ∈ 0, 1, 2, … = 𝑃 𝑋 ∈ 0, 1, 2, … = 1 and 𝑋𝑛 𝑋,
as 𝑛 ∞ . Then lim𝑛 ∞ 𝐹𝑛 𝑥 = 𝐹 𝑥 , ∀𝑥 ∈ ℝ and lim𝑛 ∞ 𝐹𝑛 𝑥 − =
𝐹 𝑥 − , ∀𝑥 ∈ ℝ.
(iii) Under the assumptions of (ii), let 𝑓 and 𝑓𝑛 be the p.m.f.s of 𝑋 and 𝑋𝑛 ,
respectively, 𝑛 = 1, 2, …. Then
𝑑
𝑋𝑛 𝑋, as 𝑛 ∞ ⇔ lim 𝑓𝑛 𝑥 = 𝑓 𝑥 , ∀𝑥 ∈ 0, 1, 2, … .
𝑛 ∞

Proof.

(i) Since 𝑋 is of continuous type we have 𝐶𝐹 = ℝ, where 𝐶𝐹 is the set of continuity

points of 𝐹. The assertion now follows from Theorem 1.1.

(ii) Fix 𝑥 ∈ ℝ. If 𝑃 𝑋 = 𝑥 = 0 then 𝑥 ∈ 𝐶𝐹 and, therefore, by Theorem 1.1.

lim 𝐹𝑛 𝑥 = 𝐹 𝑥 , and lim 𝐹𝑛 𝑥 − = 𝐹 𝑥 − .

𝑛 ∞ 𝑛 ∞

Now suppose that 𝑃 𝑋 = 𝑥 > 0 . Then 𝑥 ∈ 0, 1, 2, … and 𝑃 𝑋 = 𝑥 +

0.5 = 𝑃 𝑋 = 𝑥 − 0.5 = 0. Consequently 𝑥 ± 0.5 ∈ 𝐶𝐹 ,

𝐹𝑛 𝑥 = 𝐹𝑛 𝑥 + 0.5 and 𝐹𝑛 𝑥 − = 𝐹𝑛 𝑥 − 0.5 , 𝑛 = 1,2, …

⇒ lim 𝐹𝑛 𝑥 = 𝐹 𝑥 + 0.5 = 𝐹 𝑥 and lim 𝐹𝑛 𝑥 − = 𝐹 𝑥 − 0.5 = 𝐹 𝑥 − .

𝑛 ∞ 𝑛 ∞

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 2

NPTEL- Probability and Distributions

It follows that

lim 𝐹𝑛 𝑥 = 𝐹 𝑥 and lim 𝐹𝑛 𝑥 − = 𝐹 𝑥 − , ∀𝑥 ∈ ℝ.

𝑛 ∞ 𝑛 ∞

𝑑
(iii) First suppose that 𝑋𝑛 𝑋, as 𝑛 ∞. Then, for 𝑥 ∈ 0, 1, 2, … ,

lim 𝑓𝑛 𝑥 = lim 𝑃 𝑋𝑛 = 𝑥
𝑛 ∞ 𝑛 ∞

= lim 𝐹𝑛 𝑥 − 𝐹𝑛 𝑥 −
𝑛 ∞

=𝐹 𝑥 −𝐹 𝑥− (using (ii))

=𝑃 𝑋=𝑥

=𝑓 𝑥 .

Conversely suppose that lim 𝑓𝑛 𝑥 = 𝑓 𝑥 , ∀𝑥 ∈ 0, 1, 2, … . Then, for 𝑥 ∈ ℝ,

𝑛 ∞

𝐹𝑛 𝑥 = 𝑃 𝑋𝑛 ≤ 𝑥
𝑥

= 𝑃 𝑋𝑛 = 𝑘
𝑘=0

= 𝑓𝑛 𝑘
𝑘=0

𝑥
𝑛 ∞
𝑓 𝑘
𝑘=0

=𝐹 𝑥 ,
𝑑
where 𝑥 denotes the largest integer not exceeding 𝑥. It follows that 𝑋𝑛 𝑋, as 𝑛 ∞.
▄

For the random variables of absolutely continuous type we state the following theorem
without providing its proof.

Theorem 1.2

Let 𝑋𝑛 𝑛≥1 be a sequence of random variables of absolutely continuous type with

corresponding sequence of p.d.f.s as 𝑓𝑛 𝑛≥1 . Further let 𝑋 be another random variable of

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 3

NPTEL- Probability and Distributions

absolutely continuous type with p.d.f. 𝑓 . Suppose that lim𝑛 ∞ 𝑓𝑛 𝑥 = 𝑓 𝑥 , ∀𝑥 ∈ ℝ .

𝑑
Then 𝑋𝑛 𝑋, as 𝑛 ∞. ▄
𝑑
The following example demonstrates that if 𝑋𝑛 𝑋 , as 𝑛 ∞ , then
lim𝑛 ∞ 𝐹𝑛 𝑥 − = 𝐹 𝑥 − may not hold; here 𝐹𝑛 and 𝐹 are d.f.s of 𝑋𝑛 (𝑛 = 1, 2, …) and
𝑋, respectively.

Example 1.5
1
Let 𝑋𝑛 ~ 𝑁 0, 𝑛 , 𝑛 = 1,2, … , and let 𝑋 be a random variable degenerate at 0 (i. e.,
𝑃 𝑋 = 0 ) = 1 . Then, for 𝑥 ∈ ℝ,

0, if 𝑥 < 0
𝐹 𝑥 =𝑃 𝑋≤𝑥 =
1, if 𝑥 ≥ 0

𝐹𝑛 𝑥 = 𝑃 𝑋𝑛 ≤ 𝑥

=Φ 𝑛𝑥

0, if 𝑥 < 0
𝑛 ∞ 1
, if 𝑥 = 0.
2
1, if 𝑥 > 0
𝑑
Clearly lim𝑛 ∞ 𝐹𝑛 𝑥 = 𝐹 𝑥 , ∀𝑥 ∈ 𝐶𝐹 = ℝ − 0 and, therefore, 𝑋𝑛 𝑋 ,
𝑝 1
(equivalently 𝑋𝑛 0 ) as 𝑛 ∞ . However lim𝑛 ∞ 𝐹𝑛 0 − = lim𝑛 ∞ 𝐹𝑛 0 =2≠
𝐹 0 − = 0. ▄

The following example illustrates that, in general, the limiting distribution cannot be
obtained by taking the limit of p.m.f.s/p.d.f.s.

Example 1.6

Let 𝑋𝑛 𝑛≥1 be a sequence of random variables such that

1 1 1
𝑃 𝑋𝑛 = =𝑃 𝑋𝑛 = = , 𝑛 = 1,2, …,
2𝑛 𝑛 2
and let 𝑋 be another random variable with 𝑃 𝑋 = 0 = 1. Then it is easy to verify that
𝑑
𝑋𝑛 𝑋, as 𝑛 ∞. The p.m.f. of 𝑋𝑛 is

1 1 1
𝑓𝑛 𝑥 = 2 , if 𝑥 ∈ ,
2𝑛 𝑛 ,
0, otherwise

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 4

NPTEL- Probability and Distributions

and the p.m.f. of 𝑋 is

1, if 𝑥 = 0
𝑓 𝑥 = ∙
0, otherwise

We have

lim𝑛 ∞ 𝑓𝑛 𝑥 = 0 ≠ 𝑓 𝑥 , ∀𝑥 ∈ ℝ. ▄

The following theorem provides a characterization of convergence in probability.

Theorem 1.3

Let 𝑋𝑛 𝑛≥1 be a sequence of random variables and let 𝑐 be a real constant. Then
𝑝
𝑋𝑛 𝑐, as 𝑛 ∞ ⟺ ∀𝜀 > 0, lim 𝑃 𝑋𝑛 − 𝑐 ≥ 𝜀 = 0.
𝑛 ∞

Proof. Let 𝐹𝑛 denote the d.f. of 𝑋𝑛 (𝑛 = 1, 2, …) and let 𝐹 denote the d.f. of random
𝑝
variable degenerate at 𝑐. First suppose that 𝑋𝑛 𝑐, as 𝑛 ∞. Then, for 𝑥 ∈ ℝ − 𝑐 ,

lim 𝐹𝑛 𝑥 = lim 𝑃 𝑋𝑛 ≤ 𝑥
𝑛 ∞ 𝑛 ∞

0, if 𝑥 < 𝑐
= = 𝐹 𝑥 .
1, if 𝑥 > 𝑐
Fix 𝜀 > 0. Then 𝑐 ± 𝜀 ∈ 𝐶𝐹 and therefore, using Theorem 1.1,

lim 𝑃 𝑋𝑛 − 𝑐 ≥ 𝜀 = lim 𝑃 𝑋𝑛 ≤ 𝑐 − 𝜀 + 𝑃 𝑋𝑛 ≥ 𝑐 + 𝜀
𝑛 ∞ 𝑛 ∞

= lim 𝐹𝑛 𝑐 − 𝜀 + 1 − 𝐹𝑛 𝑐 + 𝜀 − (1.3)
𝑛 ∞

= 𝐹 𝑐−𝜀 +1−𝐹 𝑐+𝜀

= 0.

Conversely, suppose that

lim 𝑃 𝑋𝑛 − 𝑐 ≥ 𝜀 = 0, ∀𝜀 > 0.
𝑛 ∞

Then, using (1.3),

lim 𝐹𝑛 𝑐 − 𝜀 + 1 − 𝐹𝑛 𝑐 + 𝜀 − = 0, ∀𝜀 > 0,
𝑛 ∞

⇒ lim 𝐹𝑛 𝑐 − 𝜀 = lim 1 − 𝐹𝑛 𝑐 + 𝜀 − = 0, ∀𝜀 > 0

𝑛 ∞ 𝑛 ∞
since 𝐹𝑛 𝑐 − 𝜀 ≥ 0 and 1 − 𝐹𝑛 𝑐 + 𝜀 − ≥ 0, ∀𝑛 ≥ 1

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 5

NPTEL- Probability and Distributions

⇒ lim 𝐹𝑛 𝑥 = 0, ∀𝑥 < 𝑐 and lim 𝐹𝑛 𝑦 − = 1, ∀𝑦 > 𝑐

𝑛 ∞ 𝑛 ∞

⇒ lim 𝐹𝑛 𝑥 = 0, ∀𝑥 < 𝑐 and lim 𝐹𝑛 𝑦 = 1, ∀𝑦 > 𝑐

𝑛 ∞ 𝑛 ∞
since 1 ≥ 𝐹𝑛 𝑦 ≥ 𝐹𝑛 𝑦 − , 𝑛 = 1,2, … .

Thus, for all 𝑥 ∈ ℝ − 𝑐 ,

0, if 𝑥 < 𝑐
lim 𝐹𝑛 𝑥 = = 𝐹(𝑥)
𝑛 ∞ 1, if 𝑥 > 𝑐
𝑝
⇒ 𝑋𝑛 𝑐, as 𝑛 ∞. ▄

In many situations the above theorem in conjunction with Markov’s inequality (see
Corollary 5.1, Module 3) turns out to be quite useful in proving convergence in
probability.

Dept. of Mathematics and Statistics Indian Institute of Technology, Kanpur 6