Lecture notes for Part A Probability

James Martin

Oxford, Michaelmas Term 2016

[email protected]

Version of 9 January 2017

 1

Review: probability spaces, random variables,

distributions, independence

1.1 Probability spaces and random variables

We start by reviewing the basic idea of a probability space introduced in last year’s course.
This framework underlies modern probability theory, even though we will seldom need to
appeal to it directly in this course.
A probability space is a collection (Ω, F, P) with the following structure:

(i) Ω is a set, which we call the sample space.

(ii) F is a collection of subsets of Ω. An element of F is called an event.

(iii) The probability measure P is a function from F to [0, 1]. It assigns a probability
to each event in F.

We can think of the probability space as modelling an experiment. The sample space Ω
represents the set of all possible outcomes of the experiment.
The set F of events should satisfy certain natural conditions:

(1) Ω ∈ F.

(2) If F contains a set A, then it also contains the complement Ac (i.e. Ω \ A).

(3) If (Ai , i ∈ I) is a finite or countably infinite collection of events in F, then their union
S
i∈I Ai is also in F.

By combining (2) and (3), we can also obtain finite or countable intersections, as well as
unions.
The probability measure P should satisfy the following conditions (the probability axioms):

(1) P(Ω) = 1.

(2) If (Ai , i ∈ I) is a finite or countably infinite collection of disjoint events, then

!
[ X
P Ai = P(Ai ). (1.1)
i∈I i∈I

1
Probability Part A, version of 2017/1/9 2

A random variable is a function defined on Ω. We will consider real-valued random

variables, i.e. functions from Ω to R.
A random variable represents an observable in our experiment; something we can “mea-
sure”.
Formally, for a function X from Ω to R to be a random variable, we require that the
events
{X(ω) ≤ x}

are contained in F, for every x ∈ R. (Then, by taking complements, unions and intersections,
we will in fact have that the event {X(ω) ∈ B} is in F for a very large class of sets B).
We will usually write X rather than X(ω) for the value taken by a random variable. Thus
if X is a random variable we can talk about the probability of the event {X ∈ B}, which we
will write as P(X ∈ B).
Within one experiment, there will be many observables! That is, on the same probability
space we can consider many different random variables.

Remarks:

(a) For very simple models, there may be a natural way to set up the sample space Ω (e.g.
to represent the set of possible outcomes of the throw of a die or a coin). For more
complicated models, this quickly becomes less straightforward. In practice, we hardly
ever want to consider Ω directly; instead we work directly with the “events” and “random
variables” (the “observables”) in the experiment.

(b) In contrast, there are settings in probability theory where we care a lot about the collection
of events F, and its structure. (For example, modelling a process evolving in time, we
might have a family of different collections Ft , t ≥ 0, where Ft represents the set of events
which can be observed by watching the evolution of the process up to time t). However,
for the purposes of this course we will hardly ever worry about F directly; we will be safe
to assume that F will always contain any event that we wish to consider.

1.1.1 Examples
Here are some examples of systems (or “experiments”) that we might model using a prob-
ability space, and, for each one, some examples of random variables that we might want to
consider within our model:

• We throw two dice, one red and one blue. Random variables: the score on the red die;
the score on the blue die; the sum of the two; the maximum of the two; the indicator
function of the event that the blue score exceeds the red score....

• A Geiger counter detecting particles emitted by a radioactive source. Random variables:

the time of the kth particle detected, for k = 1, 2, . . . ; the number of particles detected
in the time interval [0, t], for t ∈ R+ ....

• A model for the evolution of a financial market. Random variables: the prices of various
stocks at various times; interest rates at various times; exchange rates at various times....
Probability Part A, version of 2017/1/9 3

• The growth of a colony of bacteria. Random variables: the number of bacteria present
at a given time; the diameter of the colonised region at given times; the number of
generations observed in a given time interval....

• A call-centre. The time of arrival of the kth call; the length of service required by the
kth caller; the wait-time of the kth caller in the queue before receiving service....

1.2 Probability distributions

We consider the distribution of a random variable X. This can be summarised by the
distribution function (or cumulative distribution function) of X, defined by

F (x) = P(X ≤ x)

for x ∈ R. (Once we know F , we can derive the probabilities P(X ∈ B) for a very wide class
of sets B by taking complements, intersections and unions.)
Any distribution function F must obey the following properties:

(1) F is non-decreasing.

(2) F is right-continuous.

(3) F (x) → 0 as x → −∞.

(4) F (x) → 1 as x → ∞.

Remark 1.1. Note that two different random variables can have the same distribution! For
example, consider the model of two dice mentioned above. If the dice are “fair”, then the
distribution of the score on the blue die might be the same as the distribution of the score
on the red die. However, that does not mean that the two scores are always the same! They
are two different “observables” within the same experiment.
d
If two random variables X and Y have the same distribution, we write X = Y .

We single out two important classes of random variables: discrete and continuous.

1.2.1 Discrete random variables

A random variable X is discrete if there is a finite or countably infinite set B such that
P(X ∈ B) = 1.
We can represent the distribution of a discrete random variable X by its probability mass
function
pX (x) = P(X = x)

for x ∈ R. This function is zero except at a finite or countably infinite set of points. We have
P
• x pX (x) = 1.
P
• P(X ∈ A) = x∈A pX (x) for any set A ⊆ R.
Probability Part A, version of 2017/1/9 4

The points x where P(X = x) > 0 are sometimes called the atoms of the distribution of
X. In many examples these will be a set of integers such as {1, 2, . . . , n} or {0, 1, 2, . . . } or
{1, 2, 3 . . . }.
The cumulative distribution function of X has jumps at the location of the atoms, and is
constant on any interval that does not contain an atom.

1.2.2 Continuous random variables

A random variable X is called continuous if its distribution function F can be written as
an integral. That is, there is a function f such that
Z x
P(X ≤ x) = F (x) = f (u)du.
−∞

f is called the density function (or probability density function) of X.

This certainly implies that F is a continuous function (although note that not all con-
tinuous F can be written in this way). In particular, P(X = x) = F (x) − limy↑x F (y) = 0
for any x. The density function is not unique; for example, we can change the value of f at
any single point without affecting the integral of f . At points where F is differentiable, it’s
natural to take f (x) = F 0 (x). For any a < b, we have
Z b
P(a ≤ X ≤ b) = f (u)du.
a

1.3 Expectation and variance

Let X be a discrete random variable with probability mass function pX (x) = P(X = x). The
expectation (or mean) of X is defined by
X
E (X) = xpX (x), (1.2)
x

when this sum converges.

If instead X is a continuous random variable with density function f , then its expectation
is given by Z ∞
E (X) = xf (x)dx. (1.3)
−∞

We often want to express the expectation of a function of a random variable X in terms

of the density function or the mass function of X. We have
X
E g(X) = g(x)pX (x)
x

in the discrete case, and Z ∞

E g(X) = g(x)f (x)dx
−∞

in the continuous case.

It’s rather unsatisfactory that we have two different definitions of expectation for two
different cases, and no definition at all for random variables which are neither continuous
Probability Part A, version of 2017/1/9 5

nor discrete. In fact it’s not difficult to unify the definitions. A very natural way is to
consider approximations of a general random variable by discrete random variables. This is
analogous to the construction of the integral of a general function by defining the integral
of a step function using sums, and then defining the integral of a general function using an
approximation by step functions, which you saw in last year’s analysis course.
This unifies the two definitions above, and extends the definition to all types of random
variable, whether discrete, continuous or neither. We won’t pursue this here - but we will
collect together basic properties of expectation which we will use constantly:

(1) For any event A, write IA for the indicator function of A. Then E IA = P(A).

(2) If P(X ≥ 0) = 1, then E X ≥ 0.

(3) (Linearity 1): E (aX) = aE X for any constant a.

(4) (Linearity 2): E (X + Y ) = E X + E Y .

1.3.1 Variance and covariance

The variance of a random variable X is defined by

Var(X) = E [(X − E X)2 ]

which can alternatively be expressed as

Var(X) = E (X 2 ) − (E X)2 .

The covariance of two random variables X and Y is defined by

Cov(X, Y ) = E [(X − E X)(Y − E Y )]

which can alternatively be expressed as

Cov(X, Y ) = E (XY ) − (E X)(E Y ).

Note that Var(X) = Cov(X, X). From the linearity of expectation, we get a bi-linearity
property for covariance:

Cov(aX, +b, cY + d) = ac Cov(X, Y ).

As a special case we can obtain

Var(aX + b) = a2 Var(X).

We also have the property

Var(X + Y ) = Var X + Var Y + 2 Cov(X, Y )

and more generally

n
X X
Var(X1 + X2 + · · · + Xn ) = Var(Xi ) + 2 Cov(Xi , Xj ).
i=1 1≤i<j≤n
Probability Part A, version of 2017/1/9 6

1.4 Independence
Events A and B are independent if

P(A ∩ B) = P(A)P(B).

More general, a family of events (Ai , i ∈ I) (maybe infinite, even uncountable) is called
independent if for all finite subsets J of I,
!
\ Y
P Ai = P(Ai ).
i∈J i∈J

Remark 1.2. Remember that this is a stronger condition than pairwise independence! Even
for three events, it’s possible that A1 , A2 are independent, A2 , A3 are independent and A1 , A3
are independent but that A1 , A2 , A3 are not independent.
Random variables X1 , X2 , . . . , Xn are independent if for all B1 , B2 , . . . , Bn ⊂ R, the events
{X1 ∈ B1 }, {X2 ∈ B2 }, . . . , {Xn ∈ Bn } are independent.
In fact, it turns out to be enough to check that for all x1 , x2 , . . . , xn ,

P(X1 ≤ x1 , . . . , Xn ≤ xn ) = P(X1 ≤ x1 ) . . . P(Xn ≤ xn )

= FX1 (x1 ) . . . FXn (xn ).

If the random variables are all discrete, another equivalent condition is that

P(X1 = x1 , . . . , Xn = xn ) = P(X1 = x1 ) . . . P(Xn = xn ).

When X and Y are independent random variables, we have E (XY ) = E (X)E (Y ), or

equivalently Cov(X, Y ) = 0. That is, X and Y are uncorrelated. The converse is not true;
uncorrelated does not imply independent!
Various of the properties above can be summarised by the phrase “independence means
multiply”.

1.5 Examples of probability distributions

We review some of the families of probability distributions which are of particular importance
in applications and in theory.

1.5.1 Continuous distributions

Uniform distribution

X has the uniform distribution on an interval [a, b] if its probability density function is given
by 
 1 if a ≤ x ≤ b
f (x) = b−a .
0 otherwise

We write X ∼ U [a, b].

Probability Part A, version of 2017/1/9 7

Exponential distribution

X has exponential distribution with parameter (or rate) λ if its distribution function is given
by 
0 x<0
F (x) = .
1 − e−λx x ≥ 0

Its density function is 

0 x<0
f (x) = .
λe−λx x≥0

We write X ∼ Exp(λ). We have E X = 1/λ and Var X = 1/λ2 . If X ∼ Exp(λ) and

a > 0, then aX ∼ Exp(λ/a). An important property of the distribution is the memoryless
property: P(X > x + t|X > t) does not depend on t.

Normal distribution

X has the normal (or Gaussian) distribution with mean µ and variance σ 2 if its density
function is given by
(x − µ)2
 
1
f (x) = √ exp − .
2πσ 2 2σ 2
We write X ∼ N (µ, σ 2 ). The standard normal distribution is N (0, 1).
If X ∼ N (µ, σ 2 ) then aX + b ∼ N (aµ + b, a2 σ 2 ). In particular, (X − µ)/σ has standard
normal distribution.
2
If X ∼ N (µX , σX ) and Y ∼ N (µY , σY2 ) are independent, then X + Y ∼ N (µX + µY , σX2
+
2
σY ).
The normal distribution has an extremely important role in probability theory, exemplified
by the fact that it appears as the limit in the Central Limit Theorem.
We often write Φ for the distribution function of the standard normal distribution:
Z x  2
1 z
Φ(x) = √ exp − dz.
−∞ 2π 2

Gamma distribution

The family of gamma distributions generalises the family of exponential distributions. The
gamma distribution with rate λ and shape r has density
 r
 λ xr−1 e−λx x ≥ 0

f (x) = Γ(r) .
0

x<0
R∞
Here Γ(r) is the gamma function, defined by Γ(r) = 0 z r−1 e−z dz. It is the analytic contin-
uation of the factorial function, in that Γ(r) = (r − 1)! when r is an integer.
A gamma distribution with shape r = 1 is an exponential distribution.
If X ∼ Gamma(rX , λ) and Y ∼ Gamma(rY , λ) are independent, then X+Y ∼ Gamma(rX +
rY , λ). As a special case, if X1 , X2 , . . . , Xn are i.i.d. with Exp(λ) distribution, then X1 +
X2 + · · · + Xn has Gamma(n, λ) distribution.
Probability Part A, version of 2017/1/9 8

1.5.2 Discrete distributions

Discrete uniform distribution

X has the discrete uniform distribution on a set B of size n (for example the set {1, 2, . . . , n})
if 
1/n, x ∈ B
pX (x) = .
0, x∈ /B

Bernoulli distribution

X has Bernoulli distribution with parameter p if

pX (1) = p, pX (0) = 1 − p

(and so of course pX (x) = 0 for other values of x).

We have E X = p and Var X = p(1 − p).
If A is an event with P(A) = p, then its indicator function 1A has Bernoulli distribution
with parameter p.

Binomial distribution

If X1 , X2 , . . . , Xn are i.i.d. Bernoulli random variables with the same parameter p, then their
sum X1 + · · · + Xn has Binomial distribution with parameters n and p.
Equivalently, if A1 , . . . , An are independent events, each with probability p, then the total
number of those events which occur has Binomial(n, p) distribution.
If X ∼ Binomial(n, p) then
!
n k
pX (k) = p (1 − p)n−k for k ∈ {0, 1, . . . , n}.
k

E X = np and Var X = np(1 − p).

Geometric distribution

Let p ∈ (0, 1) and let X have mass function

pX (k) = (1 − p)k−1 p for k ∈ {1, 2, 3, . . . }.

Let Y = X − 1; then Y has mass function

pY (k) = (1 − p)k p for k ∈ {0, 1, 2, . . . }.

The terminology is not consistent; either X or Y might be said to have a geometric distribution
with parameter p. (Or even sometimes with parameter 1 − p).
If we have a sequence of independent trials, with probability p of success at each trial,
then X could represent the number of trials needed for the first success to occur, while Y
could represent the number of failures needed before the first success occurs.
Probability Part A, version of 2017/1/9 9

We have
P(X > k) = P(Y ≥ k) = (1 − p)k for k = 0, 1, 2, . . . .

The geometric distribution can be thought of as a discrete anaologue of the exponential

distribution. It too has a memoryless property; for k, m ∈ 0, 1, 2 . . . , the quantity P(X >
k + m|X > k) does not depend on k.

Poisson distribution

X has Poisson distribution with mean λ if

e−λ λr
P(X = r) = .
r!
E X = λ and Var X = λ.
If X ∼ Poisson(λ) and Y ∼ Poisson(µ) are independent, then X + Y ∼ Poisson(λ + µ).
The Poisson distribution arises in many applications; it is a good model for the total
number of events occurring when there are a large number of possible events which each
occur with small probability. There are close connections between the Poisson distribution
and the exponential distribution, which we will see in detail when we study Poisson processes
at the end of the course.
 2

Convergence of random variables, and limit

theorems

Let X and Y be random variables. What might we mean by saying “X and Y are close”?

(1) We might be describing a particular realisation. For example, we made an observation of

X and Y , and on this occasion |X − Y | < .

(2) We might be making a statement about the joint distribution of X and Y , for example

P(|X − Y | < ) > 1 − ,

or


E |X − Y | < .

(3) We might be comparing the distribution of X with the distribution of Y , for example

|FX (x) − FY (x)| <  for all x.

Correspondingly, there are several different things we might mean when we say that a sequence
of random variables converges to a limit.

2.1 Modes of convergence

Let X1 , X2 , . . . and X be random variables.
Note that {Xn → X as n → ∞} is an event. More formally we could write {Xn (ω) → X(ω) as n → ∞}
to emphasise the dependence on ω; the event might hold for some ω but not for others.

Definition. Xn → X almost surely (or “with probability 1”) if

P (Xn → X as n → ∞) = 1. (2.1)

We often abbreviate to “Xn → X a.s.”.

10
Probability Part A, version of 2017/1/9 11

P
Definition. Xn → X in probability (written Xn → X) if for every  > 0,
  
P Xn − X  <  → 1 as n → ∞. (2.2)

Let F1 , F2 , . . . and F be the distribution functions of X1 , X2 , . . . and X respectively.

d
Definition. Xn → X in distribution (or weakly), written Xn → X, if, for every x such
that F is continuous at x,
Fn (x) → F (x) as n → ∞. (2.3)

We will see later that these formulations are in decreasing order of strength.

2.2 Convergence in distribution

Notice that in the definition of convergence in distribution in (2.3), the random variables
involved appear only through their distributions. Hence we do not even need all the random
variables to be defined on the same probability space. This is really a definition about conver-
gence of distributions, not about convergence of random variables. The joint distribution of
the random variables does not need to be defined. This is in contrast to the definitions of al-
most sure convergence in (2.1) and of convergence in probability in (2.2), where we genuinely
do need all the random variables to be defined on the same space.
As a result, we might sometimes vary the notation by writing a distribution rather than
d
a random variable on the right-hand side; e.g. “Xn → N (0, 1)” if the limit in distribution is
d
the standard normal, or “Xn → U [0, 1]” if the limit in distribution is the uniform distribution
on [0, 1].
In many cases the limit will be deterministic; e.g. if the limit is a distribution which puts
d
all its mass at the value 0, then we will write Xn → 0.
In (2.3), why did we ask for the limit to hold only for x which are continuity points of F ,
rather than at all x? The first couple of examples (which are almost trivial) make this clear.

Example 2.1. Let Xn have the uniform distribution on the interval [−1/n, 1/n]. Then
Fn (x) → 0 for all x < 0, and Fn (x) → 1 for all x > 0.
d
So we have Xn → 0, i.e. the distribution of Xn converges to that of a deterministic random
variable which is equal to 0 with probability 1. Such a random variable has distribution
function given by F (x) = 0 for x < 0 and F (x) = 1 for x ≥ 0.
Note that Fn (0) = 1/2 for all n, while F (0) = 1. So convergence does not hold at the
point 0 itself (but this is OK, since 0 is not a continuity point of F ).

Example 2.2. Let Xn be a deterministic random variable taking the value 1/n with prob-
ability 1. Let X be a deterministic random variable taking the value 0 with probability
d
1 (as above). Then once again, Xn → X, (even though P(Xn ≤ 0) = 0 for all n while
P(X ≤ 0) = 1).

There are many situations in which a sequence of discrete random variables converges to
a continuous limit. Here is one example, showing that a geometric distribution with a small
parameter is well approximated by an exponential distribution:
Probability Part A, version of 2017/1/9 12

Example 2.3. Let Xn have geometric distribution on the positive integers, with parameter
pn , i.e. P(Xn = k) = (1 − pn )k−1 pn for k = 1, 2, . . . . Show that if pn → 0 as n → ∞, then
pn Xn converges in distribution to the exponential distribution with mean 1.
Solution: We have P(Xn > k) = (1 − pn )k for k = 0, 1, 2, . . . . For x ≥ 0, we have
 
x
P(pn Xn > x) = P Xn >
pn
  
x
= P Xn >
pn
= (1 − pn )bx/pn c
→ e−x as n → ∞

because pn → 0; here we use the fact that (1 − )x/ → e−x as  → 0, and also that
 
x/pn − x/pn is bounded.
Hence if Fn is the distribution function of pn Xn , then 1 − Fn (x) → e−x as n → ∞. So
Fn (x) → 1 − e−x for all x > 0, while Fn (x) = 0 for all x ≤ 0 and all n.
So indeed Fn (x) → F (x) for all x, where F is the distribution function of a random
variable with Exp(1) distribution.

There are several more examples on the problem sheets.

2.3 Comparison of different modes of convergence

Theorem 2.4. The following implications hold:

Xn → X almost surely ⇒ Xn → X in probability ⇒ Xn → X in distribution

The reverse implications do not hold in general.

Before starting the proof we note a useful fact, which is a simple consequence of the
countable additivity axiom for unions of disjoint sets (1.1).

Lemma 2.5. Let An , n ≥ 1 be an increasing sequence of events; that is, A1 ⊆ A2 ⊆ A3 ⊆ . . . .

Then [ 
lim P(An ) = P An . (2.4)
n→∞

Proof. Because the sequence An is increasing, it’s easy to rewrite the union as a disjoint
union:
[  


P An = P A1 ∪ A2 \ A1 ∪ A3 \ A2 ∪ . . .
∞
X

= P(A1 ) + P Ai+1 \ Ai (using countable additivity)
i=1
n−1
X
= P(A1 ) + lim P (Ai+1 \ Ai )
n→∞
1
 n−1
X 
= lim P(A1 ) + P (Ai+1 \ Ai )
n→∞
1
Probability Part A, version of 2017/1/9 13

= lim P(An ).
n→∞

Proof of Theorem 2.4.

(1) First we’ll show that convergence in probability implies convergence in distribution. Let
Fn be the distribution function of Xn , and F the distribution function of X. Fix any x such
that F is continuous at x, and fix any  > 0.
Observe that if Xn ≤ x, then either X ≤ x +  or |Xn − X| > . Hence

Fn (x) = P(Xn ≤ x)


≤ P X ≤ x +  or |Xn − X| > 



≤ P X ≤ x +  + P |Xn − X| > 
→ F (x + ) as n → ∞,

using the convergence in probability. So Fn (x) < F (x + ) +  for all large enough n.
Similarly by looking at 1 − Fn (x) = P(Xn > x), we can obtain that Fn (x) > F (x − ) − 
for all large enough n.
Since  > 0 is arbitrary, and since F is continuous at x, this implies that Fn (x) → F (x)
as n → ∞.

(2) For convergence in distribution, we don’t need the random variables to be defined on
the same probability space. But even if they are, convergence in distribution does not imply
convergence in probability. For example, suppose that X and Y are random variables with
the same distribution but with P(X = Y ) < 1. Then the sequence X, X, X, . . . converges to
Y in distribution, but not in probability.

(3) Now we’ll show that almost sure convergence implies convergence in probability. Fix  > 0
and for N ∈ N, define the event AN by

AN = {|Xn − X| <  for all n ≥ N } .

Suppose that Xn → X almost surely. If the event {Xn → X} occurs, then the event AN
S

must occur for some N , so we have P AN = 1. AN is an increasing sequence of events,
so (2.4) then gives limN →∞ P(AN ) = 1.  
  
But AN implies XN − X  < , giving P XN − X  <  → 1. Since  is arbitrary, this
means that Xn → X in probability, as desired.

(4) Finally we want to show that convergence in probability does not imply almost sure
convergence.
Consider a sequence of independent random variables Xn where P(Xn = 1) = 1/n and
P(Xn = 0) = (n − 1)/n.
Probability Part A, version of 2017/1/9 14

We have Xn → 0 in probability as n → ∞ because for any  > 0, P(|Xn − 0| < ) ≥

P(X = 0) → 1.
Since Xn only take the values 0 and 1, the event {Xn → 0} is the same as the event
S
{Xn = 0 eventually}. This is BN where BN = {Xn = 0 for all n ≥ N }.
But for any N ,
N −1 N N +1
P(BN ) = ...
N N +1N +2
= 0.
S
Hence also by countable additivity, P( BN ) = 0, and so P(Xn → 0) = 0. Hence it is not
the case that Xn converges to 0 almost surely.

Although convergence in distribution is weaker than convergence in probability, there is

a partial converse, for the case when the limit is deterministic:

Theorem 2.6. Let X1 , X2 , . . . be a sequence of random variables defined on the same prob-
ability space. If Xn → c in distribution where c is some constant, then also Xn → c in
probability.

Proof. Exercise (see problem sheet).

2.4 Weak law of large numbers

Let Sn = X1 + X2 + · · · + Xn , where Xi are i.i.d. with mean µ. The law of large numbers tells
us that, roughly speaking, Sn behaves to first order like nµ as n → ∞. The weak law phrases
this in terms of convergence in probability. (Later we will see a stronger result in terms of
almost sure convergence).

Theorem (Weak Law of Large Numbers). Let X1 , X2 , . . . be i.i.d. random variables with
finite mean µ. Let Sn = X1 + X2 + · · · + Xn . Then
Sn P
→ µ as n → ∞.
n
That is, for all  > 0,   
 Sn 
P 
 − µ <  → 1 as n → ∞.
 (2.5)
n
d
Given Theorem 2.6, we could equivalently write Snn → µ.
We will give an extremely simple proof of the weak law of large numbers, under an
additional condition (that the Xi have finite variance). To do this, we need some results
which give probability bounds on the tail of a distribution in terms of its mean and variance.

Theorem (Markov’s inequality). Let X be random variable taking non-negative values (i.e.
P(X ≥ 0) = 1). Then for any z > 0,

EX
P(X ≥ z) ≤ . (2.6)
z
Probability Part A, version of 2017/1/9 15

Proof. The proof is very easy. We consider a random variable Xz = z1{X ≥ z}. So Xz takes
the value 0 whenever X is in [0, z) and the value z whenever X is in [z, ∞). So X ≥ Xz
always (here we use the fact that X is non-negative).
Then E X ≥ E Xz = zE 1{X ≥ z} = zP(X ≥ z). Rearranging gives the result.

Theorem (Chebyshev’s inequality). Let Y be a random variable with finite mean and vari-
ance. Then for any  > 0,

Var Y
P |Y − E Y | ≥  ≤ .
2
Proof.
P |Y − E Y | ≥  = P [Y − E Y ]2 ≥ 2





E [Y − E Y ]2
≤
2
(by applying Markov’s inequality (2.6) with X = [Y − E Y ]2 and z = 2 )
Var Y
= .
2

Proof of the weak law of large numbers, for the case of random variables with finite variance.
Let Xi be i.i.d. with mean µ and variance σ2 . Recall Sn = X1 + · · · + Xn . We want to show
d
that Sn /n → µ as n → ∞.
We have E (Sn /n) = µ, and (using the independence of the Xi ),
 
Sn Var Sn
Var =
n n2
Var X1 + · · · + Var Xn
=
n2
2
nσ
= 2
n
σ2
= .
n
Fix any  > 0. Using Chebyshev’s inequality applied to the random variable Sn /n, we have
Var Snn
  

 Sn 
P  − µ ≥  ≤
n 2
σ2
= 2
n
→ 0 as n → ∞.
So indeed (2.5) holds, as required.

Remark. Observe that we could relax considerably the assumptions in the weak law of large
numbers, and still get the same result using almost the same proof. We never used at all the
assumption that the Xi all had the same distribution. We could also relax the assumption
that the Xi are independent, as long as the variance of Sn grows more slowly than n2 . For
example, if we have an upper bound on the variance of each Xi , and a bound which is
o(n2 ) on the sum 1≤i<j≤n Cov(Xi , Xj ), then exactly the same idea applies to show that
P

(Sn − E Sn )/n converges to 0 in distribution.

Probability Part A, version of 2017/1/9 16

2.5 Strong law of large numbers

In the weak law of large numbers, we proved convergence in distribution of the average of
i.i.d. random variables to the mean. The strong law says more: the convergence of occurs
with probability 1.

Theorem (Strong Law of Large Numbers). Let X1 , X2 , . . . be i.i.d. with mean µ. Let Sn =
X1 + · · · + Xn . Then
Sn
→ µ almost surely as n → ∞.
n

2.5.1 Proof of the strong law of large numbers (non-examinable)

A proof of the strong law of large numbers in full generality is somewhat involved. However,
if we assume an extra condition, namely that the distribution has a finite fourth moment,
then a very straightforward proof is possible. [NB the proof is not examinable.]

Proof of Strong Law of Large Numbers, under the additional condition E Xn4 < ∞.
Let us centre the Xn , writing Wn = Xn − µ.
Then E Wn = 0, and we have E Xn4 < ∞ ⇒ E Wn4 < ∞ (exercise).
Note also that

(E Wn2 )2 = E (Wn4 ) − Var(Wn2 )

≤ E (Wn4 ).
 
We will consider E (Sn − nµ)4 . Expanding the fourth power and using linearity of
expectation, we obtain

E (Sn − nµ)4 = E (W1 + W2 + · · · + Wn )4

   
X X X
= E Xi4 + 4 E Wi3 Wj + 3 E Wi2 Wj2
1≤i≤n 1≤i,j≤n 1≤i,j≤n
i6=j i6=j
X X
+6 E Wi2 Wj Wk + E Wi Wj Wk W l .
1≤i,j,k≤n 1≤i,j,k,l≤n
i,j,k distinct i,j,k,l distinct

(The exact constants in front of the sums are not too important!) Using independence and
E Wi = 0, most of these terms vanish. For example, E Wi3 Wj = E Wi3 E Wj = 0. We are left
with only

E (Sn − nµ)4 = nE W14 + 3n(n − 1)E [(W12 )2 ]

 

≤ 3n2 E W14 .

From this we have

∞  4 X∞  4
X Sn Sn
E −µ = E −µ
n=1
n n=1
n
∞
X 1 4
= 4
E (Sn − nµ)
n=1
n
Probability Part A, version of 2017/1/9 17

∞
X 3E W14
≤
n=1
n2
< ∞.

But if Z is a random variable with E Z < ∞, then certainly P(Z < ∞) = 1. Applying

4
this with Z = Snn − µ , we get
∞  4 !
X Sn
P − µ < ∞ = 1.
n=1
n

(an − µ)4 is finite, then certainly an → µ as n → ∞. So we can conclude that

P
Finally, if
 
Sn
P → µ as n → ∞ = 1,
n
as required.

2.6 Central limit theorem

The weak law of large numbers tells us that the distribution of Sn /n concentrates around µ
as n becomes large. The central limit theorem (CLT) goes much further, telling us that (if
the random variables Xi have finite variance) the “fluctuations” of Sn around nµ are of order
√
n. Moreover, the behaviour of these fluctuations is universal ; whatever the distribution of
√
the Xi , if we scale Sn − nµ by n, we obtain a normal distribution as the limit as n → ∞.

Theorem (Central Limit Theorem). Let X1 , X2 , . . . be i.i.d. random variables with mean µ
and variance σ 2 . Let Sn = X1 + X2 + · · · + Xn . Then
Sn − nµ d
√ → N (0, 1) as n → ∞. (2.7)
σ n
We’ll prove the CLT later using generating functions.
Remark 2.7. We can summarise the CLT in three stages:

(1) The distribution of Sn concentrates around nµ;

√
(2) The fluctuations of Sn around nµ are of order n;

(3) The asymptotic distribution of these fluctuations is normal.

These are somehow in increasing order of refinement. Some students take in the third of
these, but not the first two; they remember that the RHS in (2.7) is a normal distribution,
but are hazy about what is going on on the LHS. This is a bit perverse; without knowing the
scale of the fluctuations, or what they fluctuate around, knowing their distribution is not so
useful!

Example 2.8. An insurance company sells 10, 000 similar car insurance policies. They
estimate that the amount paid out in claims on a typical policy has mean £240 and standard
deviation £800. Estimate how much they need to put aside in reserves to be 99% sure that
the reserve will exceed the total amount claimed.
Probability Part A, version of 2017/1/9 18

Solution: Let µ = £240, σ = £800, n = 10, 000, and note Φ−1 (0.99) = 2.326 where Φ is the
distribution function of the standard normal.
Let Sn be the total amount claimed. For large n, the Central Limit Theorem tells us that
 
Sn − nµ
P √ < Φ−1 (0.99) ≈ 0.99,
σ n
i.e.
√
P Sn < Φ−1 (0.99)σ n + nµ ≈ 0.99.


√
So the amount needed in reserves is approximately Φ−1 (0.99)σ n + nµ, which in this case is
£2, 586, 080.
Notice that the reserve required per customer is about £258, which is £18 higher than
µ. We can see from the calculation above that this surplus is proportional to n−1/2 . If we
had 100 customers rather than 10,000, we would need a surplus 10 times bigger, while with
1,000,000 customers it would be 10 times smaller.
The fact that the amount per customer needed to cover the fluctuations around the mean
is decreasing in the number of customers is an example of risk pooling.
Of course, the example of identical customers is a bit simplistic, but the effect of risk
pooling that we observe is a very real one. Our analysis also assumed that the different cus-
tomers are independent – is that realistic? For car insurance, it is not such a bad assumption.
Similarly for life insurance. In the case of property insurance, it could be a very bad assump-
tion (for example, floods can damage many properties simultaneously). In that situation, the
effect of risk pooling is a lot smaller (which explains why obtaining insurance for a property
subject to a risk of flooding can be problematic, even if the risk is not that great).
Example 2.9 (Binomial distribution: CLT and Poisson approximation). Let p ∈ (0, 1) and
let Yn have Binomial(n, p) distribution. Then we can write Yn = X1 + · · · + Xn where the
Xi are i.i.d. Bernoulli(p) random variables. (We can think of the Xi as indicator functions of
independent events, all with the same probability, e.g. arising from random sampling).
The Xi each have mean p and variance p(1 − p). So we can apply the CLT to obtain
Yn − np d

√ → N 0, p(1 − p) as n → ∞.
n
Now instead of considering fixed p as above, consider random variables Wn with Binomial(n, pn )
distribution, where pn → 0 as n → ∞. Now a very different limit applies, describing a situa-
tion in which we have a very large number of trials but each one has a very small probability
of success. Let λn = npn , the mean of Wn . Suppose that λn converges to a limit λ as n → ∞,
so that the expected total number of successes stays approximately constant. Then we will
show that Wn converges in distribution to Poisson(λ).
It’s enough to show (check!) that for each fixed k = 0, 1, . . . ,
λk −λ
P(Wn = k) → e
k!
(since the RHS is the probability that a Poisson(λ) random variable takes the value k).
We have
!
n k
lim P(Wn = k) = lim pn (1 − pn )n−k
k
Probability Part A, version of 2017/1/9 19

! k  n−k
n λn λn
= lim 1−
k n n
n −k
nk λk
  
λn λn
= lim 1 − 1 −
k! nk n n
λk −λ
= e ·1
k!
as desired.
 3

Generating functions

3.1 Review of probability generating functions

Let X be a random variable taking non-negative integer values, and let pX be its probability
mass function. The probability generating function of X is defined by
∞

X
G(z) := E z X = pX (k)z k .
k=0

G is a power series whose radius of convergence is at least 1. We can recover the coefficients
of the power series, i.e. the values of the function pX , from the behaviour of G and its
derivatives at the point 0, and we can compute the moments of X from the behaviour of G
and its derivatives at 1:

Theorem 3.1.

(a) G(k) (0) = k! pX (k) for k = 0, 1, 2, . . . .


(b) G(1) = 1 and G(k) (1) = E X(X − 1) . . . (X − k + 1)] for k = 1, 2, . . . .

Here if the radius of convergence of G is exactly 1, then G(k) (1) should be taken to mean
limz↑1 G(k) (z).
From Theorem 3.1(a), we see immediately that a distribution is determined by its gener-
ating function:

Theorem 3.2 (Uniqueness theorem for probability generating functions). If X and Y have
the same generating function, then they have the same distribution.

From Theorem 3.1(b), we have, for example, E (X) = G0 (1), Var(X) = G00 (1) + G0 (1) −
[G0 (1)]2 .
Generating functions are extremely useful tools for dealing with sums of independent
random variables. Let X and Y be independent random variables with generating functions
GX and GY . Then the generating function of their sum is given by

GX+Y (z) = E z X+Y

= E zX zY

20
Probability Part A, version of 2017/1/9 21

= E z X E z Y (by independence)

= GX (z)GY (z).

(Again, “independence means multiply”).

We can also treat the sum of a random number of random variables. Let X1 , X2 , . . . be
i.i.d. random variables (taking non-negative integer values), and let N be another random
variable, also taking non-negative integer values, independent of the sequence Xi . Define
S = X1 + · · · + XN . Then we can write the generating function of S in terms of the common
generating function of the Xi and the generating function of N by

GS (z) = E z S

= E z X1 +···+XN


 

= E E z X1 +···+XN N

N 
= E E z X1
 
N
= E (GX (z))

= GN (GX (z)) .

3.2 Moment generating functions

The probability generating function is well-adapted for handling random variables which
take non-negative integer values. To treat random variables with general distribution, we
now introduce two related objects, the moment generating function and the characteristic
function.
The moment generating function of a random variable X is defined by

MX (t) := E etX .


(3.1)

(Note that we could obtain the moment generating function by substituting z = et in the
definition of the probability generating function above. An advantage of this form is that we
can conveniently consider an expansion around t = 0, whereas the expansion around z = 0,
convenient when the random variables took only non-negative integer values, no longer gives
a power series in the general case.)
For the same reason as for probability generating functions, the mgf of a sum of indepen-
dent random variables is the product of the mgfs:

Theorem 3.3.

(a) If Y = aX + b, then MY (t) = ebt MX (at).

(b) Let X1 , . . . , Xn be independent random variables, with mgfs MX1 , . . . , MXn . Then the
mgf of their sum is given by

MX1 +···+Xn (t) = MX1 (t) . . . MXn (t).

Probability Part A, version of 2017/1/9 22

Proof. (a): easy exercise. Part (b) is also straightforward:

MX1 +···+Xn (t) = E etX1 +···+tXn

= E etX1 . . . etXn

= E etX1 . . . E etXn (by independence)

= MX1 (t) . . . MXn (t).

An immediate disadvantage of the moment generating function is that it may not be well
defined. If the positive tail of the distribution is too heavy, the expectation in the definition
in (3.1) may be infinite for all t > 0: while if the negative tail is too heavy, the expectation
may be infinite for all t < 0.
For the moment generating function to be useful, we will require E et0 |X| < ∞ for some
t0 > 0. That is, X has “finite exponential moments” of some order (equivalently, the tails
of the distribution function decay at least exponentially fast). Then (exercise!) the moment
generating function is finite for all t ∈ (−t0 , t0 ), and also all the moments E X k are finite.
Most of the classical distributions that we have looked at are either bounded or have
tails that decay at least exponentially (for example uniform, geometric/exponential, normal,
Poisson...). However, distributions with heavier tails are also of great importance, especially
in many modelling contexts. For those distributions, the moment generating function is of
no use; however, we can consider a variant of it, the characteristic function (see later).
The next result explains the terminology “moment generating function”; the mgf of X
can be expanded as a power series around 0, in which the coefficients are the moments of X.

Theorem 3.4. Suppose MX (t) is finite for |t| ≤ t0 , for some t0 > 0. Then
P∞ k (X k )
(a) MX (t) = k=0 t Ek! for |t| ≤ t0 .
(k)
(b) MX (0) = E (X k ).

Informal proof.

MX (t) = E (etX )
(tX)2 (tX)3
 
= E 1 + tX + + + ...
2! 3!
t E (X ) t3 E (X 3 )
2 2
= 1 + tE(X) + + + ...,
2! 3!
using linearity of expectation. This gives (a) and taking derivatives at 0 gives (b). Exchanging
expectation with an infinite sum, as we did here, really needs extra justification. In this case
there is no problem (for example, it is always fine in the case where the sum of the absolute
values also has finite expectation – in this case this gives E e|tX| < ∞ which is easily seen to
be true); but we don’t pursue it further here.

The following uniqueness and continuity results will be key to our applications of the
moment generating function.
Probability Part A, version of 2017/1/9 23

Theorem 3.5. If X and Y are random variables with the same moment generating function,
which is finite on [−t0 , t0 ] for some t0 > 0, then X and Y have the same distribution.

Theorem 3.6. Suppose Y and X1 , X2 , . . . are random variables whose moment generating
functions MY and MX1 , MX2 , . . . are all finite on [−t0 , t0 ] for some t0 > 0. If

MXn (t) → MY (t) as n → ∞, for all t ∈ [−t0 , t0 ],

then
d
Xn → Y as n → ∞.

The proofs of the uniqueness and continuity results for mgfs are beyond the scope of
the course. They correspond to an inversion theorem from Fourier analysis, by which the
distribution function of X can be written in a suitable way as a linear mixture over t of terms
E eitX .

Example 3.7. Find the moment generating function of the exponential distribution with
parameter λ.
Solution:

M (t) = E (etX )
Z ∞
= etx f (x)dx
Z0 ∞
= λetx e−λx dx
0
Z ∞
λ
= (λ − t) exp−(λ−t)x dx
λ−t 0
λ
= for t ∈ (−∞, λ).
λ−t
In the last step we used the fact that the integrand is the density function of a random
variable, namely one with Exp(λ − t) distribution, so that the integral is 1.

Example 3.8. Find the moment generating function of a random variable with N (µ, σ 2 )
distribution. If Y1 ∼ N (µ1 , σ12 ) and Y2 ∼ N (µ2 , σ22 ) are independent, show that Y1 + Y2 ∼
N (µ1 + µ2 , σ12 + σ22 ).
Solution: Let X ∼ N (µ, σ 2 ). Then X = σZ + µ, where Z is standard normal. We have

MZ (t) = E (etZ )
Z ∞  2
1 −z
= exp(tz) √ exp dz
−∞ 2π 2
Z ∞
−(z 2 − 2tz)
 
1
= √ exp dz
−∞ 2π 2
Z ∞  2
−(z − t)2
 
t 1
= exp √ exp dz
−∞ 2 2π 2

(this is “completing the square”)

∞
t2 −(z − t)2
 Z  
1
= exp √ exp dz
2 −∞ 2π 2
Probability Part A, version of 2017/1/9 24

2
= et /2

(the same trick as before: the integrand is the density function of N (t, 1) so the integral is 1).
2 2
Then from the first part of Theorem 3.3, MX (t) = eµt MZ (σt) = eµt+σ t /2 .
For the second part,

MY1 +Y2 (t) = MY1 (t)MY2 (t)

2 2
/2 µ2 t+σ22 t2 /2
= eµ1 t+σ1 t e
2 2 2
= e(µ1 +µ2 )t+(σ1 +σ2 )t /2
.

Since this is the mgf of N (µ1 + µ2 , σ12 + σ22 ), and it is finite on an interval [−t0 , t0 ] (in fact,
for all t), the uniqueness theorem for mgfs tells us that indeed that must be the distribution
of Y1 + Y2 .

3.2.1 Proof of WLLN and CLT using moment generating functions

Let X1 , X2 , . . . be a sequence of i.i.d. random variables with finite exponential moments of
some order (i.e. their moment generating function is finite on some interval containing the
origin in its interior).
Let Sn = X1 + X2 + · · · + Xn . We can use moment generating functions to give a very
compact proof of the Central Limit Theorem for the sequence Sn (and, even more simply, the
Weak Law of Large Numbers).
Let the Xi have mean µ and variance σ 2 , and let M be their moment generating function.

Weak law of large numbers

From Taylor’s Theorem and the expansion of M as a power series around 0 (Theorem 3.4)
we can write, as h → 0,

M (h) = M (0) + hM 0 (0) + o(h) (3.2)

= 1 + hµ + o(h). (3.3)

Let M n be the mgf of Sn /n. Using the independence of the Xi , we have

M n (t) = E (etSn /n )
 
= E etX1 /n . . . etXn /n
n
= (M (t/n))
 n
t
= 1 + µ + o(t/n) as n → ∞
n
→ etµ as n → ∞.

But etµ is the mgf of a random variable which takes the constant value µ with probability 1.
d
From the continuity theorem for mgfs, Sn /n → µ as n → ∞, and we have proved the weak
law of large numbers.
Probability Part A, version of 2017/1/9 25

Central limit theorem

Let Yi = Xi − µ, and let MY be the mgf of the common distribution of the Yi . Taking one
more term in the Taylor expansion, we have that as h → 0,

h2 00
MY (h) = MY (0) + hMY0 (0) + M (0) + o(h2 )
2 Y
h2
= 1 + hE (Y ) + Var(Y ) + o(h2 )
2
= 1 + h2 σ 2 /2 + o(h2 ).
−µn
Sn √
Let M
fn be the mgf of
σ n
. Then we have
  
t(Sn − µn)
M
fn (t) = E exp √
σ n
    
t(X1 − µ) t(Xn − µ)
= E exp √ . . . exp √
σ n σ n
 n
t
= MY √
σ n
 2 n
t2

t
= 1+ +o as n → ∞
2n n
 2
t
→ exp as n → ∞.
2

But the last line is the mgf of N (0, 1). Using the continuity theorem again,

Sn − µn d
√ → N (0, 1),
σ n

and we have proved the CLT.

3.3 Using moment generating functions to prove tail bounds

Using a version of Markov’s inequality applied to a random variable like etX , we can get
bounds on the tail of a distribution in terms of the moment generating function which are
much better than we would get from, for example, Chebyshev’s inequality, which is the
application of Markov’s inequality to the random variable (X − µ)2 . (Of course, this can only
work if the moment generating function exists!)
For example, we can apply this to simple random walk. Let Xi be i.i.d. taking values −1
and 1 with probability 1/2 each. Let Sn = X1 + · · · + Xn , so that Sn is the position of simple
random walk on Z after n steps.
We know from the central limit theorem that, for large n, Sn is typically on the order of
√
n. So an event like {|Sn | > na}, for some a > 0, ought to have probability which gets small
as n → ∞.
First we bound the probability using Chebyshev. We have E Sn = 0 and Var Sn = n. So
Var Sn
P(|Sn | > na) ≤
(na)2
Probability Part A, version of 2017/1/9 26

1
= .
na2
This goes to 0 as desired but not very quickly!
Let’s try instead using the moment generating function. We have

et + e−t
E etXi =
2
= cosh t
 2
t
≤ exp for all t.
2

(The inequality cosh t < exp(t2 /2) can be checked directly by expanding the exponential
functions and comparing coefficients in the power series).
For t > 0, we can now write

P(Sn > na) = P (exp(tSn ) > exp(tna))

E exp(tSn )
≤ (this is from Markov’s inequality)
exp(tna)
 n
E exp(tXi )
=
exp(ta)


n
≤ exp t2 /2 − ta .

Note that this is true for any positive t, so we are free to choose whichever one we like.
Naturally, we want to minimise the RHS. It’s easy to check (just differentiate) that this is
done by choosing t = a, which gives

P(Sn > na) ≤ exp −na2 /2 .

By symmetry the bound on P(Sn < −na) is exactly the same. Combining the two we get

P(|Sn | > na) ≤ 2 exp −na2 /2 .

This decays much quicker than the bound from Chebyshev above!

3.4 Characteristic functions

The characteristic function is defined by replacing t by it in the definition of the moment
generating function. The characteristic function of X is given by

φX (t) := E (eitX ),

for t ∈ R. We can write

φX (t) = E (cos(tX)) + iE (sin(tX)).

As a result we can see that the characteristic function is finite for every t, whatever the
distribution of X. In fact, |φX (t)| ≤ 1 for all t.
This means that many of the results for the moment generating function which depended
on exponential tails of the distribution have analogues for the characteristic function which
Probability Part A, version of 2017/1/9 27

hold for any distribution. Just as before we have φX+Y (t) = φX (t)φY (t). The uniqueness
and continuity theorems given for mgfs hold in a similar way for characteristic functions. The
Taylor expansion of the characteristic function around the origin involves the moments of the
distribution in a similar way (except now with an added factor of ik for the kth term):

E X2 E Xk
φX (t) = 1 + itE X + i2 t2 + · · · + ik tk + o(tk ) (3.4)
2 k!
as t → 0, whenever E X k is finite. Hence by following exactly the same strategy, we could give
a proof of the central limit theorem using characteristic functions instead of mgfs. This would
now prove the CLT without any additional assumption on the distribution (only finiteness of
the variance is needed). Apart from working with complex power series instead of real power
series, there are no additional complications when translating the proof from mgfs to cfs.
When the mgf is finite in an interval containing the origin in its interior, the theory of
analytic continuation of complex functions allows us to obtain the characteristic function
easily, by replacing t with it in the mgf.

Example 3.9. (a) The mgf of N (0, 1) is exp(t2 /2), and the cf is exp((it)2 /2) = exp(−t2 /2).

(b) The mgf of Exp(1) is 1/(1 − t), and the cf is 1/(1 − it).
1
(c) Suppose X has Cauchy distribution with density f (x) = π(1+x 2 ) . The moment generating

function is infinite for all t 6= 0 (in fact, even the mean is infinite – exercise). The
characteristic function is given by
Z ∞
itX eitx
φX (t) = E e = 2
dx
−∞ π(1 + x )

and this can be evaluated by contour integration to give e−|t| .

Note that φX is not differentiable at 0; from (3.4), this corresponds to the fact that the
mean does not exist.
In fact, consider X1 , X2 , . . . Xn i.i.d. Cauchy, and Sn = X1 + · · · + Xn . Then
 n  n
t
φSn /n (t) = φ = e−|t|/n = e−|t| = φX (t).
n

So Sn /n and Xi have the same distribution! The law of large numbers and the CLT don’t
apply (since the mean does not exist).

3.4.1 Comparing moment generating functions and characteristic

functions
Question M4(a)(ii) on Part A paper AO2 from 2011 asks:

State one purpose for which you should use the characteristic function rather than the
moment generating function, and one purpose for which you would want to use the
moment generating function rather than the characteristic function.
Probability Part A, version of 2017/1/9 28

The previous section gives an obvious answer to the first part of the question: when the
distribution does not have exponentially decaying tails, the moment generating function is
not useful but the characteristic function certainly is (to prove the CLT, for example). In
the other direction, one could refer to the use of the mgf to give bounds on the tail of a
distribution. In Section 3.3 we did this using Markov’s inequality applied to the random
variable etX ; replacing this with eitX would give nothing sensible, since that function is not
real-valued, let alone monotonic.
 4

Joint distribution of continuous random

variables

4.1 Review of jointly continuous random variables

The joint cumulative distribution function of two random variables X and Y is defined
by
FX,Y (x, y) = P(X ≤ x, Y ≤ y).

X and Y are said to be jointly continuous if their joint cdf can be written as an integral:
Z x Z y
FX,Y (x, y) = f (u, v)du dv.
u=−∞ v=−∞

Then f is said to be the joint pdf of X and Y , often written as fX,Y . As in the case of a
single random variable, we might more properly say “a joint pdf” rather than “the joint pdf”
because we can, for example, change the value of f at finitely many points without changing
the value of any integrals of f . But it’s natural to put

∂2
fX,Y (x, y) = FX,Y (x, y)
∂x∂y

whenever FX,Y is differentiable at (x, y).

For general (suitably nice) sets A ⊂ R2 we have
Z Z
P ((X, Y ) ∈ A) = fX,Y (x, y)dx dy.
A

We can recover the distribution of one of the random variables X or of Y by integrating

over the other one. (In this context the distribution of one of the variables is called the
marginal distribution).
Z ∞
fX (x) = fX,Y (x, y)dy
y=−∞
Z ∞
fY (y) = fX,Y (x, y)dx
x=−∞

29
Probability Part A, version of 2017/1/9 30

A function of X and Y is itself a random variable. Its expectation is given by

Z ∞ Z ∞
E h(X, Y ) = h(x, y)fX,Y (x, y)dx dy.
x=−∞ y=−∞

Recall that X and Y are independent if FX,Y (x, y) = FX (x)FY (y) for all x, y. Equiva-
lently, the joint density can be written as a product:

fX,Y (x, y) = fX (x)fY (y).

All the above can be naturally generalised to describe the joint distribution of more than
two random variables.

4.2 Change of variables

Often there is more than one natural coordinate system in which to view a model. We have
the following change of variables result:

Theorem 4.1. Suppose T : (x, y) 7→ (u, v) is a one-to-one mapping from some domain
D ⊆ R2 to some range R ⊆ R2 .
Define the Jacobian J as a function of (u, v) by

 ∂x ∂y 
 
 
 ∂u ∂u  
 ∂x ∂y ∂x ∂y 

J =   =  − .
  
∂u ∂v ∂v ∂u
 ∂x ∂y 
 
 
∂v ∂v
Assume that the partial derivatives involved exists and are continuous.
If X, Y have joint probability density function fX,Y , then the random variables U, V defined
by (U, V ) = T (X, Y ) are jointly continuous with joint probability density function fU,V given
by 

f
X,Y x(u, v), y(u, v) J(u, v) if (u, v) ∈ R
fU,V (u, v) = .
0 otherwise

Proof. The proof is simple using the familiar formula for change of variables in an integral.
Suppose that A ⊆ D and T (A) = B. Then, since T is one-to-one,

P ((U, V ) ∈ B) = P ((X, Y ) ∈ A)
Z Z
= fX,Y (x, y)dx dy
Z ZA
= fX,Y (x(u, v), y(u, v)) J(u, v)du dv.
B

Hence the final integrand is the joint pdf of (U, V ).

The formula for change of variables in the integral appeared in various contexts last year.
Recall the general idea: after a suitable translation, the transformation T looks locally like a
linear transformation whose matrix is the matrix of partial derivatives above. We know that
Probability Part A, version of 2017/1/9 31

the factor by which the area of a set changes under a linear transformation is given by the
determinant of the matrix of the transformation. So, locally, the Jacobian J(u, v) gives the
ratio between the area of a rectangle (x, x + dx) × (y, y + dy) and its image under T (which
is a parallelogram). Since we want the probability to stay the same, and probability is area
times density, we should rescale the density by the same amount J(u, v).

Example 4.2. Let X, Y be i.i.d. exponentials with rate λ. Let U = X/(X + Y ), V = X + Y .

What is the joint distribution of (U, V )?
Solution:

fX,Y (x, y) = λe−λx λe−λy

= λ2 e−λ(x+y)

for (x, y) ∈ R2+ . The transformation (u, v) = (x/(x + y), x + y) takes R2+ to [0, 1] × R+ . It is
inverted by x = uv, y = v(1 − u). The Jacobian is given by

 ∂x ∂y 
 
   
 ∂u ∂u  v −v 
J =   = 
  
u 1 − u

 ∂x ∂y 
 
 
∂v ∂v
 
=  − v(1 − u) − uv 
= v.

So we have


fU,V (u, v) = fX,Y x(u, v), y(u, v) J


= λ2 e−λ x(u,v)+y(u,v) J
= vλ2 e−λv

for (u, v) ∈ [0, 1] × R+ .

This factorises into a product of a function of u and a function of v (the function of u is
trivial). So U and V are independent, with

fU (u) = 1, u ∈ [0, 1]
fV (v) = λ2 ve−λv , v ≥ 0

So U ∼ U [0, 1] and V ∼ Gamma(2, λ), independently.

Example 4.3. Let X and Y be independent Exp(λ) as in the previous example, and now
let U = X − Y , V = X + Y . This transformation takes R2+ to the set {(u, v) : |u| < v}. The
inverse transformation is
u+v v−u
x= , y=
2 2
with Jacobian  
 1 1
 2 2  1
J =  1 1 = .
− 2 2  2
Probability Part A, version of 2017/1/9 32

(Notice that any linear transformation always has constant Jacobian). So we have

u+v v−u


f
X,Y 2 , 2 J for |u| < v
fU,V (u, v) =
0 otherwise

 1 λ2 expλv for |u| < v
= 2
0 otherwise .

It looks like the pdf factorises into a product as in the previous example. But here this is
not really the case! – because of the restriction to |u| < v. In fact, U and V could not be
independent here, otherwise we could not have P(|U | < V ) = 1.
From the previous example we already know that V ∼ Gamma(2, λ). What is the marginal
distribution of U ?
Z ∞
1 2 −λv
fU (u) = λ e dv
v=|u| 2
 ∞
1
= − λe−λv
2 |u|
1 −λ|u|
= λe .
2
We see that the distribution of U is symmetric around 0, and by adding the density at u and
−u, the distribution of |U | has pdf λe−λ|u| and so again has Exp(λ) distribution.

Example 4.4 (General formula for the sum of continuous random variables). If X and Y
are jointly continuous with density function fX,Y , what is the distribution of X + Y ? We can
change variables to U = X + Y, V = X. This transformation has Jacobian 1 (check!), and we
obtain fU,V (u, v) = fX,Y (v, u − v).
To obtan the marginal distribution of X + Y , which is U , we integrate over v:
Z ∞
fX+Y (u) = fX,Y (v, u − v)dv.
−∞

An important case is when X and Y are independent. Then we obtain the convolution
formula: Z ∞
fX+Y (u) = fX (v)fY (u − v)dv.
−∞

4.2.1 Multivariate distributions

Everything above can be generalised to the case of the joint distribution of n > 2 random
variables. The Jacobian is now the determinant of an n × n matrix.

4.3 Multivariate normal distribution

Let Z1 , Z2 , . . . , Zn be i.i.d. standard normal random variables. Their joint density function
can be written as
n  2
Y 1 z
fZ (z) = √ exp − i
i=1
2π 2
Probability Part A, version of 2017/1/9 33

 
1 1 T
= exp − z z .
(2π)n/2 2
Define W1 , . . . , Wn by      
W1 Z1 µ1
 W2   Z2   µ2 
     
 .  = A .  +  . 
 .   .   . 
 .   .   . 
Wn Zn µn
where A is some n × n matrix.
Assume A is invertible. Then by change of variables (the Jacobian is constant) we get
 
1 1 T T −1


fW (w) = exp − (w − µ) AA (w − µ) .
(2π)n/2 | det A| 2
The matrix Σ := AAT is the covariance matrix in the sense that Cov(Wi , Wj ) = (AAT )ij
(check, e.g. for n = 2 if you want an easy case). W1 , . . . , Wn are said to have the multivariate
normal distribution with mean vector µ and covariance matrix Σ.
For the case n = 2, one can manipulate to obtain (with X = W1 , Y = W2 )

fX,Y (x, y)
(x − µX )2 2ρ(x − µX )(y − µY ) (y − µY )2
  
1 1
= exp − 2 − +
2 (1 − ρ2 ) σY2
p
2πσX σY 1 − ρ2 σX σX σY
2
where σX and σY2 are the variances of X and Y , µX and µY are the means, and ρ is the
correlation coeffecient between X and Y which is defined by
Cov(X, Y )
ρ=
σX σY
which lies in (−1, 1).
Note that
(1) The density depends only on µX , µY , σX , σY and ρ.

(2) X and Y are independent ⇔ ρ = 0. (⇒ is true for any joint distribution; ⇐ is a special
property of joint normal.)
A special case is the standard bivariate normal where σX = σY = 1 and µX = µY = 0.
Then  2
x − 2ρxy + y 2

1
fX,Y (x, y) = exp − .
2(1 − ρ2 )
p
2π 1 − ρ2
!
1 ρ
In this case Σ = .
ρ 1

4.4 Conditional densities

The basic definition of conditional probability: for two events A and B with P(A) > 0, the
conditional probability of B given A is
P(A ∩ B)
P(B|A) := . (4.1)
P(A)
Probability Part A, version of 2017/1/9 34

Applying this to the distribution of a random variable, we have for example

P({X ≤ x} ∩ A
P(X ≤ x|A) = .
P(A)

The left-hand side is a cumulative distribution function. It gives the conditional distribu-
tion of X, given A. We might denote it by FX|A (x).
In the case where X is discrete, we can write the conditional probability mass function:

pX|A (x) = P(X = x|A).

If X is continuous, we can differentiate the conditional distribution function to get a condi-

tional density function fX|A (x), and then for a set C,
Z
P(X ∈ C|A) = fX|A (x)dx.
x∈C

The conditional expectation of X given A is the expectation of the conditional distribution,

which is given by X
E (X|A) = pX|A (x)
x

in the discrete case, and by Z

E (X|A) = xfX|A (x)dx

in the continuous case.

Example 4.5. Suppose X and Y are independent random variables which both have uniform
distribution on [0, 1]. Find the conditional distribution and conditional expectation of Y given
X + Y > 1.
Solution:
P(Y < y, X + Y > 1)
P(Y < y|X + Y > 1) = .
P(X + Y > 1)

Since X, Y are uniform on the square [0, 1]2 , the probability of a set is equal to its area.
The set {x + y > 1} has area 1/2, while for fixed y, the set {(x, v) : v < y, x + v > 1} has
area y 2 /2.
So the distribution function of Y given X + Y > 1 is F (y) = (y 2 /2)/(1/2) = y 2 , and the
conditional density is 2y on [0, 1], and 0 elsewhere.
R1
The conditional expectation E (Y |X + Y > 1) is 0 y × 2y dy = 2/3.

A common way in which conditional distributions arise is when we have two random
variables X and Y with some joint distribution; we observe the value of X and want to know
what this tells us about the value of Y . That is, what is the conditional distribution of Y
given X = x?
When X is a discrete random variable, everything works fine; since P(X = x) will be
positive, we can use the approach above.
However, if X is continuous, then P(X = x) will be 0 for every x. Now we have a problem,
since if the event A in (4.1) has probability 0, then the definition makes no sense.
Probability Part A, version of 2017/1/9 35

To resolve this problem, rather than conditioning directly on {X = x}, we look at the
distribution of Y conditioned on {x ≤ X ≤ x + }. If the joint distribution is well-behaved
(as it will be in all the cases that we wish to consider), we can obtain a limit as  ↓ 0, which
we define as the distribution of Y given X = x.
As  → 0, we have
Z y Z x+
fX,Y (u, v)du dv
v=−∞ u=x


P Y ≤y x≤X ≤x+ =
 Z x+
fX (u)du
u=x
Z y
 fX,Y (x, v)dv
v=−∞
∼
fX (x)
Z y
fX,Y (x, v)
= dv. (4.2)
v=−∞ fX (x)

So we define FY |X (y|x), the conditional distribution function of Y given X = x, as

the right-hand side of (4.2).
Differentiating with respect to y, we obtain the conditional density function of Y
given X = x, written as fY |X (y|x):

fX,Y (x, y)
fY |X (y|x) = .
fX (x)

These definitions make sense whenever fX (x) > 0. In that case, note that fY |X (.|x) is
R∞
indeed a density function, because we have defined fX (x) = −∞ fX,Y (x, y)dy. (Notice that
the denominator fX (x) doesn’t involve y at all; it’s just a normalising constant).
The idea is that the following two procedures are equivalent:

(1) generate (X, Y ) according to the joint density function fX,Y ;

(2) first generate X according to the density function fX , and then having observed X = x,
generate Y according to the density function fY |X (.|x).

Example 4.6 (Simple example). Let (X, Y ) be uniform on the triangle {0 < y < x < 1}.
Then 
2 0 < y < x < 1
fX,Y (x, y) = .
0 otherwise

For the conditional density of Y given X = x,

fX,Y (x, y)
fY |X (y|x) =
fX (x)

2/f (x) 0 < y < x
X
= ,
0 otherwise

provided x ∈ (0, 1). We don’t need to calculate fX (x), since it is just a normalising constant.
Since the conditional density function of Y is constant in y, we see that Y is uniform on [0, x],
Probability Part A, version of 2017/1/9 36

with distribution function given by


0 y<0


y
FY |X (y|x) = 0≤y≤x.
x

1 y>x


The conditional mean of Y given X = x is x/2.

Example 4.7 (Bivariate normal). Let X and Y be jointly normal with means µ1 and µ2
respectively, variances σ12 and σ22 respectively, and correlation coefficient ρ. What is the
conditional distribution of Y given X = x?
Rather than working directly from the joint density function, we can proceed by writing
Y as the sum of two terms, one which is a function of X and one which is independent of X.
First let us write X and Y as functions of independent standard normals Z1 and Z2 . If
we put

X = σ1 Z1 + µ1
p
Y = ρσ2 Z1 + 1 − ρ2 σ2 Z2 + µ2

then indeed X and Y have the desired means, variances and covariance (check!).
Then we can write
σ2 p
Y = ρ (X − µ1 ) + 1 − ρ2 σ2 Z2 + µ2 .
σ1
The first term is a function of X and the second term, involving only Z2 , is independent of
X.
So conditional on X = x, the distribution of Y is the distribution of
σ2 p
ρ (x − µ1 ) + 1 − ρ2 σ2 Z2 + µ2 ,
σ1
which is normal with mean ρ σσ21 (x − µ1 ) + µ2 and variance (1 − ρ2 )σ22 .
Note the way the variance of this conditional distribution depends on ρ. We say that ρ2
is the “amount of the variance of Y explained by X”. Consider the extreme cases. If ρ = ±1,
then the conditional variance is 0. That is, Y is a function of X and once we observe X, there
is no longer any uncertainty about the value of Y . If ρ = 0, the conditional variance and the
unconditional variance are the same; observing X tells us nothing about Y .

4.5 Cautionary tale

The definition above of conditional distribution given the value of a continuous random vari-
able makes sense in context, but keep in mind that conditioning directly on events on prob-
ability zero is not valid, and as a result the objects involved are not robust to seemingly
innocent manipulation! Consider the following example:
Example 4.8 (Borel’s paradox). Consider the uniform distribution on the half-disc C =
{(x, y) : y ≥ 0, x2 + y 2 ≤ 1}. The joint density of X and Y is given by

 2 (x, y) ∈ C
f (x, y) = π .
0 otherwise
Probability Part A, version of 2017/1/9 37

What is the conditional distribution of Y given X = 0? Its density is given by

2/π
fY |X (y|X = 0) =
fX (0)

for y ∈ [0, 1], and 0 elsewhere. So the distribution is uniform on [0, 1] (we don’t need to
calculate fX (0) to see this, since it’s only a normalising constant).
We could change variables and represent the same distribution in polar coordinates. Then
R and Θ are independent; R has density 2r on [0,1] and Θ is uniform on [0, π). (See first
question on problem sheet 2 for the transformation to polar coordinates. But in this case
where the density of X, Y is uniform on a set, one can also easily derive the joint distribution
of R and Θ directly by considering areas of subsets of the set C).
Note that the events {X = 0} and {Θ = π/2} are the same.
What is the conditional distribution of R given Θ = π/2? Since R and Θ are independent,
it still has density 2r on [0, 1]. This is not uniform on [0, 1].
But when X = 0, i.e. when Θ = π/2, R and Y are the same thing. So the distribution of
R given Θ = π/2 ought to be the same as the distribution of Y given X = 0, shouldn’t it?
What is happening is that, although the events {X = 0} and {Θ = π/2} are the same,
it is not the case that the events {|X| < } and {|Θ − π/2| < } are the same. When we
condition X to be within  of 0, we restrict to a set which is approximately a rectangle (the
left-hand picture below). However, when we condition Θ to be near π/2, we restrict to a thin
sector of the circle, which is approximately a triangle (the right-hand picture below). In the
second case, we bias the point chosen to lie higher up. As  → 0, this bias persists; the two
limits are not the same!

What this “paradox” illustrates is that conditioning for continuous random variables in-
volves a limit, and that it can be important exactly how the limit is taken. The procedure
whereby we generate X from fX and then Y from fY |X makes sense in terms of a particular
set of variables; but the conditional densities involved are not robust to a change of variables.
 5

Markov chains: Introduction

Let Xn , n = 0, 1, 2, . . . be a “random process”, taking values in some set I called the state
space. That is, X0 , X1 , X2 , . . . are random variables with Xn ∈ I for all n.
Often Xn represents some quantity evolving in time. So far we have been working with
random variables taking values which are real numbers of some kind, but there is no problem
in considering a more general state space. For example, we might consider processes of the
following kind:

• I = Z2 , Xn =position at nth step of a “random walk” on the two-dimensional lattice.

• I = {A, B, C, . . . , a, b, c, . . . , ., ?, !, . . . }, Xn =nth character in a text or in an email.

• I = {C, G, A, T } (representing cytosine, guanine, adenine, thymine, the four bases of

DNA), Xn =base appearing in nth position in a DNA sequence.

We will assume that the state space I is finite or countably infinite (i.e. discrete). A
(probability) distribution on I is a collection λ = (λi , i ∈ I) with λi ≥ 0 for all i, and
P
λi = 1. This is really just the same idea as the probability mass function of a discrete
random variable. We will often think of λ as a row vector. We will say that a random
variable Y taking values in I has distribution λ if P(Y = i) = λi for all i.

5.1 Markov chains

Let X = (X0 , X1 , X2 , . . . ) be a sequence of random variables taking values in I. The process
X is called a Markov chain if for any n ≥ 0 and any i0 , i1 , . . . , in+1 ∈ I,



P Xn+1 = in+1 Xn = in , . . . , X0 = i0 = P Xn+1 = in+1 Xn = in . (5.1)

(To be precise, we should restrict (5.1) to cases where these conditional probabilities are
well-defined, i.e. where the event {Xn = in , . . . , X0 = i0 } has positive probability.)


The Markov chain is called (time) homogeneous if in addition P Xn+1 = j Xn = i
depends only on i and j, not on n. In that case we write


pij = P Xn+1 = j Xn = i

38
Probability Part A, version of 2017/1/9 39

(or we will often write pi,j rather than pij , according to convenience). The quantities pij are
known as the transition probabilities of the chain.
We will work almost always with homogeneous chains. To describe such a chain, it’s
enough to specify two things:

• the initial distribution λ of X0 . For each i ∈ I, λi = P(X0 = i).

• the transition matrix P = (pij )i,j∈I .

P is a square (maybe infinite) matrix, whose rows and columns are indexed by I. P is a
“stochastic matrix” which means that all its entries are non-negative and every row sums
to 1. Equivalently, every row of P is a probability distribution. The ith row of P is the
distribution of Xn+1 given Xn = i.

Theorem 5.1. For i0 , i1 , . . . , in ∈ I,

P (X0 = i0 , X1 = i1 , . . . , Xn = in ) = λi0 pi0 i1 pi1 i2 . . . pin−1 in .

Proof.

P (X0 = i0 , X1 = i1 , . . . , Xn = in )



= P (X0 = i0 ) P X1 = i1 X0 = i0 P X2 = i2 X1 = i1 , X0 = i0 × . . .


· · · × P Xn = in Xn−1 = in−1 , . . . , X0 = i0




= P (X0 = i0 ) P X1 = i1 X0 = i0 P X2 = i2 X1 = i1 . . . P Xn = in Xn−1 = in−1
= λi0 pi0 i1 pi1 i2 . . . pin−1 in ,

where we used the definition of a Markov chain to get the penultimate line.

If X is a Markov chain with initial distribution λ and transition matrix P , we will some-
times write “X ∼ Markov(λ, P )”.
Markov chains are “memoryless”. If we know the current state, any information about
previous states is irrelevant to the future evolution of the chain. We can say that “the future
is independent of the past, given the present”. This is known as the Markov property.
Notation: it will be convenient to write Pi for the distribution conditioned on X0 = i. For
example Pi (X1 = j) = pij . Similarly E i for expectation conditioned on X0 = i.

5.2 n-step transition probabilities

(n) 

Write pij = P Xk+n Xk = i . This is an n-step transition probability of the Markov
chain.

Theorem 5.2. (Chapman-Kolmogorov equations)

(n+m) P (n) (m)
(i) pij = k∈I pik pkj .
(n)
(ii) pij = (P n )i,j .
Probability Part A, version of 2017/1/9 40

Here P n is the nth power of the transition matrix. As ever, matrix multiplication is given
P
by (AB)i,j = k (A)i,k (B)k,j , whether the matrices are finite or infinite.

Proof. (i)

X 


P Xn+m = j X0 = i = P Xn = k X0 = i P Xn+m = j Xn = k, X0 = i
k
X 


= P Xn = k X0 = i P Xn+m = j Xn = k
k

(using the Markov property)

(n) (m)
X
= pik pkj .
k

(ii) Inductively,
(2)
X
pik pkj = P 2


pij = i,j
k
(3) (2) (1)
X X
P2 = P 2P = P3





pij = pik pkj = i,k
P k,j i,j i,j
,
k k

and so on.

Theorem 5.3. Let λ be the initial distribution (i.e. the distribution of X0 ). Then the distri-
bution of X1 is λP , and more generally the distribution of Xn is λP n .

Here we are thinking of λ as a row vector, so that λP n is also a row vector; (λA)i =
P
k λk Aki as usual, whether the dimensions are finite or infinite.

Proof.
X 

P (X1 = j) = P (X0 = i) P X1 = j X0 = i
i
X
= λi pij
i

= (λP )j ,

(n)
and similarly for Xn with pij and P replaced by pij and P n .

Using this result and the Markov property it’s easy to get the following property: if
(X0 , X1 , X2 , . . . ) is a Markov chain with initial distribution λ and transition matrix P , then
(X0 , Xk , X2k , . . . ) is a Markov chain with initial distribution λ and transition matrix P k .

Example 5.4 (General two-state Markov chain). Let I = {1, 2} and

!
1−α α
P = .
β 1−β

What is pn11 ? Two approaches:

Probability Part A, version of 2017/1/9 41

(1) P has eigenvalues 1 and 1 − α − β (check! Every Markov transition matrix has 1 as an
eigenvalue – why?). So we can diagonalise:
!
−1 1 0
P =U U,
0 1−α−β
!
n −1 1 0
P =U U.
0 (1 − α − β)n
We get (P n )11 = A + B(1 − α − β)n for some constants A and B.
(0) (1)
Since we know p11 = 1 and we have p11 = 1 − α, we can solve for A and B to get
(n) β α
p11 = + (1 − α − β)n . (5.2)
α+β α+β
(2) Alternatively, we can condition on the state at step n − 1:
(n) (n−1) (n−1)
p11 = p11 (1 − α) + p12 β
 
(n−1) (n−1)
= p11 (1 − α) + 1 − p11 β
(n−1)
= (1 − α − β)p11 + β.
(n)
This gives a linear recurrence relation for p11 , which we can solve using standard
methods to give (5.2) again.

5.3 A few examples

Random walk on a cycle

I = {0, 1, 2, . . . , M − 1}. At each step the walk increases by 1 (mod M ) with probability p
and decreases by 1 (mod M ) with probability 1 − p. That is,

p

 if j ≡ i + 1 mod M
pij = 1−p if j ≡ i − 1 mod M ,


0 otherwise


or  
0 p 0 0 ··· 0 1−p
1 − p 0 p 0 ··· 0 0 
 
0 1−p 0 p ··· 0 0 .
 
P =
 ..
 
.. 
 . . 
p 0 0 0 ··· 1−p 0

Simple symmetric random walk on Zd

I = Zd . At each step the walk moves from its current site to one of its 2d neighbours chosen
uniformly at random. 
1 if |i − j| = 1
pij = 2d .
0 otherwise
Probability Part A, version of 2017/1/9 42

Card-shuffling
Let I be the set of orderings of 52 cards. We can regard I as the permutation group S52 .
There are many interesting Markov chains on permutation groups. We can think of shuffling
a pack of cards. A simple and not very practical example of a shuffle: at each step, choose
a and b independently and uniformly in {1, 2, . . . , 52} and exchange the cards in positions a
and b. This gives
2



2
if α = βτ for some transposition τ
 52



pαβ = 1 .
if α = β


 52


0 otherwise

5.4 Exploring the Markov property

Let’s look at a few examples of simple processes where the Markov property holds or fails.
We can do this in the context of a simple random walk on Z.
Let Xi be i.i.d. with P(Xi = 1) = p and P(Xi = −1) = 1 − p.
Pn
Let S0 = 0 and Sn = i=1 Xi .
Then:
(1) Xn is a Markov chain. In fact, Xn are i.i.d., which is a stronger property. Given any
history, the next state is equal to 1 with probability p and −1 with probability 1−p. The
!
1−p p
matrix of the chain Xn (with rows and columns indexed by {−1, 1}) is P = .
1−p p

(2) The random walk Sn is also a Markov chain. Its transition probabilities are pi,i+1 = p
and pi,i−1 = p for all i ∈ Z.

(3) Consider the process Mn = max0≤k≤n Sk . Try drawing some possible paths of the pro-
cess Sn , and the corresponding paths of the ”maximum process” Mn . Is this maximum
process a Markov chain?
We can consider two different ways of arriving at the same state. Suppose (M0 , . . . , M4 ) =
(0, 0, 0, 1, 2). This implies S4 = 2 (the maximum process has just increased, so now the
walk must be at its current maximum.) In that case, if the random walk moves up at
the next step, then the maximum will also increase. So

P(M5 = 3|(M0 , . . . , M4 ) = (0, 0, 0, 1, 2)) = p.

Suppose instead that (M0 , . . . , M4 ) = (0, 1, 2, 2, 2). In that case, both S4 = 2 and S4 = 0
are possible (check! – find the corresponding paths). As a consequence, sometimes the
maximum will stay the same at the next step, even when the random walk moves up.
So we have
P(M5 = 3|(M0 , . . . , M4 ) = (0, 1, 2, 2, 2)) < p.

We see that the path to M4 = 2 affects the conditional probability of the next step of
the process. So Mn is not a Markov chain.
Probability Part A, version of 2017/1/9 43

The next result gives a criterion for the Markov property to hold.

Proposition 5.5. Suppose that (Yn , n ≥ 0) is a random process, and for some function f we
can write, for each n,
Yn+1 = f (Yn , Xn+1 ),

where Xn+1 is independent of Y0 , Y1 , . . . , Yn . Then (Yn ) is a Markov chain.

Proof. The idea is that to update the chain, we use only the current state and some “new”
randomness. We have


P Yn+1 = in+1 Yn = in , . . . , Y0 = i0


= P f (in , Xn+1 ) = in+1 Yn = in , . . . , Y0 = i0
= P (f (in , Xn+1 ) = in+1 ) (by independence of Xn+1 from Y0 , . . . , Yn )


= P f (in , Xn+1 ) = in+1 Yn = in (by independence of Xn+1 from Yn )


= P Yn+1 = in+1 Yn = in .

For example, for the simple random walk above, we can put Sn+1 = f (Sn , Xn+1 ), where
f (s, x) = s + x. For the card-shuffling example in the previous section, if Yn ∈ S52 is the
permutation after step n, we can put Yn+1 = f (Yn , Xn+1 ) where for a permutation β and a
transposition τ , f (β, τ ) = βτ , and where (Xn ) is an i.i.d. sequence in which each member is
uniform in the set of transpositions.

5.5 Class structure

Let i, j ∈ I. We say that “i leads to j” and write “i → j” if Pi (Xn = j) > 0 for some n ≥ 0,
(n)
i.e. pij > 0 for some n ≥ 0.
If i → j and j → i then we say “i communicates with j” and write i ↔ j.
Then ↔ is an equivalence relation (check!). It partitions the state space I into commu-
nicating classes.
A class C is called closed if pij = 0 whenever i ∈ C, j ∈
/ C, or equivalently i 6→ j for any
i ∈ C, j ∈ / C. Once the chain enters a closed class, it can never escape from it. If {i} is a
closed class then pii = 1, and i is called an absorbing state. If C is not closed it is called
open.
A chain (or more precisely a transition matrix) for which I consists of a single communi-
cating class is called irreducible. Equivalently, i → j for all i, j ∈ I.

Example 5.6. Let I = {1, 2, 3, 4, 5, 6, 7}. The communicating classes for the transition
Probability Part A, version of 2017/1/9 44

matrix  1 1
0 2 0 0 0 0 2
0 0 1 0 0 0 0
 
1 1
0 0 0 0 0

2 2
 1 1

P =
0 0 0 2 2 0 0
0 0 0 0 0 1 0
 
 
0 0 0 0 1 0 0
0 0 0 0 0 0 1
are {1, 2, 3}, {4}, {5, 6} and {7}. The closed classes are {5, 6} and {7} (so 7 is an absorbing
state). Draw a diagram to visualise the chain!

5.6 Periodicity
Consider the transition matrix  
0 1 0 0 0
1 1
0 0 0

2 2
 1 1

.
0 0 0
 2 2
1 1 1
3 0 3 0 3
0 0 0 1 0
(n)
Again, draw a diagram to visualise the chain. Note that pii = 0 whenever n is odd.
For a general chain and an state i ∈ I, o
the period of the state i is defined to be the greatest
(n) (n)
common divisor of the set n : pii > 0 . (If pii = 0 for all n > 0, then the period is not
defined). All the states in the chain above have period 2.
i is called aperiodic if this g.c.d. is 1 (and otherwise periodic). Equivalently (check!), i
(n)
is aperiodic if pii > 0 for all sufficiently large n.

Fact. All states in a communicating class have the same period.

(n)
Proof. Suppose i ↔ j and d|n whenever pii > 0.
(a) (b)
Since i and j communicate, we can find a and b with pij > 0 and pji > 0. Then also
(a+b)
pii > 0.
(m) (a+m+b)
Suppose pjj > 0. Then also pii > 0.
Then d|a + b and d|a + m + b, son also d|m. o n o
(n) (m)
This demonstrates that the sets n : pii > 0 and m : pjj > 0 have the same divisors,
and hence the same greatest common divisor.

In particular, if a chain is irreducible, then all states have the same period. If this period
is 1, we say that the chain is aperiodic (otherwise we say the chain is periodic).
Remark. Notice that both irreducibility and periodicity are “structural properties” in the
following sense: they depend only on which transition probabilities pij are positive and which
are zero, not on the particular values taken by those which are positive.

Example. Look back at the three examples in Section 5.3 and consider which are irreducible
and which are periodic.
Probability Part A, version of 2017/1/9 45

The random walk on the cycle is irreducible (since every site is accessible from every
other). It has period 2 if M is even, and is aperiodic if M is odd.
The random walk on Zd is irreducible and has period 2 for any d.
The card-shuffling chain is irreducible (because the set of transpositions is a set of gener-
ators for the group S52 ). It is aperiodic, since there is a positive transition probability from
any state to itself.

Remark. Later we will show results about convergence to equilibrium for Markov chains.
The idea will be that after a long time, a Markov chain should more or less “forget where it
started”. There are essentially two reasons why this might not happen: (a) periodicity; for
example if a chain has period 2, then it alternates between, say, “odd” and “even” states;
even an arbitrarily long time, the chain will still remember whether it started at an “odd” or
“even” state. (b) lack of irreducibility. A chain with more than one closed class can never
move from one to the other, and so again will remember its starting state for ever. When we
prove results about convergence to equilibrium, it will be under the condition that the chain
is irreducible and aperiodic.

5.7 Hitting probabilities

Let A be a subset of the state space I. Define hA
i = Pi (Xn ∈ A for some n ≥ 0), the hitting
probability of A starting from state i.
If A is a closed class, we might call hA
i the absorption probability.

Example. Let I = {1, 2, 3, 4} and

 
1 0 0 0
1 0 1
0
P = 2 2 .
 
0 1 1
2 0 2

0 0 0 1

Starting from 2, what is the probability of absorption at 4?

Write hi = Pi (Xn = 4 for some n ≥ 0). Then h4 = 1, and h1 = 0 since 1 is itself
absorbing. Also by conditioning on the first jump we have

h2 = 21 h1 + 21 h3 ,
h3 = 12 h2 + 21 h4 .

Solving, we get h2 = 1/3 and h3 = 2/3.

Theorem 5.7. The vector of hitting probabilities hAi ,i ∈ I is the minimal non-negative
solution to the equations 
1 if i ∈ A
hA
i = . (5.3)
P pij hA if i ∈ / A
j j

Here by “minimal” we mean that if (xi , i ∈ I) is another non-negative solution to the

system (5.3), then hi ≤ xi for all i.
Probability Part A, version of 2017/1/9 46

Proof. To see that hA

i satisfies (5.3), we condition on the first jump of the process. If i ∈
/ A,
then

hA
i = Pi (Xn ∈ A for some n ≥ 0)

= Pi (Xn ∈ A for some n ≥ 1)

X 

= Pi (X1 = j)P Xn ∈ A for some n ≥ 1X0 = i, X1 = j
j
X 

= pij P Xn ∈ A for some n ≥ 1X1 = j
j
X
= pij hA
j .
j

To obtain the penultimate line we applied the Markov property. Meanwhile for i ∈ A, hA i =1
by definition. So indeed (5.3) holds.
To prove minimality, suppose (xi , i ∈ I) is any non-negative solution to (5.3). We want
to show that hAi ≤ xi for all i.
We make the following claim: for any M ∈ N, and for all i,

xi ≥ Pi (Xn ∈ A for some n ≤ M ). (5.4)

We will prove (5.4) by induction on M .

The case M = 0 is easy; the LHS is 1 for i ∈ A, while the RHS is 0 for i ∈ / A.
For the induction step, suppose that for all i, xi ≥ Pi (Xn ∈ A for some n ≤ M − 1). If
i ∈ A, then again xi = 1 and (5.4) is clear. If i ∈
/ A, then
X
Pi (Xn ∈ A for some n ≤ M ) = pij Pi (Xn ∈ A for some n ∈ {1, 2, . . . , M }|X1 = j)
j
X
= pij Pj (Xn ∈ A for some n ∈ {0, 1, . . . , M − 1})
j
X
≤ pij xi
j

= xi ,

and the induction step is complete. Hence (5.4) holds for all i and M as desired. Then, using
the fact that the sequence of events in (5.4) is increasing in M , we have

xi ≥ lim Pi (Xn ∈ A for some n ≤ M )

M →∞
[

= Pi {Xn ∈ A for some n ≤ M }
M

= Pi (Xn ∈ A for some n)

= hA
i .
 Probability Part A, version of 2017/1/9 47

Important example: “Gambler’s ruin”

Let I = {0, 1, 2 . . . }. Let p ∈ (0, 1) and q = 1 − p, and consider the transition probabilities
given by

p00 = 1
pi,i−1 = q for i ≥ 1 (5.5)
pi,i+1 = p for i ≥ 1.

The name “gambler’s ruin” comes from the interpretation where the state is the current
capital of a gambler, who repeatedly bets 1 (against an infinitely rich bank). Will the gambler
inevitably go broke? But chains like this come up in a wide range of settings. Chains on
Z+ in which all transitions are steps up and down by 1 are called “birth-and-death chains”
(modelling the size of a population). This is one of the simplest examples.
Let hi = Pi (hit 0). To find hi , we need the minimal non-negative solution to

h0 = 1 (5.6)
hi = phi+1 + qhi−1 for i ≥ 1. (5.7)

If p 6= q, (5.7) has general solution

 i
q
hi = A + B .
p

We look at three cases:

p < q Jumps downwards are more likely than jumps upward. From (5.6), A + B = 1. Then
 i
for minimality, we take A = 1 and B = 0, since pq ≥ 1 for all i.
We obtain hi = 1 for all i. So with probability 1, the chain will hit 0.
 i
p > q Again A + B = 1. Also pq → 0 as i → ∞, so we need A ≥ 0 for a non-negative
 i
solution. Then for a minimal solution, we will want A = 0, B = 1, since 1 ≥ pq for
all i.
 i
Hence hi = pq . The chain has a positive probability of “escaping to infinity”.

p = q Now the general solution of (5.7) is hi = A + Bi. From i = 0 we get A = 1. For

non-negativity we need B ≥ 0, and then for minimality B = 0. We get hi = 1 again.
Now there is no drift, but still with probability 1 the chain will hit 0 eventually.

Remark. Notice that we could have seen hi = αi for some α, by a direct argument. Since the
chain can only descend by one step at a time,

Pi (hit 0) = Pi (hit i − 1)Pi−1 (hit i − 2) . . . P1 (hit 0), (5.8)

and all terms in the product are the same, since the transition probabilities are the same at
every level.
Probability Part A, version of 2017/1/9 48

5.8 Recurrence and transience

If the chain starts in state i, what is the chance that it returns to i at some point in the
future? We can distinguish two possibilities:

(1)
Pi (Xn = i for some n ≥ 1) = p < 1.
Then the total number of visits to i has geometric distribution with parameter 1 − p
(since each time we return to i, we have chance 1 − p of never returning again). We
have
Pi (hit i infinitely often) = 0.
The state i is called transient.

(2)
Pi (Xn = i for some n ≥ 1) = 1.
Then
Pi (hit i infinitely often) = 1.
The state i is called recurrent.

The definition is very simple, but the concept of recurrence and transience is extremely
rich (mainly for infinite chains).
There is an important criterion for recurrence and transience in terms of the transition
probabilities:
P∞ (n)
Theorem 5.8. State i is recurrent if and only if n=0 pii = ∞.
P∞ P∞
Proof. The total number of visits to i is n=0 I(Xn = i) which has expectation n=0 E I(Xn =
P∞ P∞ (n)
i) = n=0 P(Xn = i) = n=0 pii .
If i is transient, the number of visits to i is geometric with parameter 1 − p, and hence
1
with mean 1−p < ∞.
On the other hand if i is recurrent, the number of visits to i is infinite with probability 1,
and so has mean ∞.
This gives the statement of the theorem.

Theorem 5.9. (a) Let C be a communicating class. Either all states in C are recurrent,
or all are transient (so we may refer to the whole class as transient or recurrent).

(b) Every recurrent class is closed.

(c) Every finite closed class is recurrent.

Proof. Exercises – see example sheet 3. For part (a), use Theorem 5.8 to show that if i is
recurrent and i ↔ j, then j is also recurrent.

The theorem tells us that recurrence and transience are quite boring for finite chains:
state i is recurrent if and only if its communicating class is closed. But infinite chains are
more interesting! An infinite closed class may be either transient or recurrent.
Probability Part A, version of 2017/1/9 49

5.9 Random walk in Zd

Consider a simple symmetric random walk on the d-dimensional integer lattice. This is a
Markov chain with state space Zd and transition probabilities pxy = 1/2d if |x − y| = 1, and
pxy = 0 otherwise. The chain is irreducible, with period 2.
In this section we will show that the random walk is recurrent when d = 1 or d = 2 but
transient in higher dimensions.

5.9.1 d=1
The analysis after (5.7) (for p = q = 1/2) shows us that for the simple symmetric random
walk on Z, the hitting probability of 0 from any i > 0 is 1. By symmetry, the same is true
from any negative state. This shows that starting from 0, the probability of returning to 0 is
1. Hence state 0 is recurrent (and so by irreducibility the whole chain is recurrent).
An alternative approach uses Theorem 5.8. This gives a good warm-up for the approach
we will use in higher dimensions.
P∞ (n)
We need to show that n=0 p00 = ∞. We will use Stirling’s formula, which tells us that
√
n! ∼ 2π nn+1/2 e−n as n → ∞. (5.9)
√
(The constant 2π won’t be important.)
Suppose X0 = 0. If n is odd, then P0 (Xn = 0) = 0, since the chain has period 2. For
X2m = 0 we need m “ups” and m “downs” in the first 2m steps. Applying Stirling’s formula
to the binomial probability we obtain
! 
2m
(2m) 2m 1
p00 =
m 2
 2m
(2m)! 1
=
m!m! 2
1 1
∼√ . (5.10)
π m1/2
P −1/2 P (n)
Since m = ∞, we have p00 = ∞ and the chain is recurrent.

Exercise. Use Stirling’s formula to show that if p 6= q, then the chain is transient. (We
could also deduce this from the hitting probability analysis after (5.7).)

5.9.2 d=2
Why should the walk be recurrent in 1 and 2 dimensions but transient in 3 dimensions?
An intuitive answer is as follows. A d-dimensional random walk behaves in some sense like
d independent 1-dimensional walks. For the d-dimensional walk to be back at the origin,
we require all d of the 1-dimensional walks to be at 0. From (5.10), the probability that a
1-dimensional walk is at 0 decays like m−1/2 . Hence the probability that a 2-dimensional
walk is at the origin decays like m−1 , which sums to infinity, leading to recurrence, while the
corresponding probability for a 3-dimensional walk decays like m−3/2 which has finite sum,
leading to transience.
Probability Part A, version of 2017/1/9 50

In two dimensions we can make this precise in a very direct way. Let Xn be the walk in
2
Z and consider its projections onto the diagonal lines x = y and x = −y in the plane.
√
Each step of the walk increases or decreases the projection onto x = y by 1/ 2, and also
√
increases or decreases the projection onto x = −y by 1/ 2. All four possibilities are equally
likely.
Hence if we write Wn+ and Wn− for the two projections of Xn , we have that the processes
Wn+ and Wn− are independent of each other, and both of them are simple symmetric random
walks on 2−1/2 Z.
Then we have
+ −
P(X2m = 0) = P(W2m = 0)P(W2m = 0)
 2
1 1
∼ √
π m1/2
1
= .
πm
P (2m)
Hence p00 = ∞ and the walk is recurrent.

5.9.3 d=3
The trick from the previous section doesn’t work in d = 3, so we need to do a little more
combinatorics. As the walk has period 2 we have a positive chance of return to the origin
only when n is even. Each step is ±e1 , ±e2 or ±e3 where ei , i = 1, 2, 3 are the three unit
coordinate vectors. To return to the origin after 2m steps, we should have made, say, i steps
in each of the directions ±e1 , j steps in each of the directions ±e2 , and k steps in each of the
directions ±e3 for some i, j, k with i + j + k = m. Considering all the possible orderings of
these steps among the first 2m steps of the walk, we get
 2m
(2m)
X (2m)! 1
p00 =
i!2 j!2 k!2 6
i,j,k≥0
i+j+k=m
!  !2  
2m 2m
2m 1 X m 1
=
m 2 i, j, k 3
i,j,k≥0
i+j+k=m
 
!  !  ! 
2m m m
2m 1  X m 1  m 1
≤   max . (5.11)
m 2  i, j, k 3  i,j,k≥0 i, j, k 3
i,j,k≥0 i+j+k=m
i+j+k=m
!
m m!
Here, if i + j + k = m, we write = . Note that
i, j, k i!j!k!
! 
m
X m 1
= 1,
i, j, k 3
i,j,k≥0
i+j+k=m

since it is the sum of the mass function of a “trinomial(1/3, 1/3, 1/3)” distribution (consider
the number of ways of putting 3 balls into m boxes).
Probability Part A, version of 2017/1/9 51

If m is divisible by 3, say m = 3r, then it’s easy to check that the max in (5.11) is attained
when i = j = k = r, giving
!  ! 
2m m
(2m) 2m 1 m 1
p00 ≤
m 2 m/3, m/3, m/3 3
1 1 1 1
∼√ 1/2
×
2π m 2π m
1
∼ m−3/2 ,
(2π)3/2
P∞ (6r)
where we used Stirling’s formula again for the last line. Hence we have r=0 p00 < ∞.
(6r)
2 (6r−2) (6r)
4 (6r−4) P∞ (n)
Note also that p00 ≥ 61 p00 and p00 ≥ 16 p00 , so overall, n=0 p00 < ∞,
and the walk is transient.

5.9.4 d≥4
If we have a walk on Zd for d ≥ 4, we can obtain from it a walk on Z3 by looking only at
the first 3 coordinates, and ignoring any transitions that don’t change them. Since we know
that a walk on Z3 only visits the origin finitely often, the same must be true for the walk in
higher dimensions also. Hence we have transience for all d ≥ 3.

5.9.5 Mean hitting time

Let H A = min{n : Xn ∈ A}, the first hitting-time of the set A. Notice that H A is allowed to
take the value ∞. In fact if hA A A
i is the hitting probability defined above, then hi = Pi (H <
∞).
Let kiA = E i (H A ), the mean hitting time of A from i. If hA A
i < 1, then Pi (H = ∞) > 0
A A A
and certainly ki = ∞. Also maybe ki = ∞ even when hi = 1.

Theorem 5.10. The vector of mean hitting times k A = (kiA , i ∈ S) is the minimal non-
negative solution to 
0 if i ∈ A
kiA = .
A
P
1 +
j pij kj if i ∈
/A

Proof. “Condition on the first jump” again. For i ∈ / A,

X
kiA = E i (H A ) = E i (H A |X1 = j)Pi (X1 = j)
j
X
= pij (1 + kjA )
j
X
=1+ pij kjA .
j

For the minimality, one can use a similar idea to that at (5.4) in the proof of Theorem 5.7
above. Specifically, one can show by induction that if (yi ) is any non-negative solution to the


recursions, then yi ≥ E i min H A , m for all m ≥ 0; we omit the details.
 Probability Part A, version of 2017/1/9 52

5.9.6 Gambler’s ruin, continued

What is the expected hitting time of 0 from state i in the gambler’s ruin chain at (5.5)?
Let ki be the mean time to hit 0 starting from i. We give brief details (of course, one can
be more formal!).
If ki < ∞, one can see that ki = βi for some β, since (compare the remark at (5.8) above),

E i (time to 0) = E i (time to i − 1) + E i−1 (time to i − 2) + · · · + E 1 (time to 0).

To satisfy the recursion in Theorem (5.10), we need

ki = 1 + qki−1 + pki+1

which leads to (q − p)β = 1. We obtain:

1
p < q ki = q−p i; the chain takes a finite time on average to hit 0.

p > q We already know hi < 1, so certainly ki = ∞.

p = q There is no suitable β, so ki = ∞ here also, even though hi = 1. The chain hits 0 with
probability 1, but the mean time to arrive there is infinite.

5.10 Null recurrence and positive recurrence

Define mi , the mean return time to a state i by

mi : = E i (min{n ≥ 1 : Xn = i}) (5.12)

{i}
X
=1+ pij kj

{i}
(where kj is the mean hitting time of i starting from j).
This quantity will be particularly important when we consider equilibrium behaviour of
a Markov chain – loosely speaking, the long-run proportion of time spent in state i ought to
be the reciprocal of the mean return time.
If i is transient, then certainly mi = ∞ (since the return time itself is infinite with positive
probability).
If i is recurrent, then the return time is also finite, but nonetheless the mean could be
infinite.
If i is recurrent but mi = ∞, the state i is said to be null recurrent.
If mi < ∞ then the state i is said to be positive recurrent.
For similar reasons to those in Theorem 5.9, null recurrence and positive recurrence are
class properties; if one state in a communicating class is null (resp. positive) recurrent, then
every state in the class is null (resp. positive) recurrent.
If the chain is irreducible, we can therefore call the whole chain either transient, or null
recurrent, or positive recurrent.
 6

Markov chains: stationary distributions and

convergence to equilibrium

6.1 Stationary distributions

Let π = (πi , i ∈ I) be a distribution on the state space I.
We say that π is a stationary distribution, or invariant distribution, or equilibrium
distribution, for the transition matrix P if

πP = π .
P
That is, for all j, πj = i πi pij . The row vector π is a left eigenvector for the matrix P , with
eigenvalue 1.
If X0 has distribution π, then we know that Xn has distribution πP n . Hence if π is
stationary, then Xn has distribution π for all n. It follows that the sequence

(Xn , Xn+1 , Xn+2 , . . . )

has the same distribution as

(X0 , X1 , X2 , . . . )

for any n.

6.2 Main theorems

Theorem 6.1 (Existence and uniqueness of stationary distributions). Let P be an irreducible
transition matrix.

(a) P has a stationary distribution if and only if P is positive recurrent.

(b) In that case, the stationary distribution π is unique, and is given by πi = 1/mi for all
i (where mi is the mean return time to state i defined at (5.12)).

53
Probability Part A, version of 2017/1/9 54

Theorem 6.2 (Convergence to equilibrium). Suppose P is irreducible and aperiodic, with

stationary distribution π. If Xn is a Markov chain with transition matrix P and any initial
distribution, then for all j ∈ I,

P(Xn = j) → πj as n → ∞.

In particular,
(n)
pij → πj as n → ∞, for all i and j.

Theorem 6.3 (Ergodic theorem). Let P be irreducible. Let Vi (n) be the number of visits to
state i before time n, that is
n−1
X
Vi (n) = I (Xr = i) .
r=0

Then for any initial distribution, and for all i ∈ I,

Vi (n) 1
→ almost surely, as n → ∞.
n mi
That is,  
Vi (n) 1
P → as n → ∞ = 1.
n mi
The ergodic theorem concerns the “long-run proportion of time” spent in a state.
In the positive recurrent case, 1/mi = πi where π is the stationary distribution, so the
ergodic theorem says that (with probability 1) the long-run proportion of time spent in a
state is the stationary probability of that state.
In the null-recurrent or transient case, 1/mi = 0, so the ergodic theorem says that with
probability 1 the long-run proportion of time spent in a state is 0.
We can see the ergodic theorem as a generalisation of the strong law of large numbers. If
Xn is an i.i.d. sequence, then the strong law tells us that, with probability 1, the long run
proportion of entries in the sequence which are equal to i is equal to the probability that any
given entry is equal to i. The ergodic theorem can be seen as extending this to the case where
Xn is not i.i.d. but is a Markov chain.

6.3 Examples of stationary distributions

 
0 1 0
Example 6.4. Let P =  0 1/2 1/2. (Draw the diagram of the chain.)
 

1/2 0 1/2
For π to be stationary, we need

π1 = 12 π3
π2 = π1 + 12 π2
π3 = 12 π2 + 12 π3 .

One of these equations is redundant, and we need the added relation π1 + π2 + π3 = 1 to

normalise the solution (so that π is a distribution).
Probability Part A, version of 2017/1/9 55

Solving, we obtain (π1 , π2 , π3 ) = (1/5, 2/5, 2/5).

Correspondingly, the vector of mean return times is given by (m1 , m2 , m3 ) = (5, 5/2, 5/2).
From any initial state, the distribution at time n converges to π as n → ∞. For example,
(n)
p11 → 1/5 as n → ∞.
By the way, be careful to solve the equation πP = π and not to solve P π = π by mistake!
For any transition matrix P , the equation P π = π is solved by any vector π all of whose entries
are the same (why is this?) which could trap you into thinking that the uniform distribution
is stationary, which, of course, is not the case in general. We want the left eigenvector, rather
than the right eigenvector.

Example 6.5. Recall the example of a simple symmetric random walk on a cycle of size M
in Section 5.3. The distribution π with πi = 1/M for all i is stationary, since it solves

πi = 21 πi+1 + 12 πi−1

for each i. Because of the symmetry of the chain, it is not surprising that the stationary
distribution is uniform.
(n)
Is it the true that p00 → 1/M as n → ∞? If M is odd, then the chain is aperiodic (check
this!), so the answer is yes.
(n)
However, if M is even then the chain has period 2. Then p00 = 0 whenever n is odd. In
(2m)
fact p00 → 2π0 = 2/M as m → ∞ (exercise; consider the 2-step chain X0 , X2 , X4 , . . . on
the subset of the state space which consists just of even sites. Is it irreducible? What is its
stationary distribution?)

Example 6.6 (Random walk on a graph). A “graph” in the combinatorial sense is a collection
of vertices joined by edges. For example, the following graph has 6 vertices and 7 edges.
1 2
1
0 1
0
1
0 1
0

3 4 5 6
1
0 1
0 1
0 1
0
0
1 0
1 0
1 0
1

Let I be the set of vertices. Two vertices are neighbours in the graph if they are joined by an
edge. The degree of a vertex is its number of neighbours. Let di be the degree of vertex i. In
the graph above, the vector of vertex degrees is (di , i ∈ I) = (3, 2, 2, 4, 2, 1).
Assume di > 0 for all i. A random walk on the graph is a Markov chain with state space
I, evolving as follows; if i is the current vertex, then at the next step move to each of the
neighbours of i with probability 1/di .
Assume irreducibility of the chain (equivalently, that there is a path between any two
vertices in the graph). Then the stationary distribution of the chain π is unique.
In fact, the stationary probability of a vertex is proportional to its degree. To show this,
we will check that dP = d where d is the vector of vertex degrees and P is the transition
matrix of the chain:
X
dj = I(i is a neighbour of j)
i
Probability Part A, version of 2017/1/9 56

X 1
= di I(i is a neighbour of j)
i
di
X
= di pij ,
i

as required.
To obtain the stationary distribution we simply need to normalise d. So we obtain πi =
P
di / j dj .
P
For the graph above, j dj = 14, and we obtain
 
3 1 1 2 1 1
π= , , , , , .
14 7 7 7 7 14

From this we can deduce the mean return times. For example, m1 = 1/π1 = 14/3.
Notice that the chain is aperiodic. As a result, we also have convergence to the stationary
distribution. For example, starting from any initial distribution, the probability that the walk
is at vertex 1 at step n converges to 3/14 as n → ∞.

Example 6.7 (One-dimensional random walk). Consider again the familiar example of a
one-dimensional random walk. Let I = {0, 1, 2, . . . } and let

pi,i+1 = p for i ≥ 0,
pi,i−1 = q = 1 − p for i ≥ 1,
p0,0 = q.

If p > q, we found previously that the walk is transient, so no stationary distribution will
exist.
If p = q, the walk is recurrent, but the mean return time is infinite, so again there is no
stationary distribution.
If p < q, the walk is positive recurrent. For stationarity, we need πi = πi−1 p + πi+1 q
for i ≥ 1. This is (not coincidentally) reminiscent of the hitting probability equation we
previously found for the model (except the values of p and q are reversed). It has general
solution πi = A + B(p/q)i .
P
We need πi = 1, which forces A = 0 and B = (1 − p/q), giving
   i
p p
πi = 1− .
q q

That is, the stationary distribution of the walk is geometric with parameter 1 − pq .

Example 6.8 (A two-state chain and a non-irreducible chain). !

1−α α
Consider the two-state chain on {1, 2} with transition matrix P = .
β 1−β
 
β α
Solving πP = π and normalising we obtain that π = α+β , α+β .
(n)
Notice that this agrees with what we found in Example 5.4; the expression for p11 given
(n) β
in (5.2) satisfies p11 → π1 = α+β as n → ∞, as it should do because of the convergence to
equilibrium in Theorem 6.2.
Probability Part A, version of 2017/1/9 57

Now consider the chain on {1, 2, 3, 4} whose transition matrix is

 
1−α α 0 0
 β 1−β 0 0 
P = .
 
 0 0 1−γ γ 
0 0 δ 1−δ

This chain is not irreducible. We can view it as two separate

 two-state  on {1, 2} and
 chains, 
β α δ γ
{3, 4}, with no communication between them. Both α+β , α+β , 0, 0 and 0, 0, γ+δ , γ+δ
are stationary distributions. But also any mixture of these is stationary (since if π (1) and π (2)
are eigenvectors of P with eigenvalue 1, then so is any linear combination of π (1) and π (2) ).
So any distribution
 
β α δ γ
x ,x , (1 − x) , (1 − x) ,
α+β α+β γ+δ γ+δ
where x ∈ [0, 1], is stationary. (In fact, these are all the stationary distributions – exercise).
The uniqueness result in Theorem 6.1 does not apply because the transition matrix is not
irreducible.

6.4 Proof of Theorems 6.1, 6.2 and 6.3. (non-examinable)

The proofs below are given partly rather informally. They are not examinable; however, they
are very helpful in developing your intuition about the results. The “coupling” idea used in
the proof of Theorem 6.2 is particularly pretty and I certainly recommend thinking about it,
but it won’t be examined. (The results themselves are very much examinable!)

Proof of Theorem 6.3. This proof is essentially an application of the strong law of large num-
bers.
If the chain is transient, then with probability 1 there are only finitely many visits to any
state, so Vi (n) is bounded with probability 1. So
 
Vi (n)
P → 0 as n → ∞ = 1,
n
which is the result we want since mi = ∞.
Suppose instead that the chain is recurrent. In this case we will visit state i infinitely
often. Let Rk be the time between the kth and the (k + 1)st visits to i. Then R1 , R2 , R3 , . . .
are i.i.d. with mean mi (which is finite in the positive recurrent case and infinite in the null
recurrent case).
So by the strong law of large numbers,
 
R1 + R2 + · · · + RK
P → mi as K → ∞ = 1. (6.1)
K
Let TK be the time of the Kth visit to i. Then TK = T1 + X1 + X2 + · · · + XK−1 . It’s
easy to obtain that, for any c, TK /K → c if and only if (R1 + · · · + RK )/K → c. Hence from
(6.1) we have  
TK
P → mi as k → ∞ = 1. (6.2)
K
Probability Part A, version of 2017/1/9 58

Notice that TK /K is the time per visit (averaged over the first K visits) whereas Vi (n)/n
is the number of visits per unit time (averaged over the first n times). It’s straightforward
to obtain (check!) that, for any c, TK /K → c as K → ∞ if and only if Vi (n)/n → 1/c as
n → ∞.
Hence from (6.2) we have
 
Vi (n) 1
P → as n → ∞ = 1
n mi

as required.

Lemma 6.9. If P is positive recurrent, then it has stationary distribution π with πi = 1/mi .

Proof. We give an informal version of the proof, which could quite easily be made rigorous.
From the ergodic theorem, we know that (with probability 1) the long-run proportion of
visits to state i is 1/mi .
Each time the chain visits state i, it has probability pij of jumping from there to state j.
We can obtain that the long-run proportion of jumps from i to j is m1i pij .
First consider the case where the state space I is finite. By summing over i ∈ I, we get
that the long-run proportion of jumps into state j is i m1i pij .
P

But the long-run proportion of jumps into j is the same as the long-run proportion of
visits to j, which (by the ergodic theorem) is 1/mj .
We obtain
1 X 1
= pij ,
mj i
mi
P
i.e. πj = i πi pij , so that π satisfies πP = π and is stationary as desired.
If i is infinite, it is not immediate that the long-run proportion of jumps into j is the sum
over i of the long-run proportions of jumps from i to j. However (by considering as large a
finite set of i as desired) the second quantity does give an upper bound for the first, so we get
1 X 1
≤ pij
mj i
mi
P
for all j ∈ I. But summing both sides over j gives the same (finite) amount, since j pij = 1
for all i. So in fact we must have equality for all j as required.

Lemma 6.10. If π is any stationary distribution then πi = 1/mi .

Proof. Suppose π is stationary for P , and let X be a Markov chain with initial distribution
π and transition matrix P . Then by stationarity, P(Xn = i) = πi for all n, and
n−1
E Vn (i) 1X

= E I(Xn = i)
n n r=0
n−1
1X
= P(Xn = i)
n r=0
= πi . (6.3)
Probability Part A, version of 2017/1/9 59

From the ergodic theorem, for any 

  
 Vn (i) 1 
P   − > < (6.4)
n mi 

for large enough n (since almost sure convergence implies convergence in probability).
But since Vn (i)/n is bounded between 0 and 1, it follows from (6.4) (check!) that

E Vn (i) 1
→ as n → ∞.
n mi
Comparing to (6.3), we obtain πi = 1/mi .

This gives uniqueness of the stationary distribution for positive recurrent chains, and
shows that no stationary distribution can exist for null recurrent and transient chains. So we
have proved Theorem 6.1.
Finally, we prove the result on convergence to equilibrium.

Proof of Theorem 6.2. Let P be irreducible and aperiodic, with stationary distribution π.
Let λ be any initial distribution, and let (Xn , n ≥ 0) be Markov(λ, P ). We wish to show
that P(Xn = j) → πj as n → ∞, for any j.
Consider another chain (Yn , n ≥ 0) which is Markov(π, P ), and which is independent of
Z. Since π is stationary, Yn has distribution π for all n.
Let T = inf{n ≥ 0 : Xn = Yn }. We will claim that P(T < ∞) = 1; that is, the chains X
and Y will meet at some point.
Suppose this claim is true. Then define another chain Z by

X if n < T
n
Zn = .
Yn if n ≥ T

The idea is that Z starts in distribution λ, and evolves independently of the chain Y , until
they first meet. As soon as that happens, Z copies the moves of Y exactly.
Then Z is also Markov(λ, P ), since Zn starts in distribution λ and each jump is done
according to P , first by copying X up to time T , and then by copying Y after time T .
The idea is that the chain Y is “in equilibrium” (since it starts in the equilibrium distri-
bution π) so that if there is high probability that Yn = Zn , then the distribution of Zn must
be close to π. More precisely:

|P(Zn = j) − πj | = |P(Zn = j) − P(Yn = j)|

≤ P(Zn 6= Yn )
= P(T > n)
→ 0 as n → ∞.

Then we have P(Zn = j) → πj .

But the chains X and Z have the same distribution (they are both Markov(λ, P )). So we
have also shown that P(Xn = j) → πj , as required.
Probability Part A, version of 2017/1/9 60

It remains to prove the claim that T is finite with probability 1. Fix any state b ∈ I and
define Tb = inf{n ≥ 0 : Xn = Yn = b}. Then T ≤ Tb . We will show that Tb is finite with
probability 1.
Consider the process Wn = (Xn , Yn ). Since Xn and Yn evolve independently, Wn is a
Markov chain on the state space I × I with transition probabilities

p̃(i,k)(j,l) = pij pkl ,

and initial distribution

µ(i,k) = λi πk .

P is aperiodic and irreducible, so for all i, j, k, l, we have that

(n) (n) (n)
p̃(i,k)(j,l) = pij pkl > 0

for all large enough n. So P̃ is irreducible.

P̃ has an invariant distribution given by

π̃(i,k) = πi πk .

Hence P̃ is recurrent (by Theorem 6.1). But Tb = inf{n ≥ 0 : Wn = (b, b)}. Then indeed
P(Tb < ∞) = 1 (since an irreducible recurrent chain visits every state with probability 1).

Notice where the argument above fails when P is periodic. The chain Wn = (Xn , Yn ) still
has the stationary distribution of the form above, but it is not irreducible, so it may never
reach the state (b, b). (For example, if P has period 2, and the chains X and Y start out with
“opposite parity”, then they will never meet).
 7

Poisson processes

A Poisson process is a natural model for a stream of events occuring one by one in continuous
time, in an uncoordinated way. For example: the process of times of detections by a Geiger
counter near a radioactive source (a very accurate model); the process of times of arrivals of
calls at a call centre (often a good model); the process of times of arrivals of buses at a bus
stop (probably an inaccurate model; different buses are not really independent, for various
reasons).
Consider a random process Nt , t ∈ [0, ∞). (Note that “time” for our process is now a
continuous rather than a discrete set!)
Such a process is called a counting process if Nt takes values in {0, 1, 2, . . . }, and Ns ≤ Nt
whenever s ≤ t. We will also assume that N is right-continuous.
If Nt describes an arrival process, then Nt = k means that there have been k arrivals in
the time interval [0, t]. In fact we can describe the process by the sequence of arrival times,
which we might call “points” of the process. Let Tk = min{t : Nt ≥ k} for k ≥ 0. Then
T0 = 0 and Tk is the “kth arrival time”, for k ≥ 1. We also define Yk = Tk − Tk−1 for k ≥ 1.
Yk is the “interarrival time” between arrivals k − 1 and k.
For s < t, we write N (s, t] for Nt − Ns , which we can think of as the number of points of
the process which occur in the time-interval (s, t]. This is also called the “increment” of the
process N on the interval (s, t].

7.1 Poisson process: a choice of definitions

Let λ > 0. We will give two different definitions for what it means to be a Poisson process
of rate λ. Afterwards we will show that these definitions are equivalent.

Definition 7.1 (Definition of Poisson process via exponential interarrival times). Nt , t ≥ 0

is a Poisson process of rate λ if its interarrival times Y1 , Y2 , Y3 , . . . are i.i.d. with Exp(λ)
distribution.

Definition 7.2 (Definition of Poisson process via Poisson distribution of increments). Nt , t ≥

0 is a Poisson process of rate λ if:

(i) N0 = 0.

61
Probability Part A, version of 2017/1/9 62

(ii) If (s1 , t1 ), (s2 , t2 ), . . . , (sk , tk ) are disjoint intervals in R+ , then the increments N (s1 , t1 ],
N (s2 , t2 ], . . . , N (sk , tk ] are independent.

(iii) For any s < t, the increment N (s, t] has Poisson distribution with mean λ(t − s).

Property (ii) in Definition 7.2 is called the independent increments property. The
number of points falling in disjoint intervals is independent.
This can be seen as a version of the Markov property. For any t0 , the distribution of the
process (N (t0 , t0 + t], t ≥ 0), is independent of the process (Nt , t ≤ t0 ). Put another way, the
distribution of (Nt , t > t0 ) conditional on the process (Nt , t ≤ t0 ) depends only on the value
Nt 0 .

7.2 Equivalence of the definitions

We wish to show that the properties listed in Definitions 7.1 and 7.2 are equivalent. The key
idea is that the memoryless property for the exponential distribution and the independent
increments property are telling us the same thing. The argument below is somewhat informal
(but can be made completely rigorous).

Interrarival definition implies independent Poisson increments definition

Suppose we have Definition 7.1 in terms of i.i.d. exponential interarrival times. We wish to
show that it implies the statements in Definition 7.2.
Property (i) is immediate: since Y1 = T1 = min{t : Nt ≥ 1} is strictly positive with
probability 1, also N0 = 0 with probability 1.
Now let us consider the distribution of the number of points in an interval. First let us
take s = 0 in (iii), and consider N (0, t]. We want N (0, t] ∼ Poisson(λt), i.e. that for all k,

e−λt (λt)k
P N (0, t] = k = . (7.1)
k!
But we can rewrite the event on the LHS in terms of Tk and Tk+1 . Since Tk is the sum
of k independent exponentials of rate λ, we have Tk ∼ Gamma(k, λ), and similarly Tk+1 ∼
Gamma(k + 1, λ). So


P N (0, t] = k = P (Tk ≤ t, Tk+1 > t)
= P (Tk ≤ t) − P (Tk+1 ≤ t)
Z t Z t
(λx)k−1 e−λx (λx)k e−λx
= dx − dx. (7.2)
0 (k − 1)! 0 k!

Now we can check that the RHS of (7.1) and (7.2) are the same (for example, either by
integrating by parts in (7.2), or by differentiating in (7.1)). In this way we obtain that indeed
N (0, t] ∼ Poisson(λt).
Now we use the memoryless property of the exponential distribution to extend this to all
intervals and to give the independent increments property.
Probability Part A, version of 2017/1/9 63

Fix s, and suppose we condition on any outcome of the process on [0, s]. To be specific,
condition on the event that

Ns = k, T1 = t1 , T2 = t2 , . . . , Tk = tk .

Equivalently,
Y1 = t1 , Y2 = t2 − t1 , . . . , Yk = tk − tk−1 , Yk+1 > s − tk . (7.3)
The memoryless property for Yk+1 tells us that conditional on Yk+1 > s − tk , the distri-
bution of Yk+1 − (s − tk ) is again exponential with rate λ.
Combining this with the independence of the sequence Yi , we have that conditional on
(7.3), the sequence Yk+1 − (s − tk ), Yk+2 , Yk+3 , . . . is i.i.d. with Exp(λ) distribution.
But this means that, conditional on (7.3), the distribution of the process N (s, s + u], u ≥ 0
is the same as the original distribution of the process Nu , u ≥ 0.
So indeed, the property (iii) extends to all s. Further, the increment on (s, t] is independent
of the whole process on (0, s], and applying this repeatedly we get independence of any set of
increments on disjoint intervals. So Definition 7.2 holds as desired.

Poisson definition characterises the distribution of the process

With some work we could show the reverse implication using a direct calculation. Instead
we appeal to a general (although rather subtle) property. The Poisson definition specifies
the joint distribution of Nt1 , Nt2 , . . . , Ntk for any sequence t1 , t2 , . . . , tk . It turns out that
such “finite dimensional distributions”, along with the assumption that the process is right-
continuous, are enough to characterise completely the distribution of the entire process. We
won’t delve any further here into this fact from stochastic process theory. But it means that
at most one process could satisfy Definition 7.2, and since we have shown that a process
defined by Definition 7.1 does so, we have that Definition 7.2 implies 7.1 as desired.

7.2.1 The Poisson process as a limit of discrete-time processes

The calculation showing that (7.1) and (7.2) are the same is perhaps not very illuminating.
The case k = 0 is easy and is illustrated for example in Example 7.4(a) below. To get more
intuition for the relation between Poisson increments and exponential interarrivals, one can
also think about a related discrete-time process.
Let us recall some facts from earlier in the course:
d
(1) If Xn ∼ Binomial(n, λ/n), then Xn → Poisson(λ) as n → ∞. (See Example 2.9.)
d
(2) If Yn ∼ Geometric(λ/n), then Yn /n → Exp(λ) as n → ∞. (See Example 2.3.)

Now consider a sequence of independent Bernoulli trials. In each trial (or time-slot),
suppose we see a success with probability p and no event with probability 1 − p. Then in any
run of M trials, the total number of successes has Binomial(M, p) distribution. Meanwhile
the distances between consecutive successes are i.i.d. with Geometric(p) distribution.
Now consider n large. Let p = λ/n, and rescale time by a factor of 1/n, so that a time-
interval of length t corresponds to a run of tn trials. Then the number of events in a time-
interval of length t has Binomial(tn, λ/n) distribution, which is approximately Poisson(λt),
Probability Part A, version of 2017/1/9 64

while the times between consecutive sucesses have Geometric(λ/n) distribution rescaled by
1/n, which is approximately Exp(λ).
So indeed, as n → ∞, we obtain a continuous-time process in which the interarrival
times are independent exponentials, and the increments on disjoint intervals are independent
Poisson random variables. So we can see this exponential/Poisson relationship in the Pois-
son process as a limit of the geometric/binomial relationship which is already familiar from
sequences of independent trials.

7.2.2 A third definition (non-examinable)

Reflecting some of the ideas in the previous section, there is in fact a third natural definition
of the Poisson process, which we include for completeness. This involves the independent
increments property as in the case of Definition 7.2, but instead of specifying that increments
have Poisson distribution, it specifies the behaviour of the increments on small time-intervals.
Namely, the probability of seeing an event in a small interval should behave like λ multiplied
by the length of the interval, and it should be very unlikely that two or more events occur
within the interval:

Definition 7.3 (Defintion of Poisson process via infinitesimal increments). Nt , t ≥ 0 is a

Poisson process of rate λ if:

(i) N0 = 0.

(ii) If (s1 , t1 ), (s2 , t2 ), . . . , (sk , tk ) are disjoint intervals in R+ , then the increments N (s1 , t1 ],
N (s2 , t2 ], . . . , N (sk , tk ] are independent.

(iii) The distribution of N (s, s + h] is the same for all s, and as h → 0,

P(N (s, s + h] = 0) = 1 − λh + o(h)

P(N (s, s + h] = 1) = λh + o(h) (7.4)
P(N (s, s + h] ≥ 2) = o(h).

Note that any two of the conditions of (7.4) imply the third.
This kind of formulation is very natural when moving to the context of more general
continuous-time Markov jump processes (in which the rate at which jumps occur may depend
on the present state). The definition can again be shown to be equivalent to Definitions 7.1
and 7.2.

7.3 Thinning and superposition of Poisson processes

Theorem 7.1 (Superposition of Poisson processes). Let Lt and Mt be independent Poisson
processes of rate λ and µ respectively. Let Nt = Lt + Mt . Then Nt is a Poisson process of
rate λ + µ.

Proof. We work from the definition of a Poisson process in terms of independent Poisson
increments for disjoint intervals. Clearly, N0 = L0 + M0 = 0 for property (i), and also
Probability Part A, version of 2017/1/9 65

Nt satisfies property (ii) (independent increments) since Lt and Mt both have independent
increments and are independent of each other.
So we need to show property (iii). Since L(s, t] ∼ Poisson(λt) and M (s, t] ∼ Poisson(µt)
independently of each other, we have N (s, t] ∼ Poisson((λ + µ)t) as required, by familiar
properties of the Poisson distribution.

Theorem 7.2 (Thinning of a Poisson process). Let Nt be a Poisson process of rate λ. “Mark”
each point of the process with probability p, independently for different points. Let Mt be the
counting process of the marked points. Then Mt is a Poisson process of rate pλ.
Proof. Again we will work with the definition in terms of independent Poisson increments.
Properties (i) and (ii) for M follow from the same properties for N .
Now consider any interval (s, t]. We have N (s, t] ∼ Poisson(λ(t − s)), and conditional on
N (s, t] = n, we have M (s, t] ∼ Binomial(n, p).
But if N ∼ Poisson(µ), and, conditional on N = n, M ∼ Binomial(n, p), then in fact
M ∼ Poisson(pµ). This fact was proved in two different ways in the Prelims course. For
example, it can be done using generating functions: let M = X1 + X2 + · · · + XN where
Xi are i.i.d. Bernoulli random variables; then GM (s) = GN (GX (s)). Alternatively, by direct
calculation:
X
P(M = k) = P(M = k|N = n)P(N = n)
n
!
X e−µ µn n k
= p (1 − p)n−k
n! k
n≥k
..
.
e−pµ (pµ)k
= .
k!
Hence indeed we have here that M (s, t] ∼ Poisson(pλ(t − s)). So indeed property (iii)
holds as desired, and M is a Poisson process of rate pλ.

Remark 7.3. In fact, it is not too hard to prove something stronger. If L is the process of
unmarked points, then L is a Poisson process of rate (1 − p)λ, and the processes L and M
are independent.

7.4 Poisson process examples

Example 7.4. A Geiger counter near a radioactive source detects particles at an average
rate of 1 per 2 seconds. (a) What is the probability that there is no particle detected for 3
seconds after the detector is switched on? (b) What is the probability of detecting at least 3
particles in the first 4 seconds?
Solution: We model the process of detections as a Poisson process with rate λ = 0.5 (where
the unit of time is 1 second).
For part (a), P(N3 = 0) = e−3λ = e−1.5 , since N3 , the number of points up to time
3, has Poisson(3λ) distribution. Alternatively, we could calculate the same probability as
P(T1 > 3) = e−3λ since T1 , the time of the first point of the process, has distribution Exp(λ).
Probability Part A, version of 2017/1/9 66

For part (b), N4 has Poisson distribution with mean 4λ = 2. Then

P (N4 ≥ 3) = 1 − P (N4 = 0) − P (N4 = 1) − P (N4 = 2)

22 e−2
= 1 − e−2 − 2e−2 −
2!
= 1 − 5e−2 .

Example 7.5. A call centre receives calls from existing customers at rate 1 per 20 seconds,
and calls from potential new customers at rate 1 per 30 seconds. Assume that these form
independent Poisson processes. (a) What is the distribution of the total number of calls in
a given minute? (b) What is the probability that the next call to arrive is from a potential
new customer? (c) Suppose each call from a potential new customer results in a contract
with probability 1/4 independently. What is the distribution of the number of new contracts
arising from calls in a given hour?
Solution: Let the unit of time be 1 minute, so that the Poisson processes in the question
have rates 3 and 2.
(a) From Theorem 7.1, the combined process of all calls is a Poisson process of rate 5.
The number of calls in a given minute has Poisson(5) distribution.
(b) From any given moment, the time until the next “existing” call, say U1 , is exponential
with rate 3, and the time until the next “new” call, say V1 , is exponential with rate 2.
Z ∞ Z ∞
P(U1 < V1 ) = 3e−3u × 2e−2v dvdu
u=0 v=u
Z ∞
= 3e−3u e−2u du
u=0

= 3/(2 + 3)
= 3/5.

(In fact, it is not a coincidence that here the answer is the ratio of the rate of the “existing
customer” process to the rate of the two processes combined. This fact follows from Remark
7.3; we can consider a single process of rate 5 and “mark” each point with probability 3/5,
to arrive at two independent processes with rates 3 and 2. In particular, the probability that
the first point is marked is then 3/5.)
(c) The process of calls resulting in contracts is a thinning of the process of calls from
potential new customers. This gives us a new Poisson process of rate 1/4 × 2 = 1/2. So
the total number of calls resulting in new contracts in a given time interval of length 60 has
Poisson(30) distribution.

Example 7.6 (Genetic recombination model). An illustration of genetic recombination is

shown in the figure below. In most of our cells, we have two versions of each chromosome,
one inherited from our mother and one from our father. Sex cells – sperm and ova – contain
only one copy of each chromosome.
During meiosis – the process in which sperm and ova are created – the chromosomes are
broken at certain random “crossover” or “recombination” points, to form new chromosomes
out of pieces of the maternal and paternal chromosomes. The crossover points are shown as
crosses in the top line of the diagram.
Probability Part A, version of 2017/1/9 67

a b c d
maternal

paternal
a b c d

a b c d

new chromosomes

a b c d

Genes occur at particular positions along the chromosome. In early genetic research,
biologists investigated the position of genes on chromosomes by looking at how likely the
genes were to stay together, generation after generation. Genes on different chromosomes
should be passed on independently. Genes that are close together on the same chromosome
should almost always be passed on together, while genes that are on the same chromosome
but further apart should be more likely than chance to be inherited together, but not certain.
In the figure, genes b, c and d stay together but a is separated from them.
As a simple model, we can imagine the chromosome as a continuous line, and model the
recombination points as a Poisson process along it, of rate λ, say.
Consider two points a and b on the interval, representing the location of two genes. Let x
be the distance between a and b. The probability of seeing no crossover at all between a and
b is given by

P (no crossover in (a, b)) = e−λx .

But what we really want to compute is the probability of seeing an even number of crossovers
between a and b:
∞
X (λx)2k
P (even number of crossovers in (a, b)) = e−λx
(2k)!
k=0
Probability Part A, version of 2017/1/9 68

(λx)2 (λx)4
 
= e−λx 1 + + + ...
2! 4!
−λx
 λx 
e +e
= e−λx
2
1 + e−2λx
= .
2
If we observe that a and b are inherited together with probability p > 1/2, we can invert the
expression above to estimate the distance between them by
1
x=− log(2p − 1).
2λ