Probability Spaces
Lecture 4: MTH 202 (Probability and Statistics)
Atreyee Bhattacharya
January 9, 2025
Department of Mathematics, IISER Bhopal
1
(Image credit: www.cartoonstock.com) 2
Prologue
• Goal: To compute/predict the probability (/likelihood) of
occurrence of particular outcomes of random phenomena.
3
Prologue
• Goal: To compute/predict the probability (/likelihood) of
occurrence of particular outcomes of random phenomena.
• Strategy:
3
Prologue
• Goal: To compute/predict the probability (/likelihood) of
occurrence of particular outcomes of random phenomena.
• Strategy:
• Pose this as a mathematical problem by looking at the set
Ω of all possible outcomes (a particular outcome/event is a
subset) .
3
Prologue
• Goal: To compute/predict the probability (/likelihood) of
occurrence of particular outcomes of random phenomena.
• Strategy:
• Pose this as a mathematical problem by looking at the set
Ω of all possible outcomes (a particular outcome/event is a
subset) .
• Find a suitable function p : {plausible events} → [0, 1]
3
Prologue
• Goal: To compute/predict the probability (/likelihood) of
occurrence of particular outcomes of random phenomena.
• Strategy:
• Pose this as a mathematical problem by looking at the set
Ω of all possible outcomes (a particular outcome/event is a
subset) .
• Find a suitable function p : {plausible events} → [0, 1]
• Realize probability of an event A(⊂ Ω) as p(A).
3
Prologue
• Goal: To compute/predict the probability (/likelihood) of
occurrence of particular outcomes of random phenomena.
• Strategy:
• Pose this as a mathematical problem by looking at the set
Ω of all possible outcomes (a particular outcome/event is a
subset) .
• Find a suitable function p : {plausible events} → [0, 1]
• Realize probability of an event A(⊂ Ω) as p(A).
• As the difficulty level depends on Ω and the collection of
plausible events, the desired theory should work for all Ω.
3
Prologue
• Goal: To compute/predict the probability (/likelihood) of
occurrence of particular outcomes of random phenomena.
• Strategy:
• Pose this as a mathematical problem by looking at the set
Ω of all possible outcomes (a particular outcome/event is a
subset) .
• Find a suitable function p : {plausible events} → [0, 1]
• Realize probability of an event A(⊂ Ω) as p(A).
• As the difficulty level depends on Ω and the collection of
plausible events, the desired theory should work for all Ω.
• Today’s topics of discussion:
3
Prologue
• Goal: To compute/predict the probability (/likelihood) of
occurrence of particular outcomes of random phenomena.
• Strategy:
• Pose this as a mathematical problem by looking at the set
Ω of all possible outcomes (a particular outcome/event is a
subset) .
• Find a suitable function p : {plausible events} → [0, 1]
• Realize probability of an event A(⊂ Ω) as p(A).
• As the difficulty level depends on Ω and the collection of
plausible events, the desired theory should work for all Ω.
• Today’s topics of discussion:
• Recall: σ-algebras and probability measures
3
Prologue
• Goal: To compute/predict the probability (/likelihood) of
occurrence of particular outcomes of random phenomena.
• Strategy:
• Pose this as a mathematical problem by looking at the set
Ω of all possible outcomes (a particular outcome/event is a
subset) .
• Find a suitable function p : {plausible events} → [0, 1]
• Realize probability of an event A(⊂ Ω) as p(A).
• As the difficulty level depends on Ω and the collection of
plausible events, the desired theory should work for all Ω.
• Today’s topics of discussion:
• Recall: σ-algebras and probability measures
• Probability spaces and their properties
3
Recall: σ-algebras in relation with random phenomena
• Definition 1. A collection A of subsets of a set Ω is called
a σ-algebra if the following conditions hold:
4
Recall: σ-algebras in relation with random phenomena
• Definition 1. A collection A of subsets of a set Ω is called
a σ-algebra if the following conditions hold:
• ∅ ∈ A,
4
Recall: σ-algebras in relation with random phenomena
• Definition 1. A collection A of subsets of a set Ω is called
a σ-algebra if the following conditions hold:
• ∅ ∈ A,
• A ∈ A =⇒ Ac ∈ A, and
4
Recall: σ-algebras in relation with random phenomena
• Definition 1. A collection A of subsets of a set Ω is called
a σ-algebra if the following conditions hold:
• ∅ ∈ A,
• A ∈ A =⇒ Ac ∈ A, and
• for a countable collection {A1 , A2 , . . .} in A,
∞
[
An ∈ A.
n=1
4
Recall: σ-algebras in relation with random phenomena
• Definition 1. A collection A of subsets of a set Ω is called
a σ-algebra if the following conditions hold:
• ∅ ∈ A,
• A ∈ A =⇒ Ac ∈ A, and
• for a countable collection {A1 , A2 , . . .} in A,
∞
[
An ∈ A.
n=1
• The abstract definition above, helps in determining
probability of particular outcomes and events associated
with random phenomena, when Ω represents the set of all
possible outcomes of a random phenomenon.
4
Probability measures and probability spaces
• Examples 3 (σ-algebras) :
5
Probability measures and probability spaces
• Examples 3 (σ-algebras) :
• For any set Ω, A1 , := {∅, Ω} and A2 := P(Ω)(the power
set), are σ-algebras.
5
Probability measures and probability spaces
• Examples 3 (σ-algebras) :
• For any set Ω, A1 , := {∅, Ω} and A2 := P(Ω)(the power
set), are σ-algebras.
• For Ω = [a, b], there is a ‘nice’ σ-algebra L that contains all
intervals in Ω, called the Lebesgue σ-algebra (to be
discussed later!)
5
Probability measures and probability spaces
• Examples 3 (σ-algebras) :
• For any set Ω, A1 , := {∅, Ω} and A2 := P(Ω)(the power
set), are σ-algebras.
• For Ω = [a, b], there is a ‘nice’ σ-algebra L that contains all
intervals in Ω, called the Lebesgue σ-algebra (to be
discussed later!)
• Definition 2. Given a set Ω with a σ-algebra A, a
probability measure on (Ω, A) is a function p : A → [0, ∞)
satisfying
5
Probability measures and probability spaces
• Examples 3 (σ-algebras) :
• For any set Ω, A1 , := {∅, Ω} and A2 := P(Ω)(the power
set), are σ-algebras.
• For Ω = [a, b], there is a ‘nice’ σ-algebra L that contains all
intervals in Ω, called the Lebesgue σ-algebra (to be
discussed later!)
• Definition 2. Given a set Ω with a σ-algebra A, a
probability measure on (Ω, A) is a function p : A → [0, ∞)
satisfying
• p(Ω) = 1
5
Probability measures and probability spaces
• Examples 3 (σ-algebras) :
• For any set Ω, A1 , := {∅, Ω} and A2 := P(Ω)(the power
set), are σ-algebras.
• For Ω = [a, b], there is a ‘nice’ σ-algebra L that contains all
intervals in Ω, called the Lebesgue σ-algebra (to be
discussed later!)
• Definition 2. Given a set Ω with a σ-algebra A, a
probability measure on (Ω, A) is a function p : A → [0, ∞)
satisfying
• p(Ω) = 1
• For a mutually disjoint countable collection {A1 , A2 , . . .} in
A, we have
∞
[ ∞
X
p( An ) = p(An ).
n=1 n=1
5
Probability measures and probability spaces
• Examples 3 (σ-algebras) :
• For any set Ω, A1 , := {∅, Ω} and A2 := P(Ω)(the power
set), are σ-algebras.
• For Ω = [a, b], there is a ‘nice’ σ-algebra L that contains all
intervals in Ω, called the Lebesgue σ-algebra (to be
discussed later!)
• Definition 2. Given a set Ω with a σ-algebra A, a
probability measure on (Ω, A) is a function p : A → [0, ∞)
satisfying
• p(Ω) = 1
• For a mutually disjoint countable collection {A1 , A2 , . . .} in
A, we have
∞
[ ∞
X
p( An ) = p(An ).
n=1 n=1
5
Probability measures and probability spaces
• Examples 3 (σ-algebras) :
• For any set Ω, A1 , := {∅, Ω} and A2 := P(Ω)(the power
set), are σ-algebras.
• For Ω = [a, b], there is a ‘nice’ σ-algebra L that contains all
intervals in Ω, called the Lebesgue σ-algebra (to be
discussed later!)
• Definition 2. Given a set Ω with a σ-algebra A, a
probability measure on (Ω, A) is a function p : A → [0, ∞)
satisfying
• p(Ω) = 1
• For a mutually disjoint countable collection {A1 , A2 , . . .} in
A, we have
∞
[ ∞
X
p( An ) = p(An ).
n=1 n=1
The triple (Ω, A, p) is called a probability space.
5
Examples of probability spaces
• When Ω is the set of all possible outcomes of a random
phenomenon and (Ω, A, p) a probability space, then each
A ∈ A is called an event and p(A) represents the
probability of its occurrence.
6
Examples of probability spaces
• When Ω is the set of all possible outcomes of a random
phenomenon and (Ω, A, p) a probability space, then each
A ∈ A is called an event and p(A) represents the
probability of its occurrence.
• Examples 4:
6
Examples of probability spaces
• When Ω is the set of all possible outcomes of a random
phenomenon and (Ω, A, p) a probability space, then each
A ∈ A is called an event and p(A) represents the
probability of its occurrence.
• Examples 4:
1 With a finite set Ω and A = P(Ω), define p(A) := |A| |Ω| , | · | is
the cardinality. The probability space (Ω, A, p) is called a
symmetric probability space.
6
Examples of probability spaces
• When Ω is the set of all possible outcomes of a random
phenomenon and (Ω, A, p) a probability space, then each
A ∈ A is called an event and p(A) represents the
probability of its occurrence.
• Examples 4:
1 With a finite set Ω and A = P(Ω), define p(A) := |A| |Ω| , | · | is
the cardinality. The probability space (Ω, A, p) is called a
symmetric probability space.
2 In tossing a coin, Ω = {H, T }, with A = P(Ω) and p as
above, p(H) = p(T ) = 21 .
6
Examples of probability spaces
• When Ω is the set of all possible outcomes of a random
phenomenon and (Ω, A, p) a probability space, then each
A ∈ A is called an event and p(A) represents the
probability of its occurrence.
• Examples 4:
1 With a finite set Ω and A = P(Ω), define p(A) := |A| |Ω| , | · | is
the cardinality. The probability space (Ω, A, p) is called a
symmetric probability space.
2 In tossing a coin, Ω = {H, T }, with A = P(Ω) and p as
above, p(H) = p(T ) = 21 .
3 Defining p(H) = 13 and p(T ) = 32 In (2), we get a
probability space that is not symmetric.
6
Examples of probability spaces
• When Ω is the set of all possible outcomes of a random
phenomenon and (Ω, A, p) a probability space, then each
A ∈ A is called an event and p(A) represents the
probability of its occurrence.
• Examples 4:
1 With a finite set Ω and A = P(Ω), define p(A) := |A| |Ω| , | · | is
the cardinality. The probability space (Ω, A, p) is called a
symmetric probability space.
2 In tossing a coin, Ω = {H, T }, with A = P(Ω) and p as
above, p(H) = p(T ) = 21 .
3 Defining p(H) = 13 and p(T ) = 32 In (2), we get a
probability space that is not symmetric.
4 Let Ω = [a, b] and A = L as above. Define f : L → [0, ∞) by
f ([c, d]) = f ([c, d)) = f (c, d]) = f ((c, d)) = |d − c|. Then
f (A)
p : A 7→ |b−a| is a probability measure. (Ω, A, p) is called a
uniform probability space. 6
Declaration:
• If (Ω, A, p) is a probability space where Ω is a finite set, we
will typically take A = P(Ω) with p being the symmetric
probability measure (unless specified otherwise).
7
Declaration:
• If (Ω, A, p) is a probability space where Ω is a finite set, we
will typically take A = P(Ω) with p being the symmetric
probability measure (unless specified otherwise).
• When Ω is infinite and (Ω, A, p) is a probability space, it
will be either an interval or a disc or a Cartesian product of
intervals (known as rectangles) and we will mostly use
some version of the Lebesgue measure (to be elaborated on
later) on this.
7
Properties of Probability spaces
Proposition 1. Let (Ω, A, p) be a probability space. Then
1 for all A, B ∈ A, p(B) = p(B ∩ A) + p(B ∩ Ac )
8
Properties of Probability spaces
Proposition 1. Let (Ω, A, p) be a probability space. Then
1 for all A, B ∈ A, p(B) = p(B ∩ A) + p(B ∩ Ac )
2 for all A ∈ A, p(A) = 1 − p(Ac )
8
Properties of Probability spaces
Proposition 1. Let (Ω, A, p) be a probability space. Then
1 for all A, B ∈ A, p(B) = p(B ∩ A) + p(B ∩ Ac )
2 for all A ∈ A, p(A) = 1 − p(Ac )
3 p(∅) = 0
8
Properties of Probability spaces
Proposition 1. Let (Ω, A, p) be a probability space. Then
1 for all A, B ∈ A, p(B) = p(B ∩ A) + p(B ∩ Ac )
2 for all A ∈ A, p(A) = 1 − p(Ac )
3 p(∅) = 0
4 A ⊂ B =⇒ p(B) − p(A) = p(B \ A) where
B \ A = B ∩ Ac ).
8
Properties of Probability spaces
Proposition 1. Let (Ω, A, p) be a probability space. Then
1 for all A, B ∈ A, p(B) = p(B ∩ A) + p(B ∩ Ac )
2 for all A ∈ A, p(A) = 1 − p(Ac )
3 p(∅) = 0
4 A ⊂ B =⇒ p(B) − p(A) = p(B \ A) where
B \ A = B ∩ Ac ).
8
Properties of Probability spaces
Proposition 1. Let (Ω, A, p) be a probability space. Then
1 for all A, B ∈ A, p(B) = p(B ∩ A) + p(B ∩ Ac )
2 for all A ∈ A, p(A) = 1 − p(Ac )
3 p(∅) = 0
4 A ⊂ B =⇒ p(B) − p(A) = p(B \ A) where
B \ A = B ∩ Ac ).
In particular, A ⊂ B =⇒ p(A) ≤ p(B).
5 for any countable collection {A1 , A2 , . . .} in A, we have
∞
[ ∞
\
p( An ) = 1 − p( Acn ).
n=1 n=1
8
Proof of Proposition 1
Proof:
1 For any A, B ∈ A, B = (B ∩ A) ∪ (B ∩ Ac )
and these two sets are disjoint. Hence it follows that
p(B) = p(B ∩ A) + p(B ∩ Ac ).
9
Proof of Proposition 1
Proof:
1 For any A, B ∈ A, B = (B ∩ A) ∪ (B ∩ Ac )
and these two sets are disjoint. Hence it follows that
p(B) = p(B ∩ A) + p(B ∩ Ac ).
2 This is immediate as for all A ∈ A, Ω = A ∪ Ac with
A ∩ Ac = ∅, i.e. 1 = p(Ω) = p(A) + p(Ac ).
9
Proof of Proposition 1
Proof:
1 For any A, B ∈ A, B = (B ∩ A) ∪ (B ∩ Ac )
and these two sets are disjoint. Hence it follows that
p(B) = p(B ∩ A) + p(B ∩ Ac ).
2 This is immediate as for all A ∈ A, Ω = A ∪ Ac with
A ∩ Ac = ∅, i.e. 1 = p(Ω) = p(A) + p(Ac ).
3 Using (2), p(∅) = 1 − p(Ω) = 0.
9
Proof of Proposition 1
Proof:
1 For any A, B ∈ A, B = (B ∩ A) ∪ (B ∩ Ac )
and these two sets are disjoint. Hence it follows that
p(B) = p(B ∩ A) + p(B ∩ Ac ).
2 This is immediate as for all A ∈ A, Ω = A ∪ Ac with
A ∩ Ac = ∅, i.e. 1 = p(Ω) = p(A) + p(Ac ).
3 Using (2), p(∅) = 1 − p(Ω) = 0.
4 Here A = B ∩ A. Using (1), p(B) = p(A) + p(B ∩ Ac ).
Thus p(B) − p(A) = p(B \ A) ≥ 0
9
Proof of Proposition 1
Proof:
1 For any A, B ∈ A, B = (B ∩ A) ∪ (B ∩ Ac )
and these two sets are disjoint. Hence it follows that
p(B) = p(B ∩ A) + p(B ∩ Ac ).
2 This is immediate as for all A ∈ A, Ω = A ∪ Ac with
A ∩ Ac = ∅, i.e. 1 = p(Ω) = p(A) + p(Ac ).
3 Using (2), p(∅) = 1 − p(Ω) = 0.
4 Here A = B ∩ A. Using (1), p(B) = p(A) + p(B ∩ Ac ).
Thus p(B) − p(A) = p(B \ A) ≥ 0
5 Follows from (2) and De Morgan’s law which says that
∞
[ ∞
\
( An )c = Acn .
n=1 n=1
9
Consequences of Proposition 1
• Remark 1. In Proposition 1.(5), each An is an event. If
B := ∞
S c
T∞ c
n=1 An (i.e. B = n=1 An ), then
p(B) = probability that at least one of these events occurs,
p(B c )= probability that none of these events occur.
10
Consequences of Proposition 1
• Remark 1. In Proposition 1.(5), each An is an event. If
B := ∞
S c
T∞ c
n=1 An (i.e. B = n=1 An ), then
p(B) = probability that at least one of these events occurs,
p(B c )= probability that none of these events occur.
• Example 5. Three perfectly balanced and identical coins
are tossed simultaneously. What is the probability of at
least one heads?
10
Consequences of Proposition 1
• Remark 1. In Proposition 1.(5), each An is an event. If
B := ∞
S c
T∞ c
n=1 An (i.e. B = n=1 An ), then
p(B) = probability that at least one of these events occurs,
p(B c )= probability that none of these events occur.
• Example 5. Three perfectly balanced and identical coins
are tossed simultaneously. What is the probability of at
least one heads?
• Soln: As each coin has two possibilities: H and T , there
are total 23 = 8 possible outcomes of the experiment.
10
Consequences of Proposition 1
• Remark 1. In Proposition 1.(5), each An is an event. If
B := ∞
S c
T∞ c
n=1 An (i.e. B = n=1 An ), then
p(B) = probability that at least one of these events occurs,
p(B c )= probability that none of these events occur.
• Example 5. Three perfectly balanced and identical coins
are tossed simultaneously. What is the probability of at
least one heads?
• Soln: As each coin has two possibilities: H and T , there
are total 23 = 8 possible outcomes of the experiment.
10
Consequences of Proposition 1
• Remark 1. In Proposition 1.(5), each An is an event. If
B := ∞
S c
T∞ c
n=1 An (i.e. B = n=1 An ), then
p(B) = probability that at least one of these events occurs,
p(B c )= probability that none of these events occur.
• Example 5. Three perfectly balanced and identical coins
are tossed simultaneously. What is the probability of at
least one heads?
• Soln: As each coin has two possibilities: H and T , there
are total 23 = 8 possible outcomes of the experiment.
i.e. |Ω| = 8. If Ai denotes: ‘ith coin has heads’ (1 ≤ i ≤ 3),
we need to compute p(A1 ∪ A2 ∪ A3 ).
10
Consequences of Proposition 1
• Remark 1. In Proposition 1.(5), each An is an event. If
B := ∞
S c
T∞ c
n=1 An (i.e. B = n=1 An ), then
p(B) = probability that at least one of these events occurs,
p(B c )= probability that none of these events occur.
• Example 5. Three perfectly balanced and identical coins
are tossed simultaneously. What is the probability of at
least one heads?
• Soln: As each coin has two possibilities: H and T , there
are total 23 = 8 possible outcomes of the experiment.
i.e. |Ω| = 8. If Ai denotes: ‘ith coin has heads’ (1 ≤ i ≤ 3),
we need to compute p(A1 ∪ A2 ∪ A3 ).
B := (A1 ∪ A2 ∪ A3 )c = Ac1 ∩ Ac2 ∪ Ac3 is the event where no
heads occurs i.e., all three tails occur. Since, |B| = 1,
p(B) = 18 . Thus p(A1 ∪ A2 ∪ A3 ) = 1 − 18 = 87 .
10
More properties of Probability spaces
Proposition 2. Let (Ω, A, p) be a probability space. Then
1 for all A, B ∈ A (not necessarily mutually disjoint),
p(A ∪ B) = p(A) + p(B) − p(A ∩ B).
11
More properties of Probability spaces
Proposition 2. Let (Ω, A, p) be a probability space. Then
1 for all A, B ∈ A (not necessarily mutually disjoint),
p(A ∪ B) = p(A) + p(B) − p(A ∩ B).
2 for any finite collection {A1 , A2 , . . . Ak } in A (not
necessarily mutually disjoint), we have
k
[ k
X
p( Aj ) ≤ p(Aj ).
j=1 j=1
11
Proof of Proposition 2
Proof:
1 Since B = (B ∩ A) ∪ (B ∩ Ac ), it follows that
p(B) = p(B ∩ A) + p(B ∩ Ac ) (as the two sets are disjoint).
12
Proof of Proposition 2
Proof:
1 Since B = (B ∩ A) ∪ (B ∩ Ac ), it follows that
p(B) = p(B ∩ A) + p(B ∩ Ac ) (as the two sets are disjoint).
12
Proof of Proposition 2
Proof:
1 Since B = (B ∩ A) ∪ (B ∩ Ac ), it follows that
p(B) = p(B ∩ A) + p(B ∩ Ac ) (as the two sets are disjoint).
Thus p(A) + p(B) − p(A ∩ B) = p(A) + p(B ∩ Ac ).
12
Proof of Proposition 2
Proof:
1 Since B = (B ∩ A) ∪ (B ∩ Ac ), it follows that
p(B) = p(B ∩ A) + p(B ∩ Ac ) (as the two sets are disjoint).
Thus p(A) + p(B) − p(A ∩ B) = p(A) + p(B ∩ Ac ).
Also, A ∪ B = A ∪ (B ∩ Ac ) (check!) and
this is a disjoint union. Hence
12
Proof of Proposition 2
Proof:
1 Since B = (B ∩ A) ∪ (B ∩ Ac ), it follows that
p(B) = p(B ∩ A) + p(B ∩ Ac ) (as the two sets are disjoint).
Thus p(A) + p(B) − p(A ∩ B) = p(A) + p(B ∩ Ac ).
Also, A ∪ B = A ∪ (B ∩ Ac ) (check!) and
this is a disjoint union. Hence
p(A) + p(B) − p(A ∩ B) = p(A) + p(B ∩ Ac ) = p(A ∪ B).
2 This follows from induction using (1).
12
References
• P. Hoel, S. Port, C. Stone, Introduction to Probability
Theory, 1st Edition, Brooks Cole, 1972.
13
Thank you for your attention!
14
