Counter-example

▶ Random exp: Pick a number uniformly from the real line.

▶ Ω = R and hence P(R) = 1.
▶ Domain P(R) which is unimaginably complex!
▶ We have P : P(R) → [0, 1].
▶ P has the property that sets of equal ’length’ have equal
probability.
∞
S
▶ We know that R = [n, n + 1) where [n, n + 1) ∈ P(R).
n=−∞
▶ What is P[n, n + 1)?
▶ If we define P[n, n + 1) = x for all n ∈ Z then P(R) = ∞!
▶ If we define P[n, n + 1) = 0 for all n ∈ Z then P(R) = 0!

20 / 70
Counter-example

▶ What is the takeaway from the counterexample?

▶ Not all set-functions (or measures) can be calibrated to
measure every possible subset of your sample space.
▶ This is like you weighing scale at home, that is not able to
weigh a piece of paper!
▶ What is the way out?
▶ Restrict your domain to only measurable sets.
▶ Possible domain for the counter example?
▶ F = {Φ, Ω, R− , R+ }. There can be many other domains one
can define!

21 / 70
Towards sigma-algebra

▶ F = {∅, Ω, R− , R+ }.

▶ The domain F should have some nice and obvious properties.

▶ For example, ∅ and Ω in F. . Also if B ∈ F, then B c ∈ F.

▶ If A1 and A2 belong in F, then A1 ∪ A2 ∈ F and A1 ∩ A2 ∈ F.

▶ A domain with such nice properties is called as a

sigma-algebra.

22 / 70
sigma-algebra as domain for P
▶ Event space or sigma-algebra F associated with a set Ω is a
collection of subsets of Ω that satisfy
• ∅ ∈ F and Ω ∈ F
•A ∈ F =⇒ Ac ∈ F
•A1 , A2 , . . . An , . . . ∈ F =⇒ ∪∞
n=1 An ∈ F

▶ The σ−algebra is said to be closed under formation of

complements and countable unions.

▶ Is it also closed under the formation of countable

intersections?

▶ When Ω is countable and finite, is P(Ω) a sigma-algebra? Yes.

When Ω is countable and finite, we will consider power-set

P(Ω) as the domain.

23 / 70
Formal definition of Probability measure P

Definition
A probability measure P on the measurable space (Ω, F) is a
function P : F → [0, 1] s.t.
1. P(∅) = 0, P(Ω) = 1
2. For a disjoint collection of event sets A1 , A2 , . . . from F we
have !
∞
[ ∞
X
P Ai = P(Ai )
i=1 i=1

(countable additivity)

▶ The trio (Ω, F , P) is called as a probability space.

▶ Recall that when |Ω| < ∞, we consider F = 2Ω .

24 / 70
Formal definition of Probability measure P

Definition
A probability measure P on the measurable space (Ω, F) is a
function P : F → [0, 1] s.t.
1. P(∅) = 0, P(Ω) = 1
2. For a disjoint collection of event sets A1 , A2 , . . . from F we
have !
∞
[ ∞
X
P Ai = P(Ai )
i=1 i=1

(countable additivity)

▶ The trio (Ω, F , P) is called as a probability space.

▶ Identify the probability space in the coin and dice experiment.

25 / 70
Probability space for U[0, 1]
▶ Ω = [0, 1].

▶ Suppose F = {Φ, [0, 1], [0, .5), [.5, 1]}. Is there a problem in
using this as a sigma-algebra?

▶ We cannot measure probability of sets like [.25, .75] although

we know P([.25, .75]) = .5.

▶ So lets include [.25, .75] in F.

▶ Now we have F + = {Φ, [0, 1], [0, .5), [.5, 1], [.25, .75]}. Is F +
a sigma-algebra? No.

▶ Can you make it a sigma-algebra by adding missing pieces ?

26 / 70
Probability space for U[0, 1]
▶ F + = {Φ, [0, 1], [0, .5), [.5, 1], [.25, .75]}
▶ Can you make it a sigma-algebra by adding missing pieces ?
▶ Recall that sigma-algebras are closed under complements,
union and intersection.
▶ Intersection and union of [.25, .75] with sets in F + gives the
collection {[.25, .5), [.5, .75], [.25, 1], [0, 0.75]}.
▶ Adding complements, the collection enlarges by
{[.25, .5), [.5, .75], [.25, 1], [0, 0.75], [0, .25) ∪ [.5, 1], [0, .5) ∪
(.75, 1], [0, .25), (0.75, 1]}.
▶ Lets call it F ++ =
{Φ, [0, 1], [0, .5), [.5, 1],[.25, .75], [.25, .5), [.5, .75], [.25, 1],
[0, 0.75], [0, .25) ∪ [.5, 1], [0, .5) ∪ (.75, 1], [0, .25), (0.75, 1]}

27 / 70
Probability space for U[0, 1]

▶ F ++ =
{Φ, [0, 1], [0, .5), [.5, 1],[.25, .75], [.25, .5), [.5, .75], [.25, 1],
[0, 0.75], [0, .25) ∪ [.5, 1], [0, .5) ∪ (.75, 1], [0, .25), (0.75, 1]}
▶ Notice different type of sets with different brackets [), (], ()
that appear.
▶ But F ++ is still not a sigma-algebra as each red set will
demand a furthermore sets to be added.
▶ This operation we attempted is called generating a
sigma-algebra!.
▶ Continuing on these lines, the resulting sigma algebra is called
a borel-sigma algebra B[0, 1].

28 / 70
Borel sigma-algebra B[0, 1]

▶ Borel σ−algebra B[0, 1]: When Ω = [0, 1] the B[0, 1] is the

σ−algebra generated by closed sets of the form [a, b] where
a ≤ b and a, b ∈ [0, 1].

▶ Does this set contain sets of the form (a, b) or [a, b) or (a, b]?

▶ (a, b) = ∪∞ 1 1 ∞ (a, b + 1 )
n=1 [a + n , b − n ]. (a, b] = ∩ n=1 n

Borel σ−algebra B[0, 1]: B[0, 1] is the σ−algebra generated

by sets of the form [a, b] or (a, b) or (a, b] or even [a, b)
where a ≤ b and a, b ∈ [0, 1].

29 / 70
Borel sigma-algebra B(R)
▶ Borel sigma-algebra B(R):
If Ω = R, then B(R) is the sigma-algebra generated by open
sets of the form (a, b) where a ≤ b and a, b ∈ R.

▶ B(R) contains intervals of the form

[a, b]
[a, b)
(a, ∞)
[a, ∞)
(−∞, b]
(−∞, b)
{a}

▶ How would you define B(R2 )?

30 / 70
Consequences of the Probability Axioms
▶ P(Ac ) = 1 − P(A)

▶ P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

▶ If A ⊆ B, prove that P(A) ≤ P(B). (A ⊆ B has the

interpretation that Event A implies event B)

P∞
▶ P(∪∞
i=1 Bi ) ≤ i=1 P(Bi ) (Boole’s/Bonferroni’s inequality).
HW

▶ What is P(A ∪ B ∪ C )?

▶ State and prove the inclusion-exclusion principle for P(∪ni=1 Ai )

31 / 70
Impossible event v/s Zero prob. event
▶ In U[0, 1] what is P(ω = 0.5) ? = 0.

▶ Intuitive reasoning for this is that a point has zero length!

▶ If P(ω ∈ [a, b]) = b − a then P([.5, .5]) = P({.5}) = 0.

▶ This is a zero probability event. In fact, every outcome of this

experiment is a zero probability event.

▶ This implies that events of zero probability can happen and

they are not impossible events.

▶ P(∅) = 0, then is ∅ also possible ?No!

▶ What is P(ω ∈ [0, .25] ∩ [.75, 1])?

▶ P([0, .25] ∩ [.75, 1]) = P(∅) = 0 This event will never happen.

32 / 70
Impossible event v/s Zero prob. event

[
▶ Note that in the U[0, 1] experiment, Ω = {ω}
ω∈Ω
[ X
▶ P(Ω) = P( {ω}) = P({ω}) = 0.
ω∈Ω ω∈Ω

▶ What is the problem above ?

▶ Ω is an uncountable set and the probability set-function only

has a countable additive property.
[
▶ {ω} is an uncountable disjoint union!
ω∈Ω

33 / 70
Limits and Continuity

▶ How do we define limit of a sequence {a1 , a2 , . . . , }?

▶ Notation: limn→∞ an = L.

▶ How do you define limit of a function at a point c?

▶ Notation: limx →c f (x ) = L

▶ How do you define continuity of a function f (x ) at c ?

▶ When do you say a function is continuous ?

▶ (ϵ, δ)-definition of limits and continuity?

34 / 70
Limits and Continuity

Definition in terms of limits of sequences.

For a continuous function f (·), as x → c, we have f (x ) →

f (c)

For a continuous set-function S, as An → A, we have

S(An ) → S(A)

▶ Recall that P is a set-function. Is it continuous?

▶ We will see the proof shortly.

35 / 70