Chapter 3

Probabilities
STAT-UB 103
Spring 2025
Outline

▪ Probability basics (Chapter 3.1)

▪ Operations on events (Chapters 3.2 – 3.4)

▪ Conditional probability (Chapters 3.5 – 3.7)

Chapter 3.1
Probability Basics
▪ Sample space and probability (3.1)
▪ Event & probability of an event (3.1)
▪ Expectation (4.2)
Chapter 3.1
Probability Basics
▪ Sample space and probability (3.1)
▪ Event & probability of an event (3.1)
▪ Expectation (4.2)
Sample Space
Sample space: The set of all possible outcomes
of a random experiment.

Experiment 1. Observe the up face on a coin.

Sample space: 1. Head (H). 2. Tail (T)
𝑺 = {𝐻, 𝑇}
Experiment 2. Observe the up face on a die.
Sample space: 𝑺 = {1, 2, 3,4,5,6}
Experiment 3. Observe the up faces on two coins.
Sample space: 𝑺 = {𝐻𝐻, 𝐻𝑇, 𝑇𝐻, 𝑇𝑇}
3-1-1
Sample Space (Cont’d)

Experiment 4.
Observe the
outcome of a spin
on an American
roulette wheel.

Sample space:
𝑺=
{0, 00, 1, 2, … , 36}

3-1-2
Probability
Probability: The likelihood that the outcome
will occur when the experiment is performed.

Example 1. Flipping a fair coin.

1 1
ℙ 𝐻 = , ℙ 𝑇 = .
2 2
Example 2. Rolling a fair die.
1
ℙ 𝑖 = , for 𝑖 = 1,2, … , 6.
6
Example 3. Flipping two fair coins.
1
ℙ 𝐻𝐻 = ℙ 𝐻𝑇 = ℙ 𝑇𝐻 = ℙ 𝑇𝑇 = .
4
3-1-3
Probability (Cont’d)

Example 4.
Spinning a
roulette wheel.

ℙ 0 = ℙ 00
=ℙ 1 =ℙ 2
= ⋯ = ℙ 36
1
= .
38

3-1-4
Sample Space
Sample space: The set of all possible outcomes
of a random experiment.

Experiment 1. Observe the up face on a coin.

Sample space: 1. Head (H). 2. Tail (T)
𝑺 = {𝐻, 𝑇}
Experiment 2. Observe the up face on a die.
Sample space: 𝑺 = {1, 2, 3,4,5,6}
Experiment 3. Observe the up faces on two coins.
Sample space: 𝑺 = {𝐻𝐻, 𝐻𝑇, 𝑇𝐻, 𝑇𝑇}
3-1-1
Probability
Probability: The likelihood that the outcome
will occur when the experiment is performed.

Example 1. Flipping a fair coin.

1 1
ℙ 𝐻 = , ℙ 𝑇 = .
2 2
Example 2. Rolling a fair die.
1
ℙ 𝑖 = , for 𝑖 = 1,2, … , 6.
6
Example 3. Flipping two fair coins.
1
ℙ 𝐻𝐻 = ℙ 𝐻𝑇 = ℙ 𝑇𝐻 = ℙ 𝑇𝑇 = .
4
3-1-3
Probability (Cont’d)

Example 5. Flipping a fair coin

until heads appear for the first time.
• What is the sample space 𝑺?
𝑺 = 𝐻, 𝑇𝐻, 𝑇𝑇𝐻, 𝑇𝑇𝑇𝐻, …
• What is the probability that the first heads
appears on the 1st flip?
• What is the probability that the first heads
appears on the 2nd flip?
• What is the probability that the first heads
appears on the 3rd flip?
3-1-5
Probability (Cont’d)

Example 5’. Flipping a biased coin

until heads appear for the first time.
It lands on heads with probability 1/3
and tails with probability 2/3.
• What is the probability that the first heads
appears on the 1st flip?
• What is the probability that the first heads
appears on the 2nd flip?
• What is the probability that the first heads
appears on the 3rd flip?
3-1-6
Probability (Cont’d)
Rule of Pig:
• Roll a six-sided die repeatedly
to accumulate points.
• Choose to stop and bank their points at any time.
• Rolling a 1 results in losing all points earned
and ends the turn.

Example 6.
• The probability of rolling a 1 on the first roll?
• The probability of rolling a 1 on the second roll?
3-1-7
Probability (Cont’d)
Example 7. 5 balls, 2 red and 3 blue.
You draw 1 ball at a time without replacement.
Consider the first time you get a red ball.
• What is the sample space 𝑺?
• What is the probability that the first red ball
appears on the 5th draw?
• What is the probability that the first red ball
appears on the 1st draw?
• What is the probability that the first red ball
appears on the 2st draw?
3-1-8
Probability (Cont’d)
Example 8. A standard deck of
52 playing cards contains
- 12 face cards (King, Queen, Jack)
- and 40 non-face cards.
You draw one card at a time without replacement.
Consider the first time you get a face card.
• What is the sample space 𝑺?

• What is the probability that the first face card

appears on the 2nd draw?

3-1-9
Understanding Probability
Through Coin Tossing
--- Probability, Frequency and
the Law of Large Numbers (LLN)
Frequency v.s. Probability
If we do not know the probability,
how to estimate it?
• Conduct experiments
• Approximate probability by observed frequency

Example (coin tossing).

Estimate the probability of getting heads.
• Conduct multiple coin tosses and record results.
Number of Heads Observed
• Estimated Probability = .
Total Tosses Conducted

3-1-10
Law of Large Numbers (LLN)
Law of Large Numbers (LLN):
As the number of trials increases, the sample
frequency approaches the theoretical probability.

Number of Heads Observed

→ ℙ 𝐻 = 0.5,
Total Tosses Conducted
as #Toss → ∞.

Population v.s. Sample

3-1-12
Population v.s. Sample
Population:
The collection of all
possible coin tosses
that could ever happen,
stretching into infinity.

3-1-13
Population v.s. Sample
Sample:
A small subset of tosses
we actually perform
(e.g., 10, 50, or 100 tosses).

3-1-13
Population v.s. Sample

Statistical
inference
Observed Population
frequency probability

3-1-14
Population v.s. Sample
Example. Population v.s. Sample
Customer surveys All customers Survey responses
Election polling All eligible voters Surveyed voters
Quality control All products Inspected items
Product reviews All reviews Reviews from last quarter
Customer complaints All complaints Comp. from 1 product line

Population: The entire group you want to learn about.

Sample: A subset of population used to make inferences.
The larger and more representative the sample,
the better it approximates the population.
3-1-15
Chapter 3.1
Probability Basics
▪ Sample space and probability (3.1)
sample space, probability,
LLN, population vs. sample
▪ Event & probability of an event (3.1)
▪ Expectation (4.2)
Chapter 3.1
Probability Basics
▪ Sample space and probability (3.1)
▪ Event & probability of an event (3.1)
▪ Expectation (4.2)
Event
Event: A specific collection of sample points.
The probability of an event 𝑨:
Sum the probabilities of the sample points in 𝑨.

Example 1. We are tossing 2 fair coins.

What is the probability of getting at least one Head?

Example 2. We are rolling two dice.

What is the probability that the sum of the numbers on
the dice equals 7?

3-1-16
Examples of Event
Example 2. We are rolling two dice.
What is the probability that the sum of the numbers on
the dice equals 7?

3-1-17
Examples of Event
Example 3 (3.5 on page 140). USA Today outlines key
reasons for its inclusion in strategic plans. The table summarizes
these reasons. Suppose a business using diversity training is
randomly selected, and its primary reason is determined.
Reason Percentage
Comply with personnel policies (CPP) 7
Increase productivity (IP) 47
Stay competitive (SC) 38
Social responsibility (SR) 4
Other (O) 4
Total 100

3-1-18
Examples of Event
Example 3.
Reason Percentage
Comply with personnel policies (CPP) 7
Increase productivity (IP) 47
Stay competitive (SC) 38
Social responsibility (SR) 4
Other (O) 4
Total 100

Question (a): Define the experiment that generated the data

in table and list the sample points.

3-1-19
Examples of Event
Example 3.
Reason Percentage
Comply with personnel policies (CPP) 7
Increase productivity (IP) 47
Stay competitive (SC) 38
Social responsibility (SR) 4
Other (O) 4
Total 100

Question (b): Assign probabilities to the sample points.

3-1-20
Examples of Event
Example 3.
Reason Percentage
Comply with personnel policies (CPP) 7
Increase productivity (IP) 47
Stay competitive (SC) 38
Social responsibility (SR) 4
Other (O) 4
Total 100

Question (c): Probability of a business-related reason,

that is, competition or productivity?

3-1-21
Examples of Event
Example 3.
Reason Percentage
Comply with personnel policies (CPP) 7
Increase productivity (IP) 47
Stay competitive (SC) 38
Social responsibility (SR) 4
Other (O) 4
Total 100

Question (d): Probability of not social responsibility?

3-1-22
Recap: Probability

Example 4.
Spinning a
roulette wheel.

ℙ 0 = ℙ 00
=ℙ 1 =ℙ 2
= ⋯ = ℙ 36
1
= .
38

3-1-4
Examples of Event
Example 4.
Spinning a roulette wheel.
Calculate winning rate for
different ways of betting:

Straight: A single number.

3-1-23
Examples of Event
Example 4.
Spinning a roulette wheel.
Calculate winning rate for
different ways of betting:

Split: Two adjacent numbers.

3-1-24
Examples of Event
Example 4.
Spinning a roulette wheel.
Calculate winning rate for
different ways of betting:

Corner: A block of 4 numbers.

3-1-25
Examples of Event
Example 4.
Spinning a roulette wheel.
Calculate winning rate for
different ways of betting:

Dozen: 1-12, 13-24 or 25-36.

3-1-26
Chapter 3.1
Probability Basics
▪ Sample space and probability (3.1)
▪ Event & probability of an event (3.1)
▪ Expectation (4.2)
Chapter 3.1
Probability Basics
▪ Sample space and probability (3.1)
▪ Event & probability of an event (3.1)
▪ Expectation (4.2)
 Gain from Roulette
1st Dozen: 1-12.
If we bet $1 on the 1st dozen 100 times,
what would be our average gain (or loss)?
Payout for the 1st dozen: 2 : 1

Recall
st
6
ℙ win on 1 dozen = ≈ 31.58%.
19
LLN ⇒ Win ≈ 31.58 times;
Lose 100 − 31.58 = 68.42 times.

3-1-27
 Gain from Roulette
1st Dozen: 1-12.
If we bet $1 on the 1st dozen 100 times,
what would be our average gain (or loss)?
Payout for the 1st dozen: 2 : 1

LLN ⇒ Win ≈ 31.58 times;

Lose 100 − 31.58 = 68.42 times.
⇒ $2 × 31.58 − $1
× 68.42
= −$5.26
Average loss: $0.0526/time
3-1-28
Gain from Roulette
Interpret: Average loss: $0.0526/time

Samples: 𝑥1 = $2 (win),
𝑥2 = −$1 (loss), 𝑥3 = −$1,
-$1, $2, $2, -$1, -$1, -$1, ……
𝑛
1
⇒ sample mean 𝑥ҧ = ෍ 𝑥𝑖
𝑛
𝑖=1
LLN ⇒ converge to (for large 𝑛)
population mean 𝜇 = −$0.0526.
also named mean, expectation
3-1-29
Expectation
Expectation: The mean, or expected value, of
a random variable 𝑋 is
𝜇 = 𝔼 𝑋 = σ𝑥 ⋅ℙ 𝑥 .
sum over all possible values of 𝑋

Example (Roulette). Bet on the first dozen.

𝑥 ℙ 𝑥 𝑥⋅ℙ 𝑥
2 6/19 12/19
−1 13/19 −13/19
Total: 𝜇 = −1/19 ≈ −0.0526
3-1-30
Expectation (Cont’d)
Example (Roulette).
Calculate the expectation 𝜇
for each way of betting.
Straight: A single number.
Payout: 35 ∶ 1
Split: Two adjacent numbers.
Payout: 17 ∶ 1
Corner: A block of 4 numbers.
Payout: 8 ∶ 1

3-1-31
Expectation (Cont’d)
Example (Roulette).
Calculate the expectation 𝜇
for each way of betting.
Straight: A single number.
Payout: 35 ∶ 1
Split: Two adjacent numbers.
House edge:
Payout: 17 ∶ 1
You are always losing money to the house
Corner: A block of 4 numbers.
in expectation!
Payout: 8 ∶ 1

3-1-31
Expectation (Cont’d)
Example (Slot machine).
2 reels. 10 symbols per reel.
Jackpot: 2 reels show the same symbol → Win $8.
No match: No 2 reels match → Lose $1.
• What is ℙ Jackpot ?
• What is ℙ No match ?
• Find the expected value of a single play.

3-1-32
Chapter 3.1
Probability Basics
▪ Sample space and probability (3.1)
▪ Event & probability of an event (3.1)
▪ Expectation (4.2)
Chapter 3.1
Probability Basics
▪ Sample space and probability (3.1)
sample space, probability,
LLN, population vs. sample
▪ Event & probability of an event (3.1)
▪ Expectation (4.2)
Chapters 3.2 – 3.4
Operations on Events
 Complementary events (3.3)
 Intersection & mutually exclusive events (3.2 & 3.4)
 Union & the additive rule (3.2 & 3.4)
Chapters 3.2 – 3.4
Operations on Events
 Complementary events (3.3)
 Intersection & mutually exclusive events (3.2 & 3.4)
 Union & the additive rule (3.2 & 3.4)
Complementary Events
w: awake students
z : asleep students
• Venn Diagram

w Complementary
w Event 𝑨 Event 𝑨𝒄
z w w
z w
w z w w
Sample space 𝑺
3-2-1
Complementary Events (Cont’d)
Complement of event 𝑨 = 𝑨𝒄
= 𝑨 does not occur.

• Venn Diagram
Complementary
Event 𝑨𝒄
Event 𝑨

Sample space 𝑺
3-2-2
Probability of Complementary Events

ℙ 𝑨𝒄 = 1 − ℙ 𝑨

Exercise (3.11 on page 152).

Consider the experiment of tossing fair coins.
Define the following event:
𝑨 = Observe at least one head .
Question (a): Find ℙ 𝑨 if 2 coins are tossed.
Question (b): Find ℙ 𝑨 if 10 coins are tossed.

3-2-3
Probability of Complementary Events
Exercise. A restaurant finds that 85% of customers who
visit at dinner time order a drink.
Question: If 3 customers are randomly selected,
what is the probability that at least one of them
does not order a drink?
----------------------------------------------------------------------------------
Solution:

3-2-4
Probability of Complementary Events

Exercise. A hacker tries 100 different passwords, and

each attempt has a 1% chance of success.
Question: What is the probability that the hacker
succeeds at least once?
---------------------------------------------------------------------------
Solution:

3-2-5
Chapters 3.2 – 3.4
Operations on Events
 Complementary events (3.3)
 Intersection & mutually exclusive events (3.2 & 3.4)
 Union & the additive rule (3.2 & 3.4)
Chapters 3.2 – 3.4
Operations on Events
 Complementary events (3.3)
 Intersection & mutually exclusive events (3.2 & 3.4)
 Union & the additive rule (3.2 & 3.4)
Intersection
Intersection of events 𝑨 and 𝑩 = 𝑨 ∩ 𝑩
= Both of 𝑨 and 𝑩 occur.

• Venn Diagram

Event 𝑨 Event 𝑩

𝑨∩𝑩
3-2-6
Intersection (Cont’d)

Exercise. A fair six-sided die is rolled.

• Let 𝑨 be the event of rolling an even number.
• Let 𝑩 be the event of rolling a number greater than 3.

Question: Find ℙ 𝑨 ∩ 𝑩 .

3-2-7
Intersection (Cont’d)

Exercise.
A card is drawn from a standard 52-card deck.
• Let 𝑨 be the event that the card is a red card.
• Let 𝑩 be the event that the card is a face card
(Jack, Queen, or King).

Question: Find ℙ 𝑨 ∩ 𝑩 .

3-2-8
Intersection (Cont’d)

Event 𝑨 Event 𝑩

Exercise. Which of the following represents

the shaded region?
(A) 𝑨 ∩ 𝑩 (B) 𝑨𝒄 ∩ 𝑩
(C) 𝑨 ∩ 𝑩𝒄 (D) 𝑨𝒄 ∩ 𝑩𝒄

3-2-9
Intersection (Cont’d)

Event 𝑨 Event 𝑩

Sample space 𝑺
Exercise. Which of the following represents
the shaded region?
(A) 𝑨 ∩ 𝑩 (B) 𝑨𝒄 ∩ 𝑩
(C) 𝑨 ∩ 𝑩𝒄 (D) 𝑨𝒄 ∩ 𝑩𝒄
3-2-10
Mutually Exclusive Events
Events 𝑨 and 𝑩 are mutually exclusive
if 𝑨 ∩ 𝑩 contains no sample points.
ℙ 𝑨 ∩ 𝑩 = 0.
• Venn Diagram

Event 𝑨 Event 𝑩

3-2-11
Mutually Exclusive Events (Cont’d)
Exercise. If two events are mutually exclusive,
their intersection is always zero.
True or False

Exercise. If two events are mutually exclusive,

their probabilities add up to 1.
True or False
• What do you call two events whose probabilities add up to 1?
3-2-12
Mutually Exclusive Events (Cont’d)
Exercise. {Rolling a 5} and
{Rolling an even number}
on a six-sided die are mutually exclusive events.
True or False

Exercise. {Rolling an even number} and

{Rolling a prime number}
on a six-sided die are mutually exclusive events.
True or False

3-2-13
Chapters 3.2 – 3.4
Operations on Events
 Complementary events (3.3)
 Intersection & mutually exclusive events (3.2 & 3.4)
 Union & the additive rule (3.2 & 3.4)
Chapters 3.2 – 3.4
Operations on Events
 Complementary events (3.3)
 Intersection & mutually exclusive events (3.2 & 3.4)
 Union & the additive rule (3.2 & 3.4)
Union
Union of events 𝑨 and 𝑩 = 𝑨 ∪ 𝑩
= Either one or both of 𝑨 and 𝑩 occur.

• Venn Diagram

Event 𝑨 Event 𝑩

𝑨∪𝑩
3-2-14
Intersection & Union

Intersection = 𝑨 ∩ 𝑩 Union = 𝑨 ∪ 𝑩
= 𝑨 and 𝑩 occur. = 𝑨 or 𝑩 occur.

Event 𝑨 Event 𝑩 Event 𝑨 Event 𝑩

𝑨∩𝑩 𝑨∪𝑩

3-2-15
Union (Cont’d)

Exercise. A six-sided die is rolled.

• Let 𝑨 = {Rolling an even number}.
• Let 𝑩 = {Rolling a number greater than 3}.

Question: Find ℙ 𝑨 ∪ 𝑩 .

3-2-16
Union (Cont’d)
Exercise. 5 red balls, 3 blue balls & 2 green balls.
A marble is randomly drawn.
• Let 𝑨 = {Drawing a red marble}.
• Let 𝑩 = {Drawing a blue marble}.
Question: Find ℙ 𝑨 ∪ 𝑩 .

If events 𝑨 and 𝑩 are mutually exclusive, then

ℙ 𝑨∪𝑩 = ℙ 𝑨 + ℙ 𝑩 .

3-2-17
Union (Cont’d)

Event 𝑨 Event 𝑩

Sample space 𝑺
Exercise. Which of the following represents
the shaded region?
(A) 𝑨 ∩ 𝑩 𝒄 (B) 𝑨𝒄 ∪ 𝑩𝒄
(C) 𝑨 ∪ 𝑩 𝒄 (D) 𝑨𝒄 ∩ 𝑩𝒄
3-2-18
Union (Cont’d)

Event 𝑨 Event 𝑩

Sample space 𝑺
Exercise. Which of the following represents
the shaded region?
(A) 𝑨 ∩ 𝑩 𝒄 (B) 𝑨𝒄 ∪ 𝑩𝒄
(C) 𝑨 ∪ 𝑩 𝒄 (D) 𝑨𝒄 ∩ 𝑩𝒄
3-2-19
Union (Cont’d)

Exercise. A survey of 10 students found that:

• 6 students enrolled in Finance
• 5 students enrolled in Economics
• 2 students enrolled in both Finance and Economics

Question: Find ℙ 𝑨 ∪ 𝑩 .

3-2-20
Venn Diagram

𝑭inance 𝑬conomics

Sample space 𝑺

Example (Cont’d). • 2 in both Finance and Econ.

3-2-21
Venn Diagram

𝑭inance 𝑬conomics

Sample space 𝑺

Example (Cont’d). • 6 in Finance.

3-2-21
Venn Diagram

𝑭inance 𝑬conomics

Sample space 𝑺

Example (Cont’d). • 5 in Economics.

3-2-21
Venn Diagram

𝑭inance 𝑬conomics

Sample space 𝑺

Example (Cont’d). # 𝑭 ∪ 𝑬 =𝟗
3-2-21
Venn Diagram

𝑭inance 𝑬conomics

Sample space 𝑺
Example (Cont’d). • 10 in total.
⇒ ℙ 𝑭 ∪ 𝑬 = 9/10
3-2-21
Additive Rule

𝑭inance 𝑬conomics

Sample space 𝑺

ℙ 𝑭∪𝑬 =ℙ 𝑭 +ℙ 𝑬 −ℙ 𝑭∪𝑬
3-2-21
Additive Rule
The additive rule:
ℙ 𝑨 ∪ 𝑩 = ℙ 𝑨 + ℙ 𝑩 − ℙ(𝑨 ∩ 𝑩)

• Venn Diagram

Event 𝑨 Event 𝑩

3-2-22
Additive Rule (Cont’d)

Exercise. A survey of 100 MBA graduates revealed:

• 45 graduates prefer Investment Banking
• 32 graduates prefer Consulting
• 18 graduates are open to both careers

Question:
Find ℙ prefer at least one of these two career paths .

3-2-23
Additive Rule (Cont’d)
Exercise. NYU library tracks book borrowing trends
among students. The data show that:
• With a prob. 60%, a student borrows a fiction book.
• With a probability 50%, a student borrows a textbook.
• With prob. 30%, a student borrows a fiction and
a textbook.

Question:
Find ℙ borrows either fiction or nonfiction .

3-2-24
Additive Rule (Cont’d)
Exercise. Students are competing for top investment banking
internships. To improve their chances:
• 60% → The prob. that a student attends networking events.
• 50% → The prob. that a student completes a stat course.
• 75% → The prob. that a student participates in
either networking or coursework.

Question (a): What is the probability that a student does

neither and hopes for luck?
Question (b): What is the prob. that a student is hardworking
and ambitious, doing both networking and coursework?

3-2-24
Additive Rule (Cont’d)
Exercise. Captain Mad-Eye Morgan and his crew have
landed on Skull Island, where the legendary Black Serpent’s
Treasure is hidden. But the gold is split between two
dangerous locations:
• 55% → The probability that a pirate dives into the ocean
to search the sunken shipwreck.
• 45% → The probability that a pirate ventures into the jungle
to explore the ancient ruins.
• 30% → The prob. that a pirate sits back and does nothing.

The Captain’s Question: What is the probability that a pirate

is hardworking and determined, searching both the ocean and
the jungle for treasure?

3-2-24
Additive Rule (Cont’d)
Exercise. Earth is preparing for an alien invasion, and
governments are considering 3 defensive strategies:
𝑨: Develop laser-based planetary shields. ℙ(𝑨) = 0.55
𝑩: Build underground survival bunkers. ℙ(𝑩) = 0.50
𝑪: Attempt peace negotiations. ℙ 𝑪 = 0.30
• Since diplomatic efforts exclude military defense,
𝑪 is mutually exclusive to 𝑨 and 𝑩.
• Some governments invest in both shields and bunkers,
so ℙ(𝑨 ∩ 𝑩) = 0.40.
Question: What is the probability that the country
chooses at least one defense strategy?
3-2-25
Additive Rule (Cont’d)
Exercise. A space agency is sending astronauts on
three different mission types:
𝑨: Lunar Exploration. ℙ(𝑨) = 0.50
𝑩: Mars Colonization. ℙ(𝑩) = 0.30
𝑪: AI Robotics Research. ℙ 𝑪 = 0.40
• A crew cannot go to both the Moon and Mars
at the same time.
• AI can be used for both lunar and Mars research,
ℙ(𝑨 ∩ 𝑪) = 0.25, ℙ(𝑩 ∩ 𝑪) = 0.20.
Question: What is the probability that an astronaut
is assigned to at least one mission?
3-2-26
Chapters 3.2 – 3.4
Operations on Events
 Complementary events (3.3)
 Intersection & mutually exclusive events (3.2 & 3.4)
 Union & the additive rule (3.2 & 3.4)
Chapters 3.2 – 3.4
Operations on Events
 Complementary events (3.3)
 Intersection & mutually exclusive events (3.2 & 3.4)
 Union & the additive rule (3.2 & 3.4)
Chapters 3.5 – 3.7
Conditional Probability
▪ Conditional probability (3.5)
▪ Independence (3.6)
▪ Bayes’ rule (3.7)
Chapters 3.5 – 3.7
Conditional Probability
▪ Conditional probability (3.5)
▪ Independence (3.6)
▪ Bayes’ rule (3.7)
 1 𝟏 1 1 𝟏
𝟎 𝟐
3 𝟐 3 3

The cookie thief was wearing a red scarf!

Probabilities changed once we got new information.

3-3-1
Example
Example. The outcomes of two coin flips:

𝑯𝑯
𝐻𝐻 𝐻𝑇
𝑇𝐻 𝑇𝑇
′
1
Question: The probability of getting two 𝑯’s? ℙ two 𝑯 s = .
4
---------------------------------------------------------------------------
Question: If we know that there is already one 𝑯,
how does the probability of getting two 𝑯’s change?
ℙ two 𝑯′s 1
= .
ℙ already has one 𝑯 3

3-3-2
Conditional Probability

Conditional probability:

ℙ 𝐴
𝑨|𝑩
𝐵
The likelihood of an event occurring
given that another event has already occurred.

3-3-3
Conditional Probability Formula

Conditional probability formula:

ℙ 𝐴∩𝐵
ℙ 𝐴 𝐵 =
ℙ 𝐵
The likelihood of an event occurring
given that another event has already occurred.

3-3-4
Conditional Probability (Cont’d)
Example. A fair six-sided die is rolled twice. Let
• 𝑨 be the event that the sum of the two rolls is at least 10.
• 𝑩 be the event that the first roll is a 6.
Question: Find ℙ(𝑨 | 𝑩), the probability that
the sum is at least 10 given that the first roll is a 6.
1 2 3 4 5 6
1 1+1=2 1+2=3 1+3=4 1+4=5 1+5=6 1+6=7
2 2+1=3 2+2=4 2+3=5 2+4=6 2+5=7 2+6=8
3 3+1=4 3+2=5 3+3=6 3+4=7 3+5=8 3+6=9
4 4+1=5 4+2=6 4+3=7 4+4=8 4+5=9 4+6=10
5 5+1=6 5+2=7 5+3=8 5+4=9 5+5=10 5+6=11
6 6+1=7 6+2=8 6+3=9 6+4=10 6+5=11 6+6=12

3-3-5
Conditional Probability (Cont’d)
Example. A fair six-sided die is rolled twice. Let
• 𝑨 be the event that the sum of the two dice is 7.
• 𝑩 be the event that at least one die shows a 3.
Question: Find ℙ(𝑨 | 𝑩), the probability that
the sum is 7 given that at least one die is a 3.
1 2 3 4 5 6
1 1+1=2 1+2=3 1+3=4 1+4=5 1+5=6 1+6=7
2 2+1=3 2+2=4 2+3=5 2+4=6 2+5=7 2+6=8
3 3+1=4 3+2=5 3+3=6 3+4=7 3+5=8 3+6=9
4 4+1=5 4+2=6 4+3=7 4+4=8 4+5=9 4+6=10
5 5+1=6 5+2=7 5+3=8 5+4=9 5+5=10 5+6=11
6 6+1=7 6+2=8 6+3=9 6+4=10 6+5=11 6+6=12

3-3-6
Simpson’s Paradox
A university admissions office provides the following data on
applications and admissions for two departments, A and B.
Men Men Women Women
Dept.
Admitted Applied Admitted Applied
A 80 100 9 10
B 10 50 30 100
Total 90 150 39 110

Example. (a) Overall admission prob. ℙ 𝐴𝑑 𝑀 , ℙ 𝐴𝑑 𝑊 ?

(b) Department-wise admission probabilities
ℙ 𝐴𝑑 𝑀, 𝐷𝑒𝑝𝑡 𝐴 , ℙ 𝐴𝑑 𝑊, 𝐷𝑒𝑝𝑡 𝐴 ,
ℙ 𝐴𝑑 𝑀, 𝐷𝑒𝑝𝑡 𝐵 , ℙ 𝐴𝑑 𝑊, 𝐷𝑒𝑝𝑡 𝐵 ?
3-3-8
Simpson’s Paradox
A university admissions office provides the following data on
applications and admissions for two departments, A and B.
Men Men Women Women
Dept.
Admitted Applied Admitted Applied
A 80 100 90 100
B 10 50 300 1000
Total 90 150 390 1100

Example. (a) Overall admission prob.

ℙ 𝐴𝑑 𝑀 = 60%, ℙ 𝐴𝑑 𝑊 = 35.5%.
(b) Department-wise admission probabilities
ℙ 𝐴𝑑 𝑀, 𝐷𝑒𝑝𝑡 𝐴 = 80%, ℙ 𝐴𝑑 𝑊, 𝐷𝑒𝑝𝑡 𝐴 = 90%,
ℙ 𝐴𝑑 𝑀, 𝐷𝑒𝑝𝑡 𝐵 = 20%, ℙ 𝐴𝑑 𝑊, 𝐷𝑒𝑝𝑡 𝐵 = 30%?
Simpson’s Paradox (Cont’d)
Men Men Women Women
Dept.
Admitted Applied Admitted Applied
A 80 100 9 10
B 10 50 30 100

Example. (c) Interpreting Simpson’s Paradox:

• Comparing the overall probabilities, who appears to have
a higher admission rate: men or women?
• Comparing within each department, who has a higher
admission rate?
• Why do the overall and department-wise results seem
contradictory?
3-3-9
Conditional Probability (Cont’d)
Example. In a class,
• 70% of students study for the exam.
• 90% of those who study pass the exam.
• 50% of those who do not study pass the exam.
(a) What is the probability that a randomly selected student
studies and passes the exam?
(b) What is the probability that a randomly selected student
does not study but still passes the exam?
(c) What is the probability that a randomly selected student
passes the exam?

3-3-10
Multiplicative Rule of Probability
Conditional probability formula:
ℙ 𝐴∩𝐵
ℙ 𝐴 𝐵 =
ℙ 𝐵

Multiplicative rule of probability:

ℙ 𝐴∩𝐵 =ℙ 𝐵 ℙ 𝐴 𝐵
or, equivalently,
ℙ 𝐴∩𝐵 =ℙ 𝐴 ℙ 𝐵 𝐴
3-3-11
Law of Total Probability

Event 𝑨 Event 𝑩

𝑨 ∩ 𝑩𝒄 𝑨∩𝑩

ℙ 𝑨 = ℙ 𝐘𝐞𝐥𝐥𝐨𝐰 + ℙ 𝐁𝐥𝐮𝐞
ℙ 𝑨 ∩ 𝑩𝒄 ℙ 𝑨∩𝑩
3-3-12
Law of Total Probability

Event 𝑨 Event 𝑩

𝑨 ∩ 𝑩𝒄 𝑨∩𝑩
Law of total probability:
𝒄
ℙ 𝑨 =ℙ 𝑨∩𝑩 +ℙ 𝑨∩𝑩
𝒄 𝒄
=ℙ 𝑩 ℙ 𝑨 𝑩 +ℙ 𝑩 ℙ 𝑨 𝑩
3-3-12
Law of Total Probability (Cont’d)
Exercise. A weather service reports that:
• On normal days, it rains 10% of the time.
• On stormy days, it rains 80% of the time.
• The probability of a normal day is 70%, and
the probability of a stormy day is 30%.
Question: What is the probability that it rains
on a randomly chosen day?
---------------------------------------------------------------------------
Solution. ℙ Rain = ℙ Normal ℙ Rain Normal
+ℙ Stormy ℙ Rain Stormy
= 70% × 10% + 30% × 80% = 31%
.
3-3-13
Chapters 3.5 – 3.7
Conditional Probability
▪ Conditional probability (3.5)
Conditional probability
Multiplicative rule of probability
Law of total probability
▪ Independence (3.6)
▪ Bayes’ rule (3.7)
Chapters 3.5 – 3.7
Conditional Probability
▪ Conditional probability (3.5)
▪ Independence (3.6)
▪ Bayes’ rule (3.7)
Are These Events Related?
Does flipping heads tell you anything
about what number the die will show?

If not… congratulations!
You just understood independence!

3-3-15
Independence
Events 𝑨 and 𝑩 are independent:
The occurrence of 𝑩 does not alter the
probability that 𝑨 has occurred, i.e.
ℙ 𝑨 𝑩 =ℙ 𝑨 .

Equivalent form:
ℙ 𝑩 𝑨 =ℙ 𝑩
ℙ 𝑨∩𝑩 =ℙ 𝑨 ℙ 𝑩
3-3-16
Independence (Cont’d)
Example. A bag contains 3 red and 2 blue marbles.
You draw one marble, then put it back and draw again.
Define events 𝑅 = “A blue marble on 1st draw.” and
𝐵 = “A red marble on 2nd draw”.
Question: (a) What is ℙ 𝑅 ? the probability of
drawing a red marble in the first draw.
(b) What is ℙ 𝐵 ? the probability of
drawing a blue marble in the second draw.
(c) What is ℙ 𝑅 ∩ 𝐵 ? the probability of
drawing a red marble on the first draw
and a blue marble on the second draw.
(d) Are the events independent?

3-3-17
Independence (Cont’d)
Example. A bag contains 3 red and 2 blue marbles.
You draw one marble without replacement and draw again.
Define events 𝑅 = “A blue marble on 1st draw.” and
𝐵 = “A red marble on 2nd draw”.
Question: (a) What is ℙ 𝑅 ? the probability of
drawing a red marble on the first draw.
(b) What is ℙ 𝐵 ? the probability of
drawing a blue marble on the second draw.
(c) What is ℙ 𝑅 ∩ 𝐵 ? the probability of
drawing a red marble on the first draw
and a blue marble on the second draw.
(d) Are the events independent?

3-3-18
Independence (Cont’d)
Exercise. If events don’t overlap,
they are independent.
True or False

Exercise. If events are mutually exclusive,

they must be independent.
True or False

3-3-19
Independence (Cont’d)
Exercise. A bag contains 8 marbles in red and blue, with 4
marbles of each color. In each color group, exactly 1 marble is
marked as “Special”. One marble is drawn at random.
Define the events: 𝑨 = “The drawn marble is a Red” and
𝑩 = “The drawn marble is a Special”.
Question (a): Show that 𝑨 and 𝑩 are independent:
• Compute ℙ(𝑨), ℙ(𝑩), and ℙ(𝑨 ∩ 𝑩).
• Check if ℙ 𝑨 ∩ 𝑩 = ℙ 𝑨 ℙ(𝑩).
Question (b): Find ℙ 𝑨𝒄 and ℙ 𝑩𝒄 .
Question (c): Find ℙ 𝑨𝒄 ∩ 𝑩𝒄 and verify if 𝑨𝒄 and 𝑩𝒄 are
ℙ 𝑨𝒄 ∩ 𝑩𝒄 = ℙ 𝑨𝒄 ℙ 𝑩𝒄 . independent!

3-3-20
Independence (Cont’d)
Exercise. A standard deck of 52 playing cards is shuffled,
and one card is drawn at random. Define the events:
• 𝑨 = “The card is a Heart” (13 hearts in the deck).
• 𝑩 = “The card is a Face card (Jack, Queen, or King)”.
Question (a): Show that 𝑨 and 𝑩 are independent:
• Compute ℙ(𝑨), ℙ(𝑩), and ℙ(𝑨 ∩ 𝑩).
• Check if ℙ 𝑨 ∩ 𝑩 = ℙ 𝑨 ℙ(𝑩).
Question (b): Find the probabilities of their complements:
ℙ 𝑨𝒄 and ℙ 𝑩𝒄 .
Question (c): Find ℙ 𝑨𝒄 ∩ 𝑩𝒄 and verify if 𝑨𝒄 and 𝑩𝒄 are
ℙ 𝑨𝒄 ∩ 𝑩𝒄 = ℙ 𝑨𝒄 ℙ 𝑩𝒄 . independent!

3-3-21
Independence (Cont’d)

Summary:
If events 𝑨 and 𝑩 are independent,
then 𝑨𝒄 and 𝑩𝒄 ,
𝑨𝒄 and 𝑩,
𝑨 and 𝑩𝒄
are independent.

3-3-21
Chapters 3.5 – 3.7
Conditional Probability
▪ Conditional probability (3.5)
▪ Independence (3.6)
▪ Bayes’ rule (3.7)
Chapter 3
Probabilities
▪ Probability basics (Chapters 3.1 & 4.2)
Sample space & probability
Event; Expectation
▪ Operations on events (Chapters 3.2 – 3.4)
Complementary events; Intersection;
Union & the additive rule
▪ Conditional probability (Chapters 3.5 – 3.7)
Conditional probability & independence
Bayes’ rule