Bayes Theorem
Presented by:
Engr. Rogelio C. Golez, Jr
Mechanical Engineering Dept.
Theorem of Total Probability
• If the events B1, B2, …, Bk constitute a
partition of the sample space S such that
P(Bi) ≠ 0 for I = 1, 2, …, k, then for any
event A of S.
k k
P(A) = P( B A) P( B ) P( A \ B )
i 1
i
i 1
i i
EXAMPLE PROBLEMS
In certain assembly plant, three machines,
B1, B2, and B3, make 30%, 45% and 25%,
respectively, of the products. It is known
from past experience that 2%, 3% and 2%
of the products made by each machine,
respectively, are defective. Now, suppose
that a finished product is randomly
selected. What is the probability that it is
defective?
Solution:
• Consider the following:
– A: the product is defective,
– B1: the product is made by machine B1,
– B2: the product is made by machine B2,
– B3: the product is made by machine B3
Solution:
B1 P(A\B1) = 0.02
0
0.3
)=
B 1
P(
P(B2) = 0.45 B2 P(A\B2) = 0.03
P(
B
3 )=
0.
25
P(A\B3) = 0.02
B3
Applying the rule of elimination, we can write
Let P(A) = is the probability that the product is defective.
P(A) = P(B1)P(A\B1)+P(B2)P(A\B2)+P(B3)P(A\B3)
P(A) = (0.30)(0.02) + (0.45)(0.03) + (0.25)(0.02) = 0.0245
Bayes’ Rule: derivation
• Definition:
Let A and B be two events with P(B) 0.
The conditional probability of A given B is:
P( A & B)
P( A / B)
P( B)
The idea: if we are given that the event B occurred, the relevant sample space is
reduced to B {P(B)=1 because we know B is true} and conditional probability becomes
a probability measure on B.
Bayes’ Rule: derivation
P( A & B)
P( A / B)
P( B)
can be re-arranged to:
P( A & B) P( A / B) P( B)
P( A & B)
and, since also: P ( B / A) P( A & B) P( B / A) P ( A)
P ( A)
P ( A / B ) P( B ) P( A & B) P( B / A) P ( A)
P ( A / B ) P( B ) P( B / A) P( A)
P ( B / A) P( A)
P( A / B)
P( B)
Bayes’ Rule:
P( B / A) P( A)
P( A / B)
P( B)
OR
P( B / A) P( A)
P( A / B) From the
P( B / A) P( A) P( B / ~ A) P(~ A)
“Law of Total
Probability”
Bayes Theorem
Let {B1, B2, …, Bn} be a subset of the sample space S of an
experiment.
If for i = 1, 2, …, n, P (Bi) > 0, then for any event A of S with
P (A) > 0,
P ( A | Bk ) P ( Bk )
P (Bk | A)=
P( A | B1 ) P ( B1 ) P ( A | B2 ) P ( B2 ) ... P( A | Bn ) P( Bn )
P(B) is the prior probability and P(B | A) is the posterior
probability
In-Class Exercise
• If HIV has a prevalence of 3% in San Francisco,
and a particular HIV test has a false positive rate
of .001 and a false negative rate of .01, what is
the probability that a random person who tests
positive is actually infected (also known as
“positive predictive value”)?
Answer: using probability tree
P(test +)=.99 P (+, test +)=.0297
P(+)=.03
P(test - = .01)
P(+, test -)=.003
P(test +) = .001
P(-, test +)=.00097
P(-)=.97
P(-, test -) = .96903
P(test -) = .999
______________
1.0
A positive test places one on either of the two “test +” branches.
But only the top branch also fulfills the event “true infection.”
Therefore, the probability of being infected is the probability of being on the top
branch given that you are on one of the two circled branches above.
P (test &true ) .0297
P ( / test ) 96.8%
P (test ) .0297 .00097
Answer: using Bayes’ rule
P(test / true ) P (true )
P(true / test )
P(test / true ) P (true ) P (test / true) P(true)
.99(.03)
96.8%
.99(.03) .001(.97)
• A box contains seven red and thirteen blue balls. Two
balls are selected at random and are discarded without
their colors being seen. If a third ball is drawn randomly
and observed to be red, what is the probability that both
of the discarded balls were blue?
• Solve P (BB | R)
P( R | BB ) P( BB)
=
P( R | BB) P( BB) P( R | BR ) P( BR) P( R | RR ) P( RR )
7 39
*
= 18 95 0.46
7 39 6 91 5 21
* * *
18 95 18 190 18 190
An auto insurance company insures drivers of
all ages. An actuary compiled the following
statistics on the company’s insured drivers:
A randomly selected driver that the company insures
has an accident.
Calculate the probability that the driver was age 16-
20.
Solution
An insurance company issues life insurance policies
in three separate categories: standard, preferred,
and ultra-preferred. Of the company’s policyholders,
50% are standard, 40% are preferred, and 10% are
ultra-referred. Each standard policyholder has
probability 0.010 of dying in the next year, each
preferred policyholder has probability 0.005 of dying
in the next year, and each ultra-preferred
policyholder has probability 0.001 of dying in the
next year. A policyholder dies in the next year.
What is the probability that the deceased policyholder
was ultra-preferred?
Upon arrival at a hospital’s emergency room, patients are
categorized according to their condition as critical,
serious, or stable. In the past year:
(i) 10% of the emergency room patients were critical;
(ii) 30% of the emergency room patients were serious;
(iii) the rest of the emergency room patients were stable;
(iv) 40% of the critical patients died;
(vi) 10% of the serious patients died; and
(vii) 1% of the stable patients died.
Given that a patient survived, what is the probability that
the patient was categorized as serious upon arrival?
A health study tracked a group of persons for five
years. At the beginning of the study, 20% were
classified as heavy smokers, 30% as light
smokers, and 50% as nonsmokers. Results of
the study showed that light smokers were twice
as likely as nonsmokers to die during the five-
year study, but only half as likely as heavy
smokers. A randomly selected participant from
the study died over the five-year period.
Calculate the probability that the participant was a
heavy smoker.
