University of Benghazi

Faculty of Engineering
Electrical and Electronics Engineering Department

Probability and Random Process Course

EE277

Probability
Axioms and Properties of Probability, Conditional Probability,
Independence

Salma Elkawafi
[email protected]
Goals

Understand the following:

• Probability axioms, properties and theories.
• What is probability and how to define it mathematically.
• Conditional Probability and concepts of dependence.
• Multiplication Rule
• Law of Total Probability
• Bayes Theorem.
• Reliability
Probability Theory
• The theory of probability deals with averages of mass phenomena
occurring sequentially or simultaneously: telephone calls, games of chance,
noise, birth and death rates, and queueing theory, etc.
• It has been observed that certain averages approach a constant value as
the number of observations increases and this value remains the same if
the averages are evaluated over any subsequence specified before the
experiment is performed.
• The purpose of the theory is to describe and predict such averages in terms
of probabilities of events. The probability of an event A is a number 𝑷(𝑨)
assigned to this event.
Probability axioms, properties and theories
Experiment (Ex) (or trial) (Random Experiment):
any procedure that:
• can be repeated theoretically an infinite number of times
• has a well-defined set of possible outcomes
• An experiment is said to be random if it has more than
one possible outcome, and deterministic if it has only
one.
Mathematical description of an experiment:
A random experiment is described or modeled by a
mathematical construct known as a probability space.

An outcome is a result of a random experiment.

Probability axioms, properties and theories
Sample Space:
The set of all possible outcomes of an Ex.
Types of sample space:
- Finite (discrete), Infinite
- Countable (discrete)
- Uncountable (continuous)
Examples:
• If the experiment consists of flipping two coins, then the sample space consists of
the following four points: 𝑆 = {(𝐻, 𝐻), (𝐻, 𝑇), (𝑇, 𝐻), (𝑇, 𝑇)}
• If the experiment consists of tossing two dice, then the sample space consists of the
36 points S = {(i, j): i, j = 1, 2, 3, 4, 5, 6}
• If the outcome of an experiment is the order of finish in a race among the 7 horses
having post positions 1, 2, 3, 4, 5, 6, and 7, S = {all 7! permutations of (1, 2, 3, 4, 5, 6, 7)}
Note:
An Ep can have different S depending on what we are interested to observe.
Probability axioms, properties and theories
Event (E):
A subset of the sample space of an Ex.
An event is said to occur if the outcome of the experiment (Ex) is contained in E.
Note:
Any designated collection of possible outcomes of an Ex, including individual outcomes, the
entire sample space, and the null set, constitutes an event.

Example
• In the experiment consists of flipping two coins S = {(H,H), (H, T), (T,H), (T, T)}
The event that a head appears on the first coin is E = {(H,H), (H, T)}

• In the experiment consists of tossing two dice, S = {(i, j): i, j = 1, 2, 3, 4, 5, 6}

The E is the event that the sum of the dice equals 7 is E = {(1, 6), (2, 5), (3, 4), (4, 3), (5, 2),
(6, 1)}
Probability axioms, properties and theories
Example:
A lottery consists of 5 tickets, two of which will be drawn and designated as the
winners. If the two prizes to be awarded are the same, what is the sample space?
Solution
Ep – to select 2 winner tickets out of 5 tickets. Let the tickets be numbered, through 1 to 5
To define the sample space, if the prizes are same, then there are 10 possible pairs of numbers
that can be drawn.
S={(1, 2), (1, 3), (1, 4), (1, 5), (2, 3), (2, 4), (2, 5), (3, 4), (3, 5), (4, 5)}
If the prizes are different, then
S={(1, 2), (1, 3), (1, 4), (1, 5), (2, 3), (2, 4), (2, 5), (3, 4), (3, 5), (4, 5),(2, 1), (3, 1), (4, 1), (5, 1), (3, 2), (4, 2), (5,
2), (4, 3), (5, 3), (5, 4)}
where (x, y): x – ticket number of 1st prize, y – ticket # of 2nd prize
Now, let us define the events E and F, where
E(“#1 is a winning ticket”) = {(1,2), (1,3), (1,4), (1,5), (2,1), (3,1), (4,1), (5,1)}
F(“#1 is a 1st-prize ticket”) = {(1,2), (1,3), (1,4), (1,5)}
Probability axioms, properties and theories
The Classical Definition:
if we assume that all outcomes of an experiment are equally likely to
occur, then the probability of any event E equals the proportion of
outcomes in the sample space that are contained in E.

𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑖𝑛 𝐸
𝑃 𝐸 =
𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 𝑖𝑛 𝑆
Limitation:
What does "equally likely" mean? This appears to use the concept of
probability while trying to define it! We could remove the phrase
"provided all outcomes are equally likely", but then the definition
would clearly be unusable in many settings where the outcomes in did
not tend to occur equally often.
Probability axioms, properties and theories
Relative frequency Definition:

One way of defining the probability of an event is in terms of its relative

frequency.
Such a definition usually goes as follows:
𝑛(𝐸)
𝑃 𝐸 = lim
𝑛→∞ 𝑛

Limitation:
In many cases we can't even obtain a long series of repetitions due to time,
cost, or other limitations. For example, the probability of rain today can't really
be obtained by the relative frequency definition since today can‘t be repeated
again.
Probability axioms, properties and theories
Consider an experiment whose sample space is 𝑺. For each event 𝑬 of the sample
space 𝑺, we assume that a number 𝑃(𝐸) is defined and satisfies the following three
axioms:
Axiom 1
0 ≤ 𝑃(𝐸) ≤ 1
Axiom 2
𝑃(𝑆) = 1
Axiom 3
For any sequence of disjoint 𝐸1, 𝐸2, . . . (events for which 𝐸𝑖𝐸𝑗 = Ø when 𝑖 ≠ 𝑗),
∞ ∞ 𝑛 𝑛

𝑃 ራ 𝐸𝑖 = ෍ 𝑃(𝐸𝑖 ) → 𝑃 ራ 𝐸𝑖 = ෍ 𝑃(𝐸𝑖 )
𝑖=1 𝑖=1 𝑖=1 𝑖=1
Note, the null event has probability 0 of occurring.
Probability axioms, properties and theories
Example:
In a presidential election, there are four candidates. Call them A, B, C, and D. Based on
our polling analysis, we estimate that A has a 20 percent chance of winning the
election, while B has a 40 percent chance of winning. What is the probability that A or B
win the election?
Solution
𝑃 𝐴 wins 𝑜𝑟 𝐵 wins = 𝑃 𝐴 wins ∪ 𝐵 wins
= 𝑃 𝐴 wins + 𝑃 𝐵 wins
= 0.2 + 0.4
= 0.6
Probability axioms, properties and theories
Some Simple Propositions
Proposition 1:
𝑃(𝐸 𝑐 ) = 1 − 𝑃(𝐸)
The probability that an event does not occur is 1 minus the probability that it does occur.

Proposition 2:
𝐼𝑓 𝐸 ⊂ 𝐹, then 𝑃(𝐸) ≤ 𝑃(𝐹).
if the event E is contained in the event F, then the probability of E is no greater than the
probability of F

Proposition 3:
𝑃(𝐸 ∪ 𝐹) = 𝑃(𝐸) + 𝑃(𝐹) − 𝑃(𝐸𝐹)
Proposition 4:
𝑃(𝐴 − 𝐵) = 𝑃(𝐴) − 𝑃(𝐴 ∩ 𝐵)
Probability axioms, properties and theories
Example:
Suppose we have the following information:
• There is a 60 percent chance that it will rain today.
• There is a 50 percent chance that it will rain tomorrow.
• There is a 30 percent chance that it does not rain either day.
Find the following probabilities:
1. The probability that it will rain today or tomorrow. 𝑃 𝐴 ∪ 𝐵
2. The probability that it will rain today and tomorrow. 𝑃 𝐴 ∩ 𝐵
3. The probability that it will rain today but not tomorrow. 𝑃 𝐴 − 𝐵
Solution
𝑃 𝐴 = 0.6, 𝑃 𝐵 = 0.5, 𝑃(𝐴𝑐 ∩ 𝐵𝑐 ) = 0.3
𝑃 𝐴 ∪ 𝐵 = 1 − 𝑃 𝐴 ∪ 𝐵 𝑐 = 1 − 𝑃 𝐴𝑐 ∩ 𝐵𝑐 = 1 − 0.3 = 0.7
𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴 + 𝑃 𝐵 − 𝑃 𝐴 ∪ 𝐵 = 0.6 + 0.5 − 0.7 = 0.4
𝑃 𝐴 − 𝐵 = 𝑃 𝐴 − 𝑃 𝐴 ∩ 𝐵 = 0.6 − 0.4 = 0.2
Probability axioms, properties and theories
Definitions
Mutually Exclusive Events:
Two events, A and B are said to be mutually exclusive if they cannot occur together.
Ex. In tossing a die, both head and tail cannot happen at the same time.
𝑃 𝐴 ∪ 𝐵 = 𝑃 𝐴 + 𝑃(𝐵)

Equally Likely Events:

If one of the events cannot be expected to happen in preference to another, then such events are
said to be Equally Likely Events.( Or) Each outcome of the random experiment has an equal chance
of occurring.
𝑃 𝐴 = 𝑃(𝐵)
Ex. In tossing a coin, the coming of the head or the tail is equally likely.
Probability axioms, properties and theories
Definitions

Independent Events:
Two events are said to be independent, if happening or failure of one does not affect the
happening or failure of the other. Otherwise, the events are said to be dependent.
If two events, A and B are independent then the joint probability is
𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴 𝑃(𝐵)

If 𝐴 and 𝐵 are independent, then :

1) 𝐴𝑐 and 𝐵
2) 𝐴 and 𝐵 𝑐
3) 𝐴𝑐 and 𝐵 𝑐 are also independent
If 𝐴 and 𝐵 are disjoint, then : 𝐴 ∩ 𝐵 = ∅, 𝑃 𝐴 ∪ 𝐵 = 𝑃 𝐴 + 𝑃(𝐵), 𝑃 𝐴𝐵 = 0
If 𝐴 and 𝐵 are disjoint, then they are dependent and if 𝐴 and 𝐵 are independent, they are not
disjoint
Probability axioms, properties and theories
Example
To cut a painted cube into 1000 small cubes of the same size and randomly select a cube. Find
P(the cube has only three painted faces)
P(the cube has only two painted faces)
P(the cube has only one painted face)
P(the cube has No painted faces)
Solution:
We have S={all the small cubes} → |S|=1000, selecting one cube from these 1000 has the same
possibility, so we use counting for each event.
Let:
E3 := the selected cube has three faces. |E3|=8 (corners)
E2 := the selected cube has two faces. |E2|= 12 (edges) x8 (cubes/edge)
E1 := the selected cube has one face. |E1| = 6 (faces) x 8x8 (cubes/face)
E0 := the selected cube has no faces. |E0| = |S|-|E1|-|E2|-|E3|
8 96 384 512
𝑃 𝐸3 = , 𝑃 𝐸2 = , 𝑃 𝐸1 = , 𝑃 𝐸0 =
1000 1000 1000 1000
Conditional Probability
We are often interested in calculating probabilities when some partial information
concerning the result of an experiment is available; in such a situation, the desired
probabilities are conditional.

If we toss 2 dice and suppose that each of the 36 possible outcomes is equally likely.
Suppose that we observe the first die is a 3. Then, given this information, what is the
probability that the sum of the 2 dice equals 8?
Given that the initial die is a 3, the possible outcomes of our experiment are (3, 1), (3, 2),
(3, 3), (3, 4), (3, 5), and (3, 6). That is, given that the first die is a 3, the (conditional)
probability of each of the outcomes (3, 1), (3, 2), (3, 3), (3, 4), (3, 5), and (3, 6) is 1/6

If E (the event that the sum of the dice is 8 ) and F (event that the first die is a 3), then
the probability just obtained is called the conditional probability that E occurs given
that F has occurred and is denoted by:
𝑃(𝐸|𝐹).
Conditional Probability
A general formula for 𝑃(𝐸|𝐹) that is valid for all events E and F is derived in the same
manner: If the event F occurs, then, in order for E to occur, it is necessary that the actual
occurrence be a point both in E and in F; that is, it must be in EF. Now, since we know
that F has occurred, it follows that F becomes our new, or reduced, sample space;
hence, the probability that the event EF occurs will equal the probability of EF relative to
the probability of F.

That is, we have the following definition.

𝐼𝑓 𝑃(𝐹) > 0, then

𝑃 𝐸𝐹 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑒𝑙𝑒𝑚𝑒𝑛𝑡𝑠 𝑜𝑓 𝐸 ∩ 𝐹
𝑃 𝐸𝐹 = =
𝑃 𝐹 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑒𝑙𝑒𝑚𝑒𝑛𝑡𝑠 𝑜𝑓 𝐹
Conditional Probability
Properties of Conditional probability:
Let 𝐹 ⊂ 𝑆 such that 𝑃 𝐹 > 0, then
• 0 ≤ 𝑃(𝐸|𝐹) ≤ 1 for all 𝐸 ⊂ 𝑆

• if 𝐸𝑖 ∩ 𝐸𝑗 = ∅ for 𝑖 ≠ 𝑗 then 𝑃 ‫=𝑖ڂ‬1 𝐸𝑖 |𝐹

𝑛
= σ𝑛𝑖=1 𝑃 𝐸𝑖 𝐹
𝑐
• 𝑃 𝐸 𝐹 = 1−𝑃 𝐸 𝐹

• 𝑃 𝐸1 𝑈 𝐸2 𝐹) = 𝑃 𝐸1 𝐹) + 𝑃 ( 𝐸2 | 𝐹) − 𝑃(𝐸1 ∩ 𝐸2 𝐹
• 𝑃 𝐸1 𝐹 ≤ 𝑃 𝐸2 𝐹 𝑓𝑜𝑟 #𝐸1 ≤ #𝐸2

• If 𝐸 ∩ 𝐹 = ∅, 𝑃 𝐸 𝐹 = 0

• If two events, 𝐸 and 𝐹 are independent (𝑃 𝐹 ∩ 𝐸 = 𝑃 𝐸 𝑃(𝐹))

then
𝑃 𝐸 𝐹 = 𝑃 𝐸 , 𝑃 𝐹 𝐸 = 𝑃(𝐹)
Conditional Probability
Example:
A coin is flipped twice. Assuming that all four points in the sample space S = {(h, h), (h, t),
(t, h), (t, t)} are equally likely, what is the conditional probability that both flips land on
heads, given that (a) the first flip lands on heads? (b) at least one flip lands on heads?
Solution
Let:
B : the event that both flips land on heads;
F : the event that the first flip lands on heads;
A : the event that at least one flip lands on heads. The probability for

𝑃(𝐵∩𝐹) 𝑃({(ℎ,ℎ)}) 1/4 1

a) 𝑃 𝐵𝐹 = = 𝑃({(ℎ,ℎ),(ℎ,𝑡)}) = 2/4 = 2
𝑃(𝐹)
𝑃 𝐵∩𝐴 𝑃({(ℎ,ℎ)}) 1/4 1
b) 𝑃 𝐵𝐴 = = 𝑃({(ℎ,ℎ),(ℎ,𝑡),(𝑡,ℎ)}) = 3/4 = 3
𝑃 𝐴
Conditional Probability
Example:
Assume 9 cards numbered with {1,2,3,---,9}. One card has been drawn randomly.
Find
a) The probability that this card is numbered 9
b) The probability that it is greater than 8 given it is odd.
c) The probability that it is odd given that it is greater than 8.

Solution
a) Let 𝑋 = {9} → 𝑃(𝑋) = 1/9
b) Let 𝑌 = 1,3,5,7,9 odd Event
𝑃 𝑋𝑌 1/9 1
𝑃 𝑋Y = = =
𝑃 𝑌 5/9 5
𝑃 𝑋𝑌
a) 𝑃 𝑌X = = 1
𝑃 𝑋
Conditional Probability
Example:
Celine is undecided as to whether to take a French course or a chemistry course. She estimates
1 2
that her probability of receiving an A grade would be in a French course and in a chemistry
2 3
course. If Celine decides to base her decision on the flip of a fair coin, what is the probability that
she gets an A in chemistry?
Solution:
Let:
A : Celine receives an A in whatever course she takes,
F : Celine takes a French course
C : Celine takes chemistry
1
Then we have the following probabilities: 𝑃 𝐶 = 𝑃 𝐹 = “because she choose based on
2
flipping a fair coin”, Then the desired probability is 𝑃(𝐶𝐴), which is calculated by using:
𝑃 𝐶𝐴 = 𝑃 𝐶 𝑃 𝐴 𝐶
1 2 1
= ∗ =
2 3 3
Conditional Probability
Multiplication Rule, Law of Total Probability:
Let 𝐸 , 𝐹 and 𝐺 be any three events defined on 𝑆 such that 𝑃 𝐸 , 𝑃 𝐹 , 𝑃 𝐺 > 0 and from the
definition of the conditional probability we have:
𝑃 𝐸𝐹 = 𝑃 𝐸 ∩ 𝐹 = 𝑃 𝐸|𝐹 𝑃 𝐹 = 𝑃 𝐹|𝐸 𝑃(𝐸)
𝑃 𝐸𝐹𝐺 = 𝑃 𝐸(𝐹𝐺) = 𝑃 𝐸|𝐹𝐺 𝑃 𝐹𝐺 = 𝑃 𝐸|𝐹𝐺 𝑃 𝐹 𝐺 𝑃(𝐺)
The Multiplication rule
𝑃(𝐸1 𝐸2 𝐸3 · · · 𝐸𝑛 ) = 𝑃(𝐸1 )𝑃(𝐸2 |𝐸1)𝑃(𝐸3 |𝐸1 𝐸2 ) · · · 𝑃(𝐸𝑛 |𝐸1 · · · 𝐸𝑛−1 )

Law of total probability: If 𝐵1 , 𝐵2 , 𝑎𝑛𝑑 𝐵3 partition S then:

𝑃(𝐴) = 𝑃(𝐴 ∩ 𝐵1 ) + 𝑃(𝐴 ∩ 𝐵2 ) + 𝑃(𝐴 ∩ 𝐵3 )
= 𝑃(𝐴|𝐵1 )𝑃(𝐵1 ) + 𝑃(𝐴|𝐵2 )𝑃(𝐵2 ) + 𝑃(𝐴|𝐵3 )𝑃(𝐵3 )

Total Probability Theorem

𝑃 𝐴 = 𝑃 𝐴 ∩ 𝐵1 + 𝑃 𝐴 ∩ 𝐵2 +. . + 𝑃(𝐴 ∩ 𝐵𝑛 )
= 𝑃 𝐴 𝐵1 𝑃 𝐵1 + 𝑃 𝐴 𝐵2 𝑃 𝐵2 +. . + 𝑃(𝐴|𝐵𝑛 )𝑃(𝐵𝑛 )
Conditional Probability
Example
An urn contains five white chips, four black chips, and three red chips. Four chips are drawn sequentially and
without replacement. What is the probability of obtaining the sequence (white, red, white, black)?
Solution:
Let:
A: white chip is drawn on first selection , B: red chip is drawn on second selection
C: white chip is drawn on third selection D: black chip is drawn on fourth selection
Our objective is to find 𝑃(𝐴 ∩ 𝐵 ∩ 𝐶 ∩ 𝐷)
𝑃 𝐴 ∩ 𝐵 ∩ 𝐶 ∩ 𝐷 = 𝑃 𝐷 𝐴 ∩ 𝐵 ∩ 𝐶 · 𝑃(𝐴 ∩ 𝐵 ∩ 𝐶)
𝑃(𝐴 ∩ 𝐵 ∩ 𝐶 ∩ 𝐷) = 𝑃(𝐷|𝐴 ∩ 𝐵 ∩ 𝐶) · 𝑃(𝐶|𝐴 ∩ 𝐵) · 𝑃(𝐵 ∩ 𝐴)
𝑃(𝐴 ∩ 𝐵 ∩ 𝐶 ∩ 𝐷) = 𝑃(𝐷|𝐴 ∩ 𝐵 ∩ 𝐶) · 𝑃(𝐶|𝐴 ∩ 𝐵) · 𝑃(𝐵|𝐴) · 𝑃(𝐴)
5
𝑃 𝐴 =
12
3
𝑃 𝐵𝐴 =
11
4
𝑃 𝐶𝐴 ∩ 𝐵 =
10
4
𝑃 𝐷𝐴 ∩ 𝐵 ∩ 𝐶 =
9
5 3 4 4
𝑃 𝐴 ∩ 𝐵 ∩ 𝐶 ∩ 𝐷 = ∗ ∗ ∗ = 0.02
12 11 10 9
Conditional Probability
Example(sequential representation )
In a certain assembly plant, three machines, B1, B2, and B3, make 30%, 45%, and 25%, respectively,
of the products. It is known from past experience that 2%, 3%, and 2% of the products made by
each machine, respectively, are defective. Now, suppose that a finished product is randomly
selected. What is the probability that it is defective?
Solution:
Consider the following events:
A: the product is defective,
B1: the product is made by machine B1,
B2: the product is made by machine B2,
B3: the product is made by machine B3.

𝑃(𝐴) = 𝑃(𝐵1 )𝑃(𝐴|𝐵1 ) + 𝑃(𝐵2 )𝑃(𝐴|𝐵2 ) + 𝑃(𝐵3 )𝑃(𝐴|𝐵3 ).

𝑃(𝐵1 )𝑃(𝐴|𝐵1) = (0.3)(0.02) = 0.006,

𝑃(𝐵2 )𝑃(𝐴|𝐵2) = (0.45)(0.03) = 0.0135,
𝑃(𝐵3 )𝑃(𝐴|𝐵3) = (0.25)(0.02) = 0.005,
𝑃(𝐴) = 0.006 + 0.0135 + 0.005 = 0.0245.
if a product was chosen randomly and found to be
defective, what is the probability that it was made by
machine B3?

Use Bayes Theorem

Bayes Theorem (theorem on the probability of causes)
Bayes' Rule
Let 𝐴1 , 𝐴2 , . . . , 𝐴𝑛 be disjoint events that form a partition of the sample space, and
assume that 𝑃 (𝐴𝑖 ) > 0, for all 𝑖 . Then, for any event B such that 𝑃(𝐵) > 0, we have
𝑃(𝐴𝑖 )𝑃(𝐵 𝐴𝑖
𝑃(𝐴𝑖 | 𝐵) =
𝑃(𝐵)
𝑃(𝐴𝑖 )𝑃(𝐵 𝐴𝑖
𝑃(𝐴𝑖 | 𝐵) =
𝑃 𝐴1 𝑃 𝐵 𝐴1 + ⋯ + 𝑃 𝐴𝑛 𝑃 𝐵 𝐴𝑛

If we have some random experiment in which the events of interest form a partition. The
“a priori probabilities” of these events 𝑃(𝐴𝑖 ), are the probabilities of the events before
the experiment is performed. Now suppose that the experiment is performed, and we
are informed that event B occurred; the “a posteriori probabilities” are the probabilities
of the events in the partition 𝑃(𝐴𝑖 | 𝐵).
Bayes Theorem
Example:
An insurance company believes that people can be divided into two classes: those who are accident prone and
those who are not. The company’s statistics show that an accident-prone person will have an accident at some
time within a fixed 1-year period with probability 0.4, whereas this probability decreases to 0.2 for a person who
is not accident prone. If we assume that 30 percent of the population is accident prone, what is the probability
that a new policyholder will have an accident within a year of purchasing a policy?
Suppose that a new policyholder has an accident within a year of purchasing a policy. What is the probability
that he or she is accident prone? Have an
Solution: accident
Let:
Accident prone
𝐴 : the event that the policyholder is accident prone.
𝐴1 :the event that the policyholder will have an accident within a year of purchasing
the policy.

𝑃 𝐴1 = 𝑃 𝐴𝑐 ∩ 𝐴1 + 𝑃 𝐴 ∩ 𝐴1 Have an
= 𝑃 𝐴1 𝐴 𝑃 𝐴 + 𝑃 𝐴1 𝐴𝑐 𝑃 𝐴𝑐 = 0.4 0.3 + 0.2 0.7 = 0.26 accident
Not accident
𝑃(𝐴𝐴1 ) 𝑃 𝐴 𝑃(𝐴1 |𝐴) 0.3 ∗ 0.4 6 prone
𝑃 𝐴 𝐴1 = = = =
𝑃(𝐴1 ) 𝑃(𝐴1 ) 0.26 13
Bayes Theorem
Example:
A weather Satellite is sending a binary code of 0’s and 1’s describing tropical storm. Channel noise, thought, can
be expected to introduce a certain amount of transmission error. Suppose 65% of the message being relayed 0’s
and there is an 85% of a given 0 or 1 being received properly. If a 1 is received, what is the probability that a 0
was sent?
Solution:
Let:
𝐹1 = 0 is sent
𝐹2 = 1 is sent
𝐸1 = 0 is received
𝐸2 = 1 is received

𝑃(𝐹1 𝐸1 ) 𝑃 𝐸2 𝐹1 𝑃(𝐹1 ) 0.15 ∗ 0.65

𝑃 𝐹1 𝐸2 = = = = 0.247
𝑃 𝐸2 𝑃 𝐸2 |𝐹1 𝑃 𝐹1 + 𝑃 𝐸2 𝐹2 𝑃(𝐹2 ) 0.15 ∗ 0.6 + 0.85 ∗ 0.35
Reliability
In probabilistic models of the complex system involving several components, it is often convenient
to assume that the behaviors of the components are uncoupled (independent). This typically
simplifies the calculations and the analysis.

Series Connection Parallel Connection

For example, if three relays 𝑅1 , 𝑅2 𝑎𝑛𝑑 𝑅3 are connected as shown above and the probability that
they will work properly are 𝑃1 , 𝑃2 𝑎𝑛𝑑 𝑃3 respectively. If all the relay are closed at a given time, what
is the probability that 𝐴 and 𝐵 are connected ?

If the event 𝐸𝑖 denote 𝑅𝑖 works properly, 𝑖 = 1,2,3. Then 𝑃 𝐸𝑖 = 𝑝𝑖 , 𝑖 = 1,2,3

𝑃 𝐴& 𝐵 𝑐𝑜𝑛𝑛𝑒𝑐𝑡𝑒𝑑 = 𝑃 𝐸1 𝐸2 𝐸3 = 𝑃 𝐸1 𝑃 𝐸2 𝑃 𝐸3 = 𝑝1 𝑝2 𝑝3 → 𝑠𝑒𝑟𝑖𝑒𝑠 𝑐𝑜𝑛𝑛𝑒𝑐𝑡𝑖𝑜𝑛

𝑃 𝐴& 𝐵 𝑐𝑜𝑛𝑛𝑒𝑐𝑡𝑒𝑑 = 𝑃 𝐸1 ∪ 𝐸2 ∪ 𝐸3 = 1 − 𝑃 𝐸1 ∪ 𝐸2 ∪ 𝐸3 𝑐 = 1 − 𝑃 𝐸1𝑐 )𝑃 𝐸2𝑐 𝑃(𝐸3𝑐

= 1 − (1 − 𝑝1 )(1 − 𝑝2 )(1 − 𝑝3 ) → 𝑝𝑎𝑟𝑎𝑙𝑙𝑒𝑙 𝑐𝑜𝑛𝑛𝑒𝑐𝑡𝑖𝑜𝑛
Reliability
Example
The highways connecting two resort areas at A and B are shown in the following figure.
The roads sometimes being closed due to snow and ice. Suppose we let 𝐸1 , 𝐸2 and 𝐸3
denote the events that highways AB, AC, and CB are passable, respectively, and we know
from past years that on typical winter day.
2 3 2 4 1
𝑃 𝐸1 = 5 , 𝑃 𝐸2 = 4 , 𝑃 𝐸3 = 3 , 𝑃 𝐸3 𝐸2 = 5 , 𝑃 𝐸1 𝐸2 𝐸3 = 2
What is the probability that a traveler will be able to get from A to B?
Solution
E: the event that we can get from A to B
𝐸 = 𝐸1 ∪ 𝐸2 𝐸3
𝑃 𝐸 = 𝑃(𝐸1 ) + 𝑃 𝐸2 𝐸3 − 𝑃 𝐸1 𝐸2 𝐸3
= 𝑃 𝐸1 + 𝑃 𝐸3 𝐸2 𝑃 𝐸2 − 𝑃 𝐸1 𝐸2 𝐸3 𝑃 𝐸2 𝐸3
= 𝑃 𝐸1 + 𝑃 𝐸3 𝐸2 𝑃 𝐸2 − 𝑃 𝐸1 𝐸2 𝐸3 𝑃 𝐸3 |𝐸2 𝑃 𝐸2
2 4 3 1 4 3
= 5 + 5 ∗ 4 − 2 ∗ 5 ∗ 4 = 0.7
Probability axioms, properties and theories
Exercise
A small town has one fire engine and one ambulance available for emergencies. The
probability that the fire engine is available when needed is 0.98, and the probability
that the ambulance is available when called is 0.92. In the event of an injury resulting
from a burning building, find the probability that both the ambulance and the fire-
engine will be available.
Solution:

Let A and B represent the respective events that the fire engine and the ambulance
are available. Then
𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴)𝑃(𝐵) = (0.98)(0.92) = 0.9016.
Probability axioms, properties and theories
Exercise
J is taking two books along on her holiday vacation. With probability 0.5, she will like
the first book; with probability 0.4, she will like the second book; and with probability
0.3, she will like both books. What is the probability that she likes neither book?
Solution:

Let 𝐵𝑖 denote the event that J likes book 𝑖, 𝑖 = 1, 2. Then the probability that she likes at
least one of the books is:
𝑃(𝐵1 ∪ 𝐵2 ) = 𝑃(𝐵1 ) + 𝑃(𝐵2 ) − 𝑃(𝐵1 𝐵2 ) = 0.5 + 0.4 − 0.3 = 0.6

Because the event that J likes neither book is the complement of the event that she likes
at least one of them, we obtain the result
𝑃 𝐵1𝑐 𝐵2𝑐 = 𝑃 𝐵1 ∪ 𝐵2 𝑐 = 1 − 𝑃 𝐵1 ∪ 𝐵2 = 1 − 0.6 = 0.4
Probability axioms, properties and theories
Exercise
A committee of 5 is to be selected from a group of 6 men and 9 women. If the selection
is made randomly, what is the probability that the committee consists of 3 men and 2
women?
Solution:
Because each of the 15 5
possible committees is equally likely to be selected, the
desired probability is
6 9
3 2 240
=
15 1001
5
Probability axioms, properties and theories
Exercise
An urn contains n balls, one of which is special. If k of these balls are withdrawn one
at a time, with each selection being equally likely to be any of the balls that remain at
the time, what is the probability that the special ball is chosen?
Solution:

1 𝑛−1
1 𝑘−1 𝑘
P{special ball is selected}= 𝑛 =
𝑘 𝑛
Probability axioms, properties and theories
Exercise
If 3 balls are “randomly drawn” from a bowl containing 6 white and 5 black balls, what is the
probability that one of the balls is white and the other two black?
Solution:
This problem could also have been solved by regarding the outcome of the experiment
as the unordered set of drawn balls. Hence, the desired probability is :
6 5
1 2 4
=
11 11
3
This problem could also have been solved by regarding the order in which the balls are selected as
being relevant, then the sample space consists of 11 · 10 · 9 = 990 outcomes.
There are :
• 6 · 5 · 4 = 120 outcomes in which the first ball selected is white and the other two are black;
• 5 · 6 · 4 = 120 outcomes in which the first is black, the second is white, and the third is black;
• 5 · 4 · 6 = 120 outcomes in which the first two are black and the third is white.
Assuming that each outcome in the sample space is equally likely to occur, the desired probability is
120 + 120 + 120 4
=
990 11
Probability axioms, properties and theories
Exercise
A poker hand consists of 5 cards. If the cards have distinct consecutive values and
are not all of the same suit, we say that the hand is a straight. For instance, a hand
consisting of the five of spades, six of spades, seven of spades, eight of spades, and
nine of hearts is a straight. What is the probability that one is dealt a straight?
Solution:
10(45 −4)
The desired probability is = 0.0039
𝐶552
Conditional Probability
Exercise
A family has two children. It is known that at least one of the children is a girl. What is
The probability that both the children are girls?
Solution
A = event of at least one girl
B = event of two girl

𝑃 𝐵∩𝐴 1/4 1
𝑃 𝐵𝐴 = = =
𝑃 𝐴 3/4 3
Conditional Probability
Exercise
In the card game, the 52 cards are dealt out equally to 4 players- called Player A, B, C
and Player D. If Player A and Player B have a total of 8 spades among them, what is the
probability that Player C has 3 of the remaining 5 spades?
Solution
By knowing that Player A and Player B have a total of 8 spades among their 26 cards,
we can calculate the required probability for Player C as to count the possible ways of
getting 3 out of 5 over his 13 card from the remaining 26.
So, if we define the following events
C:= Player C has 3 spade,
F:= Player A and Player B have 8 spades in their 26 cards,
The required probability is
5 21
3 10
𝑃 𝐶𝐹 = = 0.339
26
13
Conditional Probability
Exercise
Celine is undecided as to whether to take a French course or a chemistry course. She
1 2
estimates that her probability of receiving an A grade would be in a French course and in a
2 3
chemistry course. If Celine decides to base her decision on the flip of a fair coin, what is the
probability that She gets an A?
Solution:
Let:
C : Celine takes chemistry
A : Celine receives an A in whatever course she takes,
F : Celine takes a French course
1
Then we have the following probabilities: 𝑃 𝐶 = 𝑃 𝐹 = “because she choose based on
2
flipping a fair coin”,
2 1 1 1 7
𝑃 𝐴 = 𝑃 𝐴|𝐶 𝑃 𝐶 + 𝑃 𝐴|𝐹 𝑃 𝐹 = ∗ + ∗ =
3 2 2 2 12
Conditional Probability
Exercise : (sequential representation )
If an aircraft is present in a certain area, a radar detects it and generates an alarm signal with probability 0.99.
If an aircraft is not present. the radar generates a (false) alarm, with probability 0. 10. We assume that an aircraft
is present with probability 0.05. What is the probability of no aircraft presence and a false alarm? What is the
probability of aircraft presence and no detection?
Solution:
Let A and B be the events
𝐴 = {an aircraft is present }, aircraft
present
𝐵 = {the radar generates an alarm} ,
𝐴𝑐 = {an aircraft is not present },
𝐵𝑐 = {the radar does not generate an alarm}

𝑃 ( 𝑛𝑜𝑡 𝑝𝑟𝑒𝑠𝑒𝑛𝑡 , 𝑔𝑒𝑛𝑒𝑟𝑎𝑡𝑒 𝑎𝑙𝑎𝑟𝑚) = 𝑃(𝐴𝑐 ∩ 𝐵) = 𝑃 ( 𝐴𝑐 ) 𝑃 𝐵 𝐴𝑐 )

= 0.95 ∗ 0. 10 = 0.095 aircraft
𝑃 ( 𝑝𝑟𝑒𝑠𝑒𝑛𝑡 , 𝑛𝑜 𝑎𝑙𝑎𝑟𝑚) = 𝑃(𝐴 ∩ 𝐵𝑐 ) = 𝑃 ( 𝐴) 𝑃 𝐵𝑐 𝐴) not present
= 0.05 ∗ 0. 010 = 0.0005

A sequential representation of the experiment

Bayes Theorem
Exercise
A blood test lab is %95 effective in detecting a certain disease when it is, in fact, present. However, the test also
gives a “false positive” result for %1 of the healthy persons tested. If %0.5 of the population actually has the
disease, what is the probability that a person has the disease given that the test result is positive?

Solution:
Let:
𝑃: The test result is positive
𝑆:The tested person is sick
𝐻: The tested person is healthy,
So, we have the following information:
𝑃(𝑆) = 0.005 and 𝑃(𝐻) = 0.995. Also we have 𝑃(𝑃|𝑆) = 0.95, 𝑎𝑛𝑑 𝑃(𝑃|𝐻) = 0.01

The required probability is

𝑃 𝑃 𝑆 𝑃(𝑆) 0.95 ∗ 0.005
𝑃 𝑆𝑃 = = = 0.323
𝑃 𝑃 𝑆 𝑃 𝑆 + 𝑃 𝑃 𝐻 𝑃(𝐻) 0.95 ∗ 0.005 + 0.01 ∗ 0.995