19R401 – Probability and Statistics

Prerequisites:

 Basics of algebra
 Basics of set theory
 Basics of calculus
 Course objective (Course outcome 1)

After the completion of CO1, you should be able to identify

random experiment, sample space and apply Baye’s rule to
formulate and solve probabilistic problems and analyse
discrete probability distributions like Binomial, Poisson and
Geometric distributions.
 Basic terms

Experiment:
An operation which can produce some well-defined outcomes, is
called an experiment.
Random Experiment:
An experiment that can result in different outcomes, even though it
is repeated in the same manner every time, is called a random
experiment.
i.e., An experiment in which all possible outcomes are known and
the exact outcome cannot be predicted in advance, is called
a random experiment.
 Basic terms…

Examples for random experiment:

• toss a coin
• roll a die

• observe the number of iPhones sold by an Apple store in Boston

in 2019
• The selection of a numbered ball (1-50) in an urn
• The time difference between two messages arriving at a message
centre
• The time difference between two different voice calls over a
particular network
 Basic terms…

Outcome: An outcome is a result of a random experiment.

Sample space : The set of all possible outcomes of a random experiment

is called the sample space of the experiment. The sample space is
denoted as S..
Trial: When we repeat a random experiment several times, we call each
one of them a trial. Thus, a trial is a particular performance of a random
experiment.

Event: A subset of the sample space.

Exhaustive events: If E1 , E2 , ..., En are n events of a sample space S and if

E1 ∪ E2 ∪ E3 ∪ ... ∪ En = S then E1 , E2 , ..., En are called exhaustive
events.
 Basic terms…

Mutually exclusive: Two events A and B of a sample space S

are mutually exclusive if the occurrence of any one of them
excludes the occurrence of the other event.
Hence, the two events A and B cannot occur simultaneously,
and thus P(A∩B) = 0.
Dependent event: Two events are dependent when the
outcome of the first event influences the outcome of the
second event. 
 Basic terms…
Impossible and Sure Events: The empty set φ and the sample space S
describe events. In fact φ is called an impossible event and S, i.e., the
whole sample space is called a sure event.

Simple or Elementary Event: If an event E has only one sample point

of a sample space, i.e., a single outcome of an experiment, it is called
a simple or elementary event.
The sample space of the experiment of tossing two coins is given by S
= {HH, HT, TH, TT} The event E1 = {HH} containing a single outcome
HH of the sample space S is a simple or elementary event.
 Basic terms…

Compound Event: If an event has more than one

sample point it is called a compound event, for
example, S = {HH, HT} is a compound event.
Complementary event: Given an event A, the
complement of A is the event consisting of all sample
space outcomes that do not correspond to the
occurrence of A. The complement of A is denoted by
A′ or AC .
 How to define a sample space in practical?
A sample space is often defined based on the
objectives of the analysis.
EX.1 : Camera Flash
Consider an experiment that selects a cell phone camera
and records the recycle time of a flash (the time taken to
ready the camera for another flash). The possible values for
this time depend on the resolution of the timer and on the
minimum and maximum recycle times.
Solution: (Different perspective to define the sample space)
 How to define a sample space in practical?...
1. S = R+ = {x | x > 0}. Because of the time is positive, it is
convenient to define the sample space as simply the
positive real line.
2. S = {x | 1.5 < x < 5} , if it is known that all recycle times are
between 1.5 and 5 seconds.
3. S = {low, medium, high} , if the objective of the analysis is
to consider only whether the recycle time is low, medium,
or high.
4. S = {yes, no}, if the objective is only to evaluate whether or
not a particular camera conforms to a minimum recycle-
time specification.
 Basic terms…

Discrete Sample Space

• A sample space is discrete if it consists of a finite or
countable infinite set of outcomes.
Continuous Sample Space
• A sample space is continuous if it contains an
interval (either finite or infinite) of real
• numbers.
 Basic terms…

Summarizing set operations in terms of events:

• The union of two events is the event that consists of all
outcomes that are contained in either of the two events. We
denote the union as E1 ∪ E2. EC

• The intersection of two events is the event that consists of

all outcomes that are contained in both of the two events. We
denote the intersection as E1 ∩ E2.
 Basic terms…

• The complement of an event in a sample space is the set of

outcomes in the sample space that are not in the event. We
denote the complement of the event E as E′.
c
The notation E is also used in other literature to denote the
complement.

Mutually Exclusive Events :

Two events, denoted as E1 and E2, such that E1 ∩ E2 = Ø are
said to be mutually exclusive.
 Basic terms…

Equally Likely Outcomes : Whenever a sample space consists

of N possible outcomes that are equally likely, the probability
of each outcome is 1/N.

Probability of an Event: For a discrete sample space, the

probability of an event E, denoted as P(E), equals the sum of
the probabilities of the outcomes in E.
 Description of a sample space

A sample space S can be

i) finite, ii) countably infinite, iii) uncountably infinite.
A sample space S is either discrete or continuous. A
discrete sample space S is either finite or countably
infinite.
A continuous sample space S is uncountably infinite.
Sample spaces can also be described graphically with tree
diagrams.
 Description of a sample space…

Ex.1 : Message Delays

Each message in a digital communication system is classified as to

whether it is received within the time specified by the system design.
If three messages are classified, use a tree diagram to represent the
sample space of possible outcomes.
Tree diagram:
Message1 OT D

Message2 OT D OT D

Message3 OT D OT D OT D OT D

The sample space is

S={ OOO,OOD,….,DDD}
 Examples for sample spaces

Describe the sample space for each of the following random

experiments
1. Each of four transmitted bits is classified as either in error
or not in error.
A: Let "e" denote a bit in error and let "o" denote a bit not in
error ("o" denotes okay). Then
S= {eeoo, eooo, oeoo, oooo, eeoe, eooe, oeoe, oooe, eeeo,
eoeo, oeeo, ooeo, eeee, eoee, oeee, ooee}
 Examples for sample spaces…

2. An ammeter that displays three digits is used to measure

current in milliamperes.
A vector with three components can describe the three digits
of the ammeter.
Each digit can be 0,1,2,...,9.
Then S is a sample space of 1000 possible three digit integers,
S = {000,001,...,999}
 Examples for sample spaces…

3. Each of three machined parts is classified as either

above or below the target specification for the part.
A: Let "a", "b" denote a part above, below the specification.
Then,
S = {aaa,aab,aba,abb,baa,bab,bba,bbb}
4. Calls are repeatedly placed to a busy phone line until a
connect is achieved.
A: c = connect, b = busy, S = {c, bc, bbc, bbbc, bbbbc, …}
 Examples for set operations

Ex.1 A sample of three calculators is selected from a

manufacturing line, and each calculator is classified as either
defective or acceptable.
Let A, B, and C denote the events that the first, second, and
third calculators respectively, are defective.

(a) Describe the sample space for this experiment with a tree
diagram.

Use the tree diagram to describe each of the following events:

 
(b) A (c) B (d)A B (e)B C
 Examples for set operations…

A: a) S = {ddd,add,dda,ada,dad, aad,daa,aaa}
b) A = {ddd,dda,dad,daa}
c) B = {ddd,dda,add,ada}
d) A∩ B = {ddd,dda}

e) B∪C = {ddd,dda,add,ada,dad,aad}
 Examples for set operations…

Ex.2. Suppose that the recycle times of two cameras are

recorded whether or not a particular camera conforms to a
minimum recycle-time specification.
Consider the sample space S = {yy, yn, ny, nn} .
Let E1 be at least one camera conforms.
Let E2 be both cameras do not conform.
Then , E1 = {yy, yn, ny} , E2 = {nn}.
Let E3 = Ø, E4 = S, E5 = {yn, ny, nn}.
Then E1 ∪ E5 =S
E1 ∩ E5 = {yn, ny}
E1′= {nn}
 Basic terms…
Permutations: To find the number of ordered sequences of
the elements of a set.
A permutation of the elements is an ordered sequence of the
elements.
For example: consider a set of elements, such as S = {a, b, c}.
Then
abc, acb, bac, bca, cab, and cba are the permutations of the
elements of S.
The number of permutations of n different elements is n!
where n! = n × (n − 1) × (n − 2) × · · · × 2 × 1.
 Basic terms…

Permutations of Subsets

The number of permutations of subsets of r elements selected from a

set of n different elements is

nPr = n × (n − 1) × (n − 2) × · · · × (n − r + 1)

= n! / (n − r)!

Permutations of Similar Objects

The number of permutations of n = n1 + n2 + · · · + nr objects of which
n1 are of one type, n2 are of a second type, …, and nr are of an r th type
is

n! / n1!n2!n3!…nr!
 Basic terms…

Combinations:
Another counting problem of interest is the number of subsets
of r elements that can be selected from a set of n elements.
Here, order is not important. These are called combinations.

Combinations:
The number of combinations, subsets of r elements that can
be selected from a set of n elements, is denoted as nCr and
nCr = n! / r!(n − r)!
 Example for combinations

Ex.1 A printed circuit board has eight different locations in

which a component can be placed. If five identical
components are to be placed on the board, how many
different designs are possible?
A: Each design is a subset of size five from the eight locations
that are to contain the components.
The number of possible designs is
8! / 5!3! = 56
 Probability
• We introduce probability for discrete sample spaces—those
with only a finite set of outcomes.
• Probability is used to quantify the likelihood, or chance, that
an outcome of a random experiment will occur.
Ex.: The chance of rain today is 30% is a statement that
quantifies our feeling about the possibility of rain.
The likelihood of an outcome is quantified by assigning a
number from the interval [0, 1] to the outcome (or a
percentage from 0 to 100%).
Higher numbers indicate that the outcome is more likely than
lower numbers.
A 0 indicates an outcome will not occur.
A probability of 1 indicates that an outcome will occur with
certainty.
 Interpretations
• Four perspectives on probability are commonly used:

• Classical (sometimes called "A priori" or "Theoretical"):

The probability of an event is the ratio of the number of cases

favorable to it, to the number of all cases possible

• Empirical (sometimes called "A posteriori" or "Frequentist"):

Empirical probability uses the number of occurrences of an outcome

within a sample set as a basis for determining the probability of that

outcome. The number of times "event X" happens out of 100 trials

will be the probability of event X happening.

 Interpretations…
• Subjective:

Subjective probability is a type of probability derived from an

individual's personal judgement or own experience about
whether a specific outcome is likely to occur. It contains no formal
calculations and only reflects the subject's opinions and past
experience.
• Axiomatic.

We collect the assumptions into a set of axioms of probability that

must be satisfied in any random experiment.
 Axioms of Probability
• Axioms of Probability

Probability is a number that is assigned to each member of a

collection of events from a random experiment that satisfies the
following properties:
(1) P(S) = 1 where S is the sample space
(2) 0 ≤ P(E) ≤ 1 for any event E
(3) For two events E1 and E2 with E1 ∩ E2 = Ø , P(E1 ∪ E2) = P(E1) +
P(E2)
 Addition Rules
• Probability of a Union

1. P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

2. If A and B are mutually exclusive events, P(A ∪ B) = P(A) + P(B).

3. P(A ∪ B ∪ C) = P(A) + P(B) + P(C) − P(A ∩ B)− P(A ∩ C) − P(B ∩ C) +

P(A ∩ B ∩ C)
• Mutually Exclusive Events:

A collection of events, E1, E2, …, Ek, is said to be mutually exclusive

if for all pairs, Ei ∩ Ej = Ø
For a collection of mutually exclusive events, P(E1 ∪ E2 ∪… ∪Ek) =
P(E1) + P(E2) + · · · P(Ek)
 Examples
• Ex.1 : Table: Location in Sputtering Tool
Contamination Center Edge Total
Low 514 68 582
High 112 246 358
Total 626 314 940
The table lists the history of 940 wafers in a semiconductor
manufacturing process. Suppose that 1 wafer is selected at
random. Denote H the event that the wafer contains high levels
of contamination, C the event that the wafer is in the center of a
sputtering tool. What is P(H ∩ C) ? P(H ∪ C)?

Solution:
 Examples…
P(H) = 358/ 940.
P(C) = 626/ 940.

Also, P(H ∩ C) is the probability that the wafer is from the center
of the sputtering tool and contains high levels of contamination.
Therefore, P(H ∩ C) = 112∕940

P(H ∪ C) can be determined to be

P(H ∪ C) = P(H) + P(C) − P(H ∩ C)
= 358/940 + 626/940− 112/940
= 872/940