2DI90
Probability &
Statistics
2DI90 – Intro and Chapter 2 of MR
Before we start…
Lecturer: Rui M. Castro
Office: MF4.075
Phone: (040 247) 2499
Email: rmcastro@[Link] (include the code 2DI90 in the subject)
Debankur Muhkerjee Lorenzo Federico Noud Kroon
2
Mayank Mayank Carla Rusch-Groot Rolf Morel
Before we start…
Setup of the course:
• 9 weeks, consisting of lectures, informal quizzes, and
tutorials/intermezzos
• Office hours (in lieu of instructions)
Evaluation:
• Four mandatory homework assignments
• Electronic test in week 9
• Final written exam
Prerequisites:
• TU/e calculus course or equivalent
All this information is available on the course webpage
[Link] 3
Before we start…
Study Materials:
• (recommended) “Applied Statistics and Probability for
•Engineers” (Montegomery & Runger) (4th, 5th or 6th edition)
• Statistical Compendium (dikt. nr. 2218)
• Notes and assignments distributed during the course
Announcements and other course materials:
• I’ll post everything on the course webpage
[Link]
CHECK THIS PAGE REGULARLY FOR NEWS,
ANNOUNCEMENTS, ETC…
4
Introduction
Why Probability?
• Many systems are very complex (e.g. an operating system)
• It is not easy to model complex systems
• Sometimes we cannot predict how users are going to behave
5
6
Introduction
Why Probability?
• Many systems are very complex (e.g. an operating system)
• It is not easy to model complex systems
• Sometimes we cannot predict how users are going to behave
THE OUTCOME OF SOME EXPERIMENTS
IS “UNPREDICTIBLE” !!!
What can we do with probabilistic models?
• Use randomness to model various events that can
happen.
• Use stochastic models to describe how users/clients
interact with a system (e.g. TCP/IP, queuing models)
• Quantify the reliability of a system 7
Relation to Statistics
Probability and Statistics are NOT the same thing!!!
• Probability provides the foundation of statistics
• Statistics is concerned with making inference from data,
assumed to be collected according to a probabilistic model
Probabilistic Model Sample
Population (World)
Statistics – Inference about the 8
population using the sample
What is this Course About?
After successfully completing this course you should be able
to:
• Perform elementary probability calculations with
stochastic models
• Identify situations where probabilistic models are
adequate and useful, and be able to simulate them
• Perform elementary statistical analyses of data
• Use statistical software in a proper way to obtain both
qualitative and quantitative data analysis results
• Recognize scenarios where linear regression is an
adequate tool, and perform simple statistical analyses in
such settings
9
An Example Out of the Street
Almost anyone is familiar with the three-card monte or
shell game:
• There are three ‘shells’, and under one of these there is
a little ball
• If you correctly identify the shell containing the ball you
double your money, otherwise you lose.
10
© Jenijen (in flickr)
A Variation of this Game
• I shuffle the ball under one of the shells without you seeing
where it goes
• You choose one of the shells
• After you made your choice I lift one of the two other shells
and show you that it is empty
• Then I give you two options:
• You can stay with the initial shell (STAY - A)
• You can pick any other shell (SWITCH - B)
• Finally I lift the shell you chose and you win if the ball is
under it.
What is the best strategy??? 11
Sample Spaces and Events (§ 2.1 MR)!
Definition: Random Experiment
Examples:
• Time to reboot a server (in seconds)
• Measuring the clock frequency in a motherboard (in GHz)
• Age (in terms of number of birthdays) of a person
• Measuring the temperature evolution in a GPU while
performing a benchmark task (in deg. Celsius) 12
Sample Space!
Definition: Sample Space
Examples:
• Time to reboot a server (in seconds)
• Measuring the clock frequency in a motherboard (in GHz)
• Age (in terms of number of birthdays) of a person
• Measuring the temperature evolution in a GPU while
performing a benchmark task (in deg. Celsius) 13
Discrete and Continuous Sample Spaces!
Examples:
• Time to reboot a server
• Measuring the clock frequency in a motherboard
• Age (in terms of number of birthdays) of a person
• Measuring the temperature evolution in a GPU while
performing a benchmark task
14
More Examples!
The choice between discrete or continuous sample spaces
might depend on the particular objective of the study…
• Messaging in a communication system:
•Three time sensitive messages are sent. These will either
be on time, or delayed
What is the sample space?
15
Events!
Definition: Events
16
Events!
Example:
Events are just sets contained in the sample space !!!
17
Set Operations!
Definition: Mutually Exclusive Events (Disjoint Events)
18
Venn Diagrams!
19
Counting Techniques!
Sometimes we need to count the number of possible outcomes.
So far we seen only easy cases, but things can get complicated
pretty quickly…
Example 1: Options in a car
• Manual or automatic transmission
• With or without air conditioning
• Three different stereo systems
• Four possible exterior colors
How many possibilities are there?
• A - 11
• B - 32
• C - 48
• D - 54 20
Permutations!
Example: 4 friends (Alice, Bob, Eve and Robert) place 4 pieces
of paper with their names in a hat. After shuffling them each
one of them takes one piece.
What are the possible outcomes of this experiment?
In this case there are 4x3x2x1=24 possibilities
Sinterklaas Challenge: If you have n friends how many
possible outcomes exist such that no one takes their own name
out of the hat???
(if n=4 the answer is 9) 21
Permutations!
Example: A motherboard has 8 slots, where we want to place 4
different cards. How many possibilities are there?
Why?
• Take card one and place it in one of the 8 positions (8
possibilities).
• Take card 2 and place it in one of the 7 remaining positions
• ... and so on
22
Combinations!
Example: A motherboard has 8 memory slots, and we want to
put 4 identical memory cards in it.
Why?
• Same as before, but now the order of the cards doesn’t
matter. So, there are 4! possible ordering of the 4 cards all
yield the same outcome.
23
Sampling With or Without Replacement!
Example: I ask you to take five cards out of a deck, one by one
without putting them back (sampling without replacement)
Example: I ask you to take a card out of the deck and look at
it, then replace it, I shuffle the deck and ask you to take a
card again, and so on, five times (sampling with replacement)
Which type of sampling is more adequate depends on the
context (but typically sampling with replacement is easier
to study)
24
Interpretation of Probability (§ 2.2 MR)"
The concept of probability is a very abstract one:
• Idea: quantify the “likelihood” or “chance” of the outcome
of an experiment…
• One can have a subjective interpretation of probability:
This might not be very satisfying, as different
people will assign different values for the
probability of an event… 25
Interpretation of Probability!
• A frequentist interpretation of probability:
This is simply the fraction of times E happens when we repeat
the experiment a large number of times !
Example:
• Take the first card out of a shuffled deck
• Consider the event {the card selected is an ace}
What do we exactly mean by the repetition of an
26
experiment ???
Axioms of Probability!
We will restrict ourselves to discrete sample spaces for now, to
avoid some technical difficulties…
By taking an axiomatic approach we can make sure everything is
well defined, if we take the axioms for granted…
27
Axioms of Probability!
Example: Random experiment – Flipping a coin
If we have a fair coin then
In general, if you have N possible outcomes:
28
Basic Properties"
Just from the axioms we can deduce a number of simple, but
useful properties:
Lemma:
29
Addition Rules (§ 2.3 MR)"
Actually, the probability of an event is just a way to measure it !!
- you can think of it as the generalized volume of the set.
Lemma:
If the events are mutually exclusive then (and only then!):
30
Proof:!
We can obviously generalize all this to multiple events… 31
Conditional Probability (§ 2.4 MR)"
This is a very important concept. In most cases events “share”
information, and so we want to see how can we take this into
account
Example: taking a card out of a shuffled deck
32
Conditional Probability!
Definition: Conditional Probability
33
An Illustrative Toyish Example!
Sammy has only four types of pairs of socks in his
drawer:
• Red or Blue socks
• Striped or plain
Striped \ Plain
Red 13 16
Blue 5 8
34
An Illustrative Toyish Example!
Sammy has only four types of pairs of socks in his
drawer:
• Red or Blue socks
• Striped or plain
Striped \ Plain 42
Red 13 16 29 Striped Plain
18 24
Blue 5 8 13 Red Blue Red Blue
18 24 42 13 5 16 8
35
An Illustrative Toyish Example!
Sammy takes a pair of socks from his drawer at random (he
is equally likely to take any of the pairs):
• What is the probability that it is a red pair?
• Given he took a pair of red socks, what is the probability
these have stripes?
Striped \ Plain
Red 13 16 29
Blue 5 8 13
18 24 42 36
An Illustrative Toyish Example!
42
Striped Plain
18 24
Red Blue Red Blue
13 5 16 8
Striped
18
Red Blue
13 5
37
Multiplication Rules (§ 2.5 MR)"
Multiplication Rule:
Total Probability Rule:
38
Total Probability Rule for Multiple Events!
39
Independence of Events (§ 2.6 MR)"
Sometimes two events might not be “related”:
• Given that the outcome of the experiment is in A might not
affect the probability that this same outcome is also in B…
Example: Take a card out of a shuffled deck
This means that B doesn’t give any probabilistic 40
information about A !!!
Definition of Independence!
Definition: Independence
Independence is a truly probabilistic concept, and not a set
relationship !!!
Challenge: Can you construct a probability model and three events
41
that are pairwise independent but not jointly independent?
Properties!
Lemma:
Proof: left as exercise…
42
Independence is IMPORTANT !!!!
Independence is a key assumption in most probability models,
e.g.:
• We assume each memory chip from a certain manufacturer
fails independently from one another
• Processes arriving to a server from different users are
assume to arrive independently from one another
Example: There are three elevators in the MetaForum, but one is
currently broken. The other two fail independently with probability
0.2 and 0.1 respectively.
What is the probability I cannot take one of the elevators to get
to the 7th floor?
A – 0.1 B – 0.2 C – 0.3 D – 0.02 43
A Simple Example!
We can think of this setting as a circuit:
Elevator 1
Elevator 2
44
Another Example!
Example: A harddrive backup unit uses redundancy to ensure
reliable operation. It consists of two redundant data storage units
(each with probability of failure of 1/100) and two independent
power supply units each with probabilities of failure respectively
1/50 (electric grid unit) and 1/120 (battery unit).
What is the overall probability of failure of such system?
Data Units Power Units
45
Bayes’s Rule (§ 2.7 MR)"
In many situations in practice what we can measure (estimate) are
conditional probabilities. It is rather useful to be able to relate
various conditional probabilities to one another.
Reverend Thomas Bayes (1702-1761):
Despite it’s simplicity, this is a very
powerful result, and forms the basis of
many inference procedures used
nowadays (e.g. both the GPS system and
the communication encoding used in 46
cellphones rely on Bayes’s rule).
Bayes’s Rule!
47
A Real Life Example!
Gerd Gigerenzer is a cognitive psychologist who studies how people
miscalculate risk.
In one study, Gigerenzer and his colleagues asked doctors in
Germany and the United States to estimate the probability that a
woman with a positive mammogram actually has breast cancer, even
though she’s in a low-risk group: 40 to 50 years old, with no
symptoms or family history of breast cancer. To make the
question specific, the doctors were told to assume the following
statistics — couched in terms of percentages and probabilities —
about the prevalence of breast cancer among women in this cohort,
and also about the mammogram’s sensitivity and rate of false
positives:
48
(see [Link] for a general audience article)
A Real Life Example!
The probability that one of these women has breast cancer is 0.8
percent. If a woman has breast cancer, the probability is 90
percent that she will have a positive mammogram. If a woman does
not have breast cancer, the probability is 7 percent that she will
still have a positive mammogram.
Imagine a woman who has a positive mammogram. What is the
probability that she actually has breast cancer?
The doctors answer for this question ranged from 1 to 99%.
What is the correct answer ???
Enter your answer as a percentage
(e.g. 47% corresponds to 47)
49
A Real Life Example!
The probability that one of these women has breast cancer is 0.8
percent. If a woman has breast cancer, the probability is 90
percent that she will have a positive mammogram. If a woman does
not have breast cancer, the probability is 7 percent that she will
still have a positive mammogram.
Imagine a woman who has a positive mammogram. What is the
probability that she actually has breast cancer?
50
A Real Life Example!
So a woman between 40 and 50 with a positive
mammogram has only about 9.5% chance of actually 51
having breast cancer…
A Paradoxical Example!
Example: Real experimental data for two different treatments of
kidney stones is shown below. Their success was studied when
patients had both small and large stones in four groups of patients:
Treatment A Treatment B
Successful Successful
Yes No Yes No
Stone Small 81 6 Stone Small 234 36
size Large 192 71 size Large 55 25
Which treatment is better?
52
Successful
Treatment A!
Yes No
Stone Small 81 6
size Large 192 71
Let’s compute the overall success probability of treatment A:
Let’s also compute conditional success probabilities:
53
Successful
Treatment B!
Yes No
Stone Small 234 36
size Large 55 25
Let’s compute the overall success probability of treatment B:
Let’s also compute conditional success probabilities:
54
A Paradoxical Example!
In terms of the overall success rate we have
So treatment B seems to be better overall…
But, For each class of patients we have
So for patients with small stones treatment A is better, and for
patients with large stones treatment A is also better…
Where’s the catch? 55
Successful
The Two Studies!
Many patients with Yes No
Few patients with small
large stones
stones Small 81 6
Stone
size Large 192 71
This is known as Simpson’s paradox, and continues to
confuse many people, even nowadays…
Few patients with large
Successful Many patients with stones
small stones
Yes No
Stone Small 234 36
size Large 55 25 56
Random Variables (§ 2.8 MR)"
Often it is useful to consider random variables. These map the
outcomes of a random experiment to a number.
Example:
• X is the random variable corresponding to the temperature of
the room at time t.
• x is the measured temperature of the room at time t.
57
Random Variables!
Next in the course:
• We are going to study both discrete and continuous random
variables, and learn about probabilistic settings that are useful
to model real-world situations…
58
