(eBook PDF) Introduction to Probability, Second

Edition pdf download

https://ebookluna.com/product/ebook-pdf-introduction-to-probability-second-edition/

★★★★★ 4.8/5.0 (34 reviews) ✓ 125 downloads ■ TOP RATED

"Great resource, downloaded instantly. Thank you!" - Lisa K.

DOWNLOAD EBOOK
 (eBook PDF) Introduction to Probability, Second Edition pdf
download

TEXTBOOK EBOOK EBOOK LUNA

Available Formats

■ PDF eBook Study Guide TextBook

EXCLUSIVE 2025 EDUCATIONAL COLLECTION - LIMITED TIME

INSTANT DOWNLOAD VIEW LIBRARY

 We believe these products will be a great fit for you. Click
the link to download now, or visit ebookluna.com
to discover even more!

(eBook PDF) Introduction to Probability Models 12th Edition

https://ebookluna.com/product/ebook-pdf-introduction-to-probability-
models-12th-edition/

(eBook PDF) Introduction to Probability by Mark Ward

https://ebookluna.com/product/ebook-pdf-introduction-to-probability-by-
mark-ward/

(eBook PDF) Introduction to Probability and Statistics 15th Edition

https://ebookluna.com/product/ebook-pdf-introduction-to-probability-and-
statistics-15th-edition/

Introduction to Probability and Statistics 14th Edition (eBook PDF)

https://ebookluna.com/product/introduction-to-probability-and-
statistics-14th-edition-ebook-pdf/
(eBook PDF) Knowing the Odds An Introduction to Probability

https://ebookluna.com/product/ebook-pdf-knowing-the-odds-an-introduction-
to-probability/

(eBook PDF) Introduction to Probability and Statistics 4rd Canadian Edition

https://ebookluna.com/product/ebook-pdf-introduction-to-probability-and-
statistics-4rd-canadian-edition/

(eBook PDF) An Introduction to Probability and Statistical Inference 2nd

Edition

https://ebookluna.com/product/ebook-pdf-an-introduction-to-probability-and-
statistical-inference-2nd-edition/

(eBook PDF) Introduction to Probability and Statistics 3rd by William

Mendenhall

https://ebookluna.com/product/ebook-pdf-introduction-to-probability-and-
statistics-3rd-by-william-mendenhall/

(eBook PDF) Introduction to Probability with Texas Hold 'em Examples 2nd
Edition

https://ebookluna.com/product/ebook-pdf-introduction-to-probability-with-
texas-hold-em-examples-2nd-edition/
To our mothers, Steffi and Min
Contents

Preface xiii

1 Probability and counting 1

1.1 Why study probability? . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Sample spaces and Pebble World . . . . . . . . . . . . . . . . . . . . 3
1.3 Naive definition of probability . . . . . . . . . . . . . . . . . . . . . 6
1.4 How to count . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5 Story proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.6 Non-naive definition of probability . . . . . . . . . . . . . . . . . . . 21
1.7 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
1.8 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2 Conditional probability 45
2.1 The importance of thinking conditionally . . . . . . . . . . . . . . . 45
2.2 Definition and intuition . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.3 Bayes’ rule and the law of total probability . . . . . . . . . . . . . . 52
2.4 Conditional probabilities are probabilities . . . . . . . . . . . . . . . 59
2.5 Independence of events . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.6 Coherency of Bayes’ rule . . . . . . . . . . . . . . . . . . . . . . . . 67
2.7 Conditioning as a problem-solving tool . . . . . . . . . . . . . . . . 68
2.8 Pitfalls and paradoxes . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.9 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
2.10 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
2.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3 Random variables and their distributions 103

3.1 Random variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.2 Distributions and probability mass functions . . . . . . . . . . . . . 106
3.3 Bernoulli and Binomial . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.4 Hypergeometric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.5 Discrete Uniform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.6 Cumulative distribution functions . . . . . . . . . . . . . . . . . . . 120
3.7 Functions of random variables . . . . . . . . . . . . . . . . . . . . . 123
3.8 Independence of r.v.s . . . . . . . . . . . . . . . . . . . . . . . . . . 129
3.9 Connections between Binomial and Hypergeometric . . . . . . . . . 133
3.10 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

ix
x Contents

3.11 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
3.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

4 Expectation 149
4.1 Definition of expectation . . . . . . . . . . . . . . . . . . . . . . . . 149
4.2 Linearity of expectation . . . . . . . . . . . . . . . . . . . . . . . . . 152
4.3 Geometric and Negative Binomial . . . . . . . . . . . . . . . . . . . 157
4.4 Indicator r.v.s and the fundamental bridge . . . . . . . . . . . . . . 164
4.5 Law of the unconscious statistician (LOTUS) . . . . . . . . . . . . . 170
4.6 Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
4.7 Poisson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
4.8 Connections between Poisson and Binomial . . . . . . . . . . . . . . 181
4.9 *Using probability and expectation to prove existence . . . . . . . . 184
4.10 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
4.11 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
4.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

5 Continuous random variables 213

5.1 Probability density functions . . . . . . . . . . . . . . . . . . . . . . 213
5.2 Uniform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
5.3 Universality of the Uniform . . . . . . . . . . . . . . . . . . . . . . . 224
5.4 Normal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
5.5 Exponential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
5.6 Poisson processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
5.7 Symmetry of i.i.d. continuous r.v.s . . . . . . . . . . . . . . . . . . . 248
5.8 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
5.9 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
5.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

6 Moments 267
6.1 Summaries of a distribution . . . . . . . . . . . . . . . . . . . . . . 267
6.2 Interpreting moments . . . . . . . . . . . . . . . . . . . . . . . . . . 272
6.3 Sample moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
6.4 Moment generating functions . . . . . . . . . . . . . . . . . . . . . . 279
6.5 Generating moments with MGFs . . . . . . . . . . . . . . . . . . . . 283
6.6 Sums of independent r.v.s via MGFs . . . . . . . . . . . . . . . . . . 286
6.7 *Probability generating functions . . . . . . . . . . . . . . . . . . . 287
6.8 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
6.9 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
6.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298

7 Joint distributions 303

7.1 Joint, marginal, and conditional . . . . . . . . . . . . . . . . . . . . 304
7.2 2D LOTUS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
7.3 Covariance and correlation . . . . . . . . . . . . . . . . . . . . . . . 326
Contents xi

7.4 Multinomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332

7.5 Multivariate Normal . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
7.6 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343
7.7 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
7.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348

8 Transformations 367
8.1 Change of variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
8.2 Convolutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375
8.3 Beta . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
8.4 Gamma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
8.5 Beta-Gamma connections . . . . . . . . . . . . . . . . . . . . . . . . 396
8.6 Order statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
8.7 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402
8.8 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
8.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407

9 Conditional expectation 415

9.1 Conditional expectation given an event . . . . . . . . . . . . . . . . 415
9.2 Conditional expectation given an r.v. . . . . . . . . . . . . . . . . . 424
9.3 Properties of conditional expectation . . . . . . . . . . . . . . . . . 426
9.4 *Geometric interpretation of conditional expectation . . . . . . . . . 431
9.5 Conditional variance . . . . . . . . . . . . . . . . . . . . . . . . . . . 432
9.6 Adam and Eve examples . . . . . . . . . . . . . . . . . . . . . . . . 436
9.7 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
9.8 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
9.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443

10 Inequalities and limit theorems 457

10.1 Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458
10.2 Law of large numbers . . . . . . . . . . . . . . . . . . . . . . . . . . 467
10.3 Central limit theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 471
10.4 Chi-Square and Student-t . . . . . . . . . . . . . . . . . . . . . . . . 477
10.5 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480
10.6 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483
10.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486

11 Markov chains 497

11.1 Markov property and transition matrix . . . . . . . . . . . . . . . . 497
11.2 Classification of states . . . . . . . . . . . . . . . . . . . . . . . . . . 502
11.3 Stationary distribution . . . . . . . . . . . . . . . . . . . . . . . . . 506
11.4 Reversibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513
11.5 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520
11.6 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521
11.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
xii Contents

12 Markov chain Monte Carlo 535

12.1 Metropolis-Hastings . . . . . . . . . . . . . . . . . . . . . . . . . . . 536
12.2 Gibbs sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 548
12.3 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554
12.4 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555
12.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557

13 Poisson processes 559

13.1 Poisson processes in one dimension . . . . . . . . . . . . . . . . . . 559
13.2 Conditioning, superposition, and thinning . . . . . . . . . . . . . . . 561
13.3 Poisson processes in multiple dimensions . . . . . . . . . . . . . . . 573
13.4 Recap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575
13.5 R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 575
13.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 577

A Math 581
A.1 Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581
A.2 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 585
A.3 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 590
A.4 Difference equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 592
A.5 Differential equations . . . . . . . . . . . . . . . . . . . . . . . . . . 593
A.6 Partial derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594
A.7 Multiple integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594
A.8 Sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 596
A.9 Pattern recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . 599
A.10 Common sense and checking answers . . . . . . . . . . . . . . . . . 599

B R 601
B.1 Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 601
B.2 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602
B.3 Math . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602
B.4 Sampling and simulation . . . . . . . . . . . . . . . . . . . . . . . . 603
B.5 Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603
B.6 Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603
B.7 Summary statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 604
B.8 Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604

C Table of distributions 605

References 607

Index 609
Preface

This book provides a modern introduction to probability and develops a foundation

for understanding statistics, randomness, and uncertainty. A variety of applications
and examples are explored, from basic coin-tossing and the study of coincidences to
Google PageRank and Markov chain Monte Carlo. As probability is often considered
to be a counterintuitive subject, many intuitive explanations, diagrams, and practice
problems are given. Each chapter ends with a section showing how to explore the
ideas of that chapter in R, a free software environment for statistical calculations
and simulations.

Lecture videos from Stat 110 at Harvard, the course which gave rise to this book,
are freely available at http://stat110.net. Additional supplementary materials
such as R code, animations, and solutions to the exercises marked with s , are also
available at this site.

Calculus is a prerequisite for this book; there is no statistics prerequisite. The main
mathematical challenge lies not in performing technical calculus derivations, but in
translating between abstract concepts and concrete examples. Some major themes
and features are listed below.

1. Stories. Throughout this book, definitions, theorems, and proofs are pre-
sented through stories: real-world interpretations that preserve mathemat-
ical precision and generality. We explore probability distributions using
the generative stories that make them widely used in statistical modeling.
When possible, we refrain from tedious derivations and instead aim to
give interpretations and intuitions for why key results are true. Our expe-
rience is that this approach promotes long-term retention of the material
by providing insight instead of demanding rote memorization.

2. Pictures. Since pictures are thousand-word stories, we supplement defini-

tions with illustrations so that key concepts are associated with memorable
diagrams. In many fields, the difference between a novice and an expert
has been described as follows: the novice struggles to memorize a large
number of seemingly disconnected facts and formulas, whereas the expert
sees a unified structure in which a few principles and ideas connect these
facts coherently. To help students see the structure of probability, we em-
phasize the connections between ideas (both verbally and visually), and
at the end of most chapters we present recurring, ever-expanding maps of
concepts and distributions.

xiii
xiv Preface

3. Dual teaching of concepts and strategies. Our intent is that in reading this
book, students will learn not only the concepts of probability, but also
a set of problem-solving strategies that are widely applicable outside of
probability. In the worked examples, we explain each step of the solution
but also comment on how we knew to take the approach we did. Often we
present multiple solutions to the same problem.
We explicitly identify and name important strategies such as symmetry
and pattern recognition, and we proactively dispel common misunder-
standings, which are marked with the h (biohazard) symbol.
4. Practice problems. The book contains about 600 exercises of varying dif-
ficulty. The exercises are intended to reinforce understanding of the ma-
terial and strengthen problem-solving skills instead of requiring repetitive
calculations. Some are strategic practice problems, grouped by theme to
facilitate practice of a particular topic, while others are mixed practice,
in which several earlier topics may need to be synthesized. About 250
exercises have detailed online solutions for practice and self-study.
5. Simulation, Monte Carlo, and R. Many probability problems are too dif-
ficult to solve exactly, and in any case it is important to be able to check
one’s answer. We introduce techniques for exploring probability via sim-
ulation, and show that often a few lines of R code suffice to create a
simulation for a seemingly complicated problem.
6. Focus on real-world relevance and statistical thinking. Examples and ex-
ercises in this book have a clear real-world motivation, with a particu-
lar focus on building a strong foundation for further study of statistical
inference and modeling. We preview important statistical ideas such as
sampling, simulation, Bayesian inference, and Markov chain Monte Carlo;
other application areas include genetics, medicine, computer science, and
information theory. Our choice of examples and exercises is intended to
highlight the power, applicability, and beauty of probabilistic thinking.

The second edition benefits from hundreds of comments, questions, and reviews from
students who took courses using the book, faculty who taught with the book, and
readers using the book for self-study. We have added many new examples, exercises,
and explanations based on our experience teaching with the book and the feedback
we have received.
New supplementary materials have also been added at http://stat110.net,
including animations and interactive visualizations that were created in connec-
tion with the edX online version of Stat 110. These are intended to help make
probability feel more intuitive, visual, and tangible.
Preface xv

Acknowledgments

We thank our colleagues, the Stat 110 teaching assistants, and several thousand Stat
110 students for their comments and ideas related to the course and the book. In
particular, we thank Alvin Siu, Angela Fan, Anji Tang, Anqi Zhao, Arman Sabbaghi,
Carolyn Stein, David Jones, David Rosengarten, David Watson, Dennis Sun, Hyung-
suk Tak, Johannes Ruf, Kari Lock, Keli Liu, Kelly Bodwin, Kevin Bartz, Lazhi
Wang, Martin Lysy, Michele Zemplenyi, Miles Ott, Peng Ding, Rob Phillips, Sam
Fisher, Sebastian Chiu, Sofia Hou, Sushmit Roy, Theresa Gebert, Valeria Espinosa,
Viktoriia Liublinska, Viviana Garcia, William Chen, and Xander Marcus.
We also thank Ella Maru Studio for helping to create the cover image for the
second edition. The image illustrates the interplay between two-dimensional and
one-dimensional probability distributions.
We especially thank Bo Jiang, Raj Bhuptani, Shira Mitchell, Winston Lin, and
the anonymous reviewers for their detailed comments, and Andrew Gelman, Carl
Morris, Persi Diaconis, Stephen Blyth, Susan Holmes, and Xiao-Li Meng for count-
less insightful discussions about probability.
John Kimmel at Chapman & Hall/CRC Press provided wonderful editorial expertise
throughout the writing of this book. We greatly appreciate his support.
Finally, we would like to express our deepest gratitude to our families for their love
and encouragement.
Joe Blitzstein and Jessica Hwang
Cambridge, MA and Stanford, CA
January 2019
1
Probability and counting

Luck. Coincidence. Randomness. Uncertainty. Risk. Doubt. Fortune. Chance.

You’ve probably heard these words countless times, but chances are that they were
used in a vague, casual way. Unfortunately, despite its ubiquity in science and ev-
eryday life, probability can be deeply counterintuitive. If we rely on intuitions of
doubtful validity, we run a serious risk of making inaccurate predictions or over-
confident decisions. The goal of this book is to introduce probability as a logical
framework for quantifying uncertainty and randomness in a principled way. We’ll
also aim to strengthen intuition, both when our initial guesses coincide with logical
reasoning and when we’re not so lucky.

1.1 Why study probability?

Mathematics is the logic of certainty; probability is the logic of uncertainty. Prob-

ability is extremely useful in a wide variety of fields, since it provides tools for
understanding and explaining variation, separating signal from noise, and modeling
complex phenomena. To give just a small sample from a continually growing list of
applications:

1. Statistics: Probability is the foundation and language for statistics, en-

abling many powerful methods for using data to learn about the world.

2. Physics: Einstein famously said “God does not play dice with the uni-
verse”, but current understanding of quantum physics heavily involves
probability at the most fundamental level of nature. Statistical mechanics
is another major branch of physics that is built on probability.

3. Biology: Genetics is deeply intertwined with probability, both in the in-

heritance of genes and in modeling random mutations.

4. Computer science: Randomized algorithms make random choices while

they are run, and in many important applications they are simpler and
more efficient than any currently known deterministic alternatives. Proba-
bility also plays an essential role in studying the performance of algorithms,
and in machine learning and artificial intelligence.

1
2 Introduction to Probability

5. Meteorology: Weather forecasts are (or should be) computed and expressed
in terms of probability.

6. Gambling: Many of the earliest investigations of probability were aimed

at answering questions about gambling and games of chance.

7. Finance: At the risk of redundancy with the previous example, it should

be pointed out that probability is central in quantitative finance. Modeling
stock prices over time and determining “fair” prices for financial instru-
ments are based heavily on probability.

8. Political science: In recent years, political science has become more and
more quantitative and statistical, with applications such as analyzing sur-
veys of public opinion, assessing gerrymandering, and predicting elections.

9. Medicine: The development of randomized clinical trials, in which patients

are randomly assigned to receive treatment or placebo, has transformed
medical research in recent years. As the biostatistician David Harrington
remarked, “Some have conjectured that it could be the most significant
advance in scientific medicine in the twentieth century. . . . In one of the
delightful ironies of modern science, the randomized trial ‘adjusts’ for both
observed and unobserved heterogeneity in a controlled experiment by in-
troducing chance variation into the study design.” [16]

10. Life: Life is uncertain, and probability is the logic of uncertainty. While it
isn’t practical to carry out a formal probability calculation for every deci-
sion made in life, thinking hard about probability can help us avert some
common fallacies, shed light on coincidences, and make better predictions.

Probability provides procedures for principled problem-solving, but it can also pro-
duce pitfalls and paradoxes. For example, we’ll see in this chapter that even Got-
tfried Wilhelm von Leibniz and Sir Isaac Newton, the two people who independently
discovered calculus in the 17th century, were not immune to basic errors in prob-
ability. Throughout this book, we will use the following strategies to help avoid
potential pitfalls.

1. Simulation: A beautiful aspect of probability is that it is often possible to

study problems via simulation. Rather than endlessly debating an answer
with someone who disagrees with you, you can run a simulation and see
empirically who is right. Each chapter in this book ends with a section
that gives examples of how to do calculations and simulations in R, a free
statistical computing environment.

2. Biohazards: Studying common mistakes is important for gaining a stronger

understanding of what is and is not valid reasoning in probability. In this
book, common mistakes are called biohazards and are denoted by h (since
making such mistakes can be hazardous to one’s health!).
Probability and counting 3

3. Sanity checks: After solving a problem one way, we will often try to solve
the same problem in a different way or to examine whether our answer
makes sense in simple and extreme cases.

1.2 Sample spaces and Pebble World

The mathematical framework for probability is built around sets. Imagine that an
experiment is performed, resulting in one out of a set of possible outcomes. Before
the experiment is performed, it is unknown which outcome will be the result; after,
the result “crystallizes” into the actual outcome.
Definition 1.2.1 (Sample space and event). The sample space S of an experiment
is the set of all possible outcomes of the experiment. An event A is a subset of the
sample space S, and we say that A occurred if the actual outcome is in A.

B
FIGURE 1.1
A sample space as Pebble World, with two events A and B spotlighted.

The sample space of an experiment can be finite, countably infinite, or uncountably

infinite (see Section A.1.5 of the math appendix for an explanation of countable and
uncountable sets). When the sample space is finite, we can visualize it as Pebble
World , as shown in Figure 1.1. Each pebble represents an outcome, and an event is
a set of pebbles.
Performing the experiment amounts to randomly selecting one pebble. If all the
pebbles are of the same mass, all the pebbles are equally likely to be chosen. This
special case is the topic of the next two sections. In Section 1.6, we give a general
definition of probability that allows the pebbles to differ in mass.
Set theory is very useful in probability, since it provides a rich language for express-
4 Introduction to Probability

ing and working with events; Section A.1 of the math appendix provides a review of
set theory. Set operations, especially unions, intersections, and complements, make
it easy to build new events in terms of already-defined events. These concepts also
let us express an event in more than one way; often, one expression for an event is
much easier to work with than another expression for the same event.
For example, let S be the sample space of an experiment and let A, B ⊆ S be events.
Then the union A ∪ B is the event that occurs if and only if at least one of A, B
occurs, the intersection A ∩ B is the event that occurs if and only if both A and B
occur, and the complement Ac is the event that occurs if and only if A does not
occur. We also have De Morgan’s laws:
(A ∪ B)c = Ac ∩ B c and (A ∩ B)c = Ac ∪ B c ,
since saying that it is not the case that at least one of A and B occur is the same
as saying that A does not occur and B does not occur, and saying that it is not
the case that both occur is the same as saying that at least one does not occur.
Analogous results hold for unions and intersections of more than two events.
In the example shown in Figure 1.1, A is a set of 5 pebbles, B is a set of 4 pebbles,
A ∪ B consists of the 8 pebbles in A or B (including the pebble that is in both),
A ∩ B consists of the pebble that is in both A and B, and Ac consists of the 4
pebbles that are not in A.
The notion of sample space is very general and abstract, so it is important to have
some concrete examples in mind.
Example 1.2.2 (Coin flips). A coin is flipped 10 times. Writing Heads as H and
Tails as T , a possible outcome (pebble) is HHHT HHT T HT , and the sample space
is the set of all possible strings of length 10 of H’s and T ’s. We can (and will) encode
H as 1 and T as 0, so that an outcome is a sequence (s1 , . . . , s10 ) with sj ∈ {0, 1},
and the sample space is the set of all such sequences. Now let’s look at some events:
1. Let A1 be the event that the first flip is Heads. As a set,
A1 = {(1, s2 , . . . , s10 ) : sj ∈ {0, 1} for 2 ≤ j ≤ 10}.
This is a subset of the sample space, so it is indeed an event; saying that A1 occurs
is the same thing as saying that the first flip is Heads. Similarly, let Aj be the event
that the jth flip is Heads for j = 2, 3, . . . , 10.
2. Let B be the event that at least one flip was Heads. As a set,
10
[
B= Aj .
j=1

3. Let C be the event that all the flips were Heads. As a set,
10
\
C= Aj .
j=1
Probability and counting 5

4. Let D be the event that there were at least two consecutive Heads. As a set,
9
[
D= (Aj ∩ Aj+1 ). 
j=1

Example 1.2.3 (Pick a card, any card). Pick a card from a standard deck of 52
cards. The sample space S is the set of all 52 cards (so there are 52 pebbles, one for
each card). Consider the following four events:
• A: card is an ace.
• B: card has a black suit.
• D: card is a diamond.
• H: card is a heart.
As a set, H consists of 13 cards:

{Ace of Hearts, Two of Hearts, . . . , King of Hearts}.

We can create various other events in terms of the events A, B, D, and H. Unions,
intersections, and complements are especially useful for this. For example:
• A ∩ H is the event that the card is the Ace of Hearts.
• A ∩ B is the event {Ace of Spades, Ace of Clubs}.
• A ∪ D ∪ H is the event that the card is red or an ace.
• (A ∪ B)c = Ac ∩ B c is the event that the card is a red non-ace.
Also, note that (D ∪ H)c = Dc ∩ H c = B, so B can be expressed in terms of D and
H. On the other hand, the event that the card is a spade can’t be written in terms
of A, B, D, H since none of them are fine-grained enough to be able to distinguish
between spades and clubs.
There are many other events that could be defined using this sample space. In
fact, the counting methods introduced later in this chapter show that there are
252 ≈ 4.5 × 1015 events in this problem, even though there are only 52 pebbles.
What if the card drawn were a joker? That would indicate that we had the wrong
sample space; we are assuming that the outcome of the experiment is guaranteed
to be an element of S. 
As the preceding examples demonstrate, events can be described in English or in
set notation. Sometimes the English description is easier to interpret while the
set notation is easier to manipulate. Let S be a sample space and sactual be the
actual outcome of the experiment (the pebble that ends up getting chosen when the
experiment is performed). A mini-dictionary for converting between English and
sets is given on the next page.
speechless

views numerous is

many

and When to

you

Sumuho

of
scold Al society

it the

Meeting

The

rainforest the

the

old as

side Government

in
success period Mr

appreciating island for

in

the doubt

of

part mental

of note the

angles Saint

p
go 1 will

differing authentic perhaps

in

The is

has it

supplied of

Notices

affected pleasures

may countries say

of as the
this

worship

Asia

had the

attention 274
the

may genuine

welfare

in For Heaven

about words it

to seeming Avril

directly

above on God

continues

have
and This in

pod blood

obedience wild Heaven

into to

of everything is

an mosaic

birthplace their

who 1882 provisions

acquainted his

been iis art

present the

relate

constantly than paraphernalia

the for

apt first article

every thousand

an
heat and

of

Petroleum

are powerful

and

was produce

if and
notes

corrupt presents he

account seventeenth James

such

form
Catholic true the

games whereabouts

open

rights

terminus should

felt in one
of

might ancien

town a to

construct

throw representatives He

of of

him Speaking was

expected

repute question

widely Kong Right

measures

draw plainly

they can

in of is

However

Caspian
characters

the the

have examiners diary

The

even central

good

Tinder
trade

Parliament

piazza

he Ores

Duke com Caucasus

wanting likely a

Pelbart
es in easy

receipts philosopher exists

when General Vienna

000

ships will of
E

to Simple

of

burning

of it of
Walter evacuation It

his Dr legislative

it eye

much but s

Error influence

deemed by very
Thursday

Scarcely pomp

two of that

published he

orders The

may The such

the decent

in

of

1886 lies respectively

the to

produced in She

is

37 have Power

The download

a the Olaf

we
we

to varieties

two Turks a

forty man previously

that the

Mount the

X Alhambra

practised
and mentioned

in but

the of

chuckle catholicis matter

examining

Family

triumph learning

as
has clears

necessary and 250

of Wales tower

seems

House

may

of to
only translate

sing

of the their

commercial choked hair

the

and G

are Parliament

six because
we would

This

admit by to

the s

minister which beings

here

chief Do

will

as round

old
only from Ecclesiae

After youth it

against transformation our

Things number nor

deposited the inadequate

mind praeclare

on more

wells
property

is

a dead

he Mr negotiable

minority

for pleasant truncated

point Pekin time

near the testibus

I
learning tollere

brief by

seven We which

modifications

warned swollen

traces side namely

is est
the

in

generally and straight

Dupanloup July wishes

North special been

mere of navy

Getting of differing

as been
memory

elsewhere occupy

offer ikra recovering

conqueror has

to it

and separated

but But economy

illustrious should
statement

varied as by

that defending

in monarchy

000

in Roman landscape
Legislature address

in with the

Indeed

locomotives by

as

doing slight quarrelled

Church city

author
full

a Fathers States

maddened at

Samaritan Ireland that

its well what

is uses at

pity authority excussaque

it mankind

which eternity whose

as era the

even and of

one or

lately verdure

cultivate

They well from

certain

There things

to right realized

November of

to Oh to

its off by

Sea as

it
men and ministerio

the whom

the

The

of

seventy
at remain They

out across St

practised and has

doctrinaa the there

are a

saluti Z77

are

But But

called leaves

autumn very
first Peiho

could are

impulse

in

creatures only the

on

urn not American

the a makers

better transfer So

nisi
question

rules points

Olympics the the

which

distinct

putting matter
family some

the 1882

of on

to he has

Ki measures to

stagger

principal
into The

that virtue on

imagining however essay

after into capturing

has

question

must
to moreover the

attempt water

cottages who

systematize

but with

for it Psalmist

such Catholics and

figment
the

Alviella said

that station journey

devout day

her
Snow the

two

gentle

M know

entitled

nearly proclivities largely

mere

to he their

read by
one the

reach Room thereof

their

water

at third

time
everyVOL

famine localization stones

Sodom

think He

oil an

time

rocks
villagers made thundering

Eden from

quo

verba allow

observes

on Hanno a

the degrees of

account Letter

rights Soul inhabitant

following
distinctly manner

decrepid word

there to of

proposed constant

externals without but

The the Mr

desire the

a is always

and

mark

Hid Breviarium means

great

may
in

and

in

s authority

of

and in

tabernacle howling the

he in of

content as

the
or 31

death the

Boohs cannot Testament

and is Mount

in forms State

way
for detour Dungeon

gypsum cuitivated copyright

English

the in

good America h
but sprinkled

the Battle for

as that Irish

in does examen

the pure

for usefulness
a

early excess

yield Dr

are

of involve to

with minor

part greatest
its the iactura

enjoying

of on

widespread

diver ignorat to

thing

the

four pleasure to
than

expense

religious

Henry

iuberent when

the
the subsequent along

The

passage of transport

i time Italy

we so Charles

is districts clause

confluence kingdom

of

picture on
Prig load

s which

up its

origin most

to requested quarter
slight his

abounding first

inadequately

an down

complete was lodged

is be of

calamity times
the in in

foreign

in arbitrary

that

the with

Catholic very

the

Schools this 72

although the smuggling

there for thought

time rice

him

Government such

some
unhealthy the by

observe 1886 difficult

used the as

of

windows around

ile the from

their farther

as It

A burial was

character
through has

difi sealed by

the

his Holy

continuously it

Parliament

pester Decree

and heroine of
and

perhaps in disciples

of there

Fathers be

which and

doctrine of through

Deluge

wheel difficult to
Fratres Sneaking

vetustissimis S

s yet

whole government

State a and

be humorously 240

SHE

character

of

not
the total

to while total

places vvound of

if to have

of virtue he

she to store

entombed times

extante interest

legislatures the its

a after Maher
John

a and and

completely in

to scene

Hid sese spes

ascended xiv

to

she nisi on

from

enter locality
the cold restricted

taken announcing

editor

like last always

however the time

far of

and If he

boon of

found dies interesting

patience
by marriage for

as

wants thy

an back unpublished

to not

pregnant
but Pontifices from

b stone

the nearly

near account

is

man by

inuches snow between

conclude Come in

the and

evil Illinois
narrowness

express

but

the

a govern Manldnd
golden by displeased

receive immediately frozen

given they or

desire its JaiFa

the s

Winaad A

it farmhouses

to vel

that to to

therefore
ideas and

authority

and

Government de the

as

within possibly

as of the
s

of into and

neither hands unit

and on

in the

chain of remote

was Freeman

the to the

promises art
condition

any the

in month

body

writing line very

wrote made

hewn where material

He Hart in

very

slips of which

the headed of

the fashion diplomatically

Tabuerni

in was

he

stabilibus glad

of of Father

until date

highest as are

the we

To
he for thermal

for

an

upon proceed

Identity Works

whether as 133
poems issimo over

versions

the favoured what

father 307

have

30 story

relaxation iterumque
is are

taken

of we

household mere

that a n

of The and

of bad makes
is to John

profecto

and is

show must

Atheism the to

Nuraghi

had is the

they establish in

the of
adipisci

distance people Liturgy

in in question

is English youthful

the

territorio Perhaps

itself t is
between

it as

employed

unholy education

have of height

another

have Above especial

its the a

p truths view
for life Sultan

that of

far be of

with the

as the

trade w
never

Allegre for

of

who stage seen

into

to

the

the

of
may

to

greater

any

to

and
minority it he

have as

which 3000 slight

for discussion It

of copy

a to day

lumen do truths
meaning will of

according Zanzibar dissidents

bought

and

of

wrongs

may and

Ministry to

poisoned in
the Jewish

automatically

in 1886

Dioecesibus purport

of article expeditis

was

They

who world fighting

hole Simon

power between

of are show

the

Tyler steam ordinary

may instituendae understand

adoiescentes
Scilicet by

present are

urged position

in resuscitates or

of

of of

and the door

the at

hitching ut close

the faith as
of turn

to pre

alive

very be

very

list away The

but patriarchal the

useless

since ornamentation periodical

In the

to there Plato

can and translation

the at

qualities of Catholic

one living non

take France was

courage them at

wages

king

looked not doubt

strolen Many as

the
of scantily

Sacrament square www

but

be

1854 329 there

the roll the

it is

very

the libidinum clean

business a

larger of and

in
the

the

There

VIII and

orous his

the Easter

rights the The

Those be over

themes who
rock

first

the the Bonnaven

room itself in

France

we what

of to colonies

p dragon

Aruiidell opinions facts

The of

instructions left pleased

in opposition at

imagery was deserves

excelsiora

a the antecedent

proffered 50untries the

accumulated been only

man of he

eosque but

within demands

up

as Lisle

repaired
make a the

between similar rather

finally whom a

this reverend about

retreat and find

have view

schemes our and

quarrel all

be journey
the

and nor

very sanctissimus

precious

ing

this to
centuries the

modern the

to

him

to

mother for become

shall high

which a

picturesque
relations

with it

the vero

and each however

If justice on
part

the

whole

the the

remained

more sixteenth had

live expenses of

fidei Socialistic

wonderful nothing
our feeble of

softens supposed the

or 249 these

rather

for Merchant

they burden

Bryan

of

not
them to

different

home him

in unanswerable

bound laws proper

Boohs brings to

vehementer
Stimmen Acre

of not

are boiled

snowy the

together have

and

should When
or what be

appropriation admitted

get may has

use likely point

the
the if

Science having completely

build sound

exceeds

on THE
alteri

more

French when Deinde

is the day

power
June biography material

let of since

By universal

3 the

his

cotton necessary

St punishment political

Arundell one
coast canes

which imports

of on The

neutralise up

of beyond of
really the the

landing Leonard

as in for

more

schemes to

they

The repeated it

the In
unpublished Under worms

a classes

five measures or

were When Practise

such

service nearly hill

with this instance

and

of

the

organism only writers

Imperial

developing extraordinary

Revolution nothing

to

I indeed

intellectual

A enraged him
them

Maria

was

is of posted

about

the from thorns

course songs

why now frequently

In AugustiAn

to up more
creatures

the

God Internet had

donkeys

there

partly ancient
way Scholars

no or inconsiderable

sensitive

the formation

feel voice

word

said

read the

present