10510134: Probability Theory and Mathematical Statistics Fall 2023

Recitation 1

1.1 Sets and Events

1.1.1 Review

Definition 1.1 (Set Operations).

1. The complement of an event A, denoted by Ac and read “not A”, is the set of all outcomes in
S that are not contained in A.
2. The intersection of two events A and B, denoted by A ∩ B and read “A and B” is the event
consisting of all outcomes that are in both A and B.
3. The union of two events A and B, denoted by A ∪ B and read “A or B” is the event consisting
of all outcomes in at least one of the events.

Definition 1.2 (Disjoint Events). Two events A and B are called disjoint or mutually exclusive if
they have no outcomes in common, or mathematically, A ∩ B = ∅.

Definition 1.3 (Partition). We say that events A1 , . . . , An is a partition of an event A if Ai , Aj are

disjoint for any i ̸= j, and
A = A1 ∪ · · · ∪ An .

Theorem 1.4 (De Morgan s Laws). Let A1 , . . . , An be n events in the sample space of some experiment.
Then

1. (A1 ∪ · · · ∪ An )c = Ac1 ∩ · · · ∩ Acn ;

2. (A1 ∩ · · · ∩ An )c = Ac1 ∪ · · · ∪ Acn .

1.1.2 Exercise

Exercise 1. Consider the experiment of rolling a dice. Define event A as “the resulting number is
even”, and define event B as “the resulting number is at least 4”.

1. Write down the sets corresponding to events A and B.

2. What are Ac , B c , A ∩ B, A ∪ B, A \ B, A ∩ Ac and A ∪ Ac ?

1-1
1-2

Answer:
A = {2, 4, 6}, B = {4, 5, 6}
Ac = {1, 3, 5}, B c = {1, 2, 3}
A \ B = {2}
A ∩ B = {4, 6}, A ∪ B = {2, 4, 5, 6}
A ∩ Ac = ∅, A ∪ Ac = {1, 2, 3, 4, 5, 6} = S

Exercise 2. For any three events A, B, C defined on a sample space S, prove the following laws of sets
operations:

1. Complementation:
c
(Ac ) = A,
∅c = S,
S c = ∅,
2. Commutativity ( ) of union and intersection:

A∪B =B∪A
A ∩ B = B ∩ A.

3. Associativity ( ) of union and intersection:

(A ∪ B) ∪ C = A ∪ (B ∪ C)
(A ∩ B) ∩ C = A ∩ (B ∩ C)

4. Distributivity laws ( ):

A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C),
A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C).

More generally, for any n ≥ 1,

( ) n ( ∩
∩ ∪
n ∪ )
B Ai = B Ai ,
i=1 i=1
(n ) n (
∪ ∩ ∩ ∪ )
B Ai = B Ai .
i=1 i=1

Answer:

1. First note that

Ac = {x | x ∈ S and x ∈
/ A} .
 1-3

In order to prove that (Ac )c = A, we need to prove (Ac )c ⊆ A and A ⊆ (Ac )c . For any x ∈ (Ac )c ,
we have x ̸∈ Ac but x ∈ S, so we must have x ∈ A. This means that (Ac )c ⊆ A. On the other
hand, for any x ∈ A, x ̸∈ Ac and x ∈ S so x ∈ (Ac )c . Therefore, we have (Ac )c = A.
Second, to prove ∅c = S, we again need to prove S ⊆ ∅c and ∅c ⊆ S. The former is trivial because
any x ∈ S surely does not belong to ∅ as ∅ contains no element. The latter is trivial because the
definition of set complement is relative to the sample space S. Any element in the complement
of any event has to belong to S to begin with.
Third,
S c = {x | x ∈ S and x ̸∈ S} .

Obviously, no element x can satisfy both x ∈ S and x ̸∈ S. Thus S c = ∅.

2. For ∀x ∈ A ∪ B, (x ∈ A or x ∈ B) ⇔ (x ∈ B or x ∈ A) ⇔ (x ∈ B ∪ A). We have A ∪ B ⊆ B ∪ A
For ∀x ∈ B ∪ A, (x ∈ B or x ∈ A) ⇔ (x ∈ A or x ∈ B)⇔ (x ∈ A ∪ B). We have B ∪ A ⊆ A ∪ B.
Thus we have A ∪ B = B ∪ A.
We can similarly prove that A ∩ B = B ∩ A, just via replacing all ∪ by ∩ and all “or” by “and”
in the proof above.
3. Any x ∈ A ∪ (B ∪ C) ⇔ x ∈ A or x ∈ B ∪ C ⇔ x ∈ A or x ∈ B or x ∈ C ⇔ x ∈ A ∪ B or x ∈
C ⇔ x ∈ (A ∪ B) ∪ C. Thus A ∪ (B ∪ C) = (A ∪ B) ∪ C.
We can similarly prove (A ∩ B) ∩ C = A ∩ (B ∩ C) via replacing all ∪ by ∩ and all “or” by “and”
in the proof above.
4. To prove A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C), we need to prove A ∪ (B ∩ C) ⊆ (A ∪ B) ∩ (A ∪ C)
and (A ∪ B) ∩ (A ∪ C) ⊆ A ∪ (B ∩ C) respectively.
Let’s prove A ∪ (B ∩ C) ⊆ (A ∪ B) ∩ (A ∪ C). First, A ⊆ (A ∪ B) ∩ (A ∪ C) because A ⊆ (A ∪ B)
and A ⊆ (A ∪ C). Second, B ∩ C ⊆ B ⊆ A ∪ B and B ∩ C ⊆ C ⊆ A ∪ C, which together imply
B ∩ C ⊆ (A ∪ B) ∩ (A ∪ C). Therefore, A ∪ (B ∩ C) ⊆ (A ∪ B) ∩ (A ∪ C).
Next, let’s prove (A ∪ B) ∩ (A ∪ C) ⊆ A ∪ (B ∩ C). For any x ∈ (A ∪ B) ∩ (A ∪ C), we have x ∈ A
∪B and x ∈ A ∪ C. There are two possible cases:
• Case 1: x ∈
/ A. Since x ∈ A ∪ B and x ∈ A ∪ C, then we must have x ∈ B and x ∈ C, namely
x ∈ B ∩ C ⊆ A ∪ (B ∩ C).
• Case 2: x ∈ A. Then x ∈ A ∪ (B ∩ C) is trivial.
Thus in either case, we have x ∈ A ∪ (B ∩ C). This means that (A ∪ B) ∩ (A ∪ C) ⊆ A ∪ (B ∩ C).
Therefore,
A ∪ (B ∩ C) = (A ∪ B) ∩ (A ∪ C).

Now we generalize this to multiple sets via induction ( ) in order to prove the following
holds for any integer n ≥ 1. ( )
∪ ∩
n n ( ∪
∩ )
B Ai = B Ai .
i=1 i=1

This equation trivially holds for n = 1. The proof above also shows that this equation holds for
1-4

n = 2. Now assume that it holds for n = m. We prove that it holds for n = m + 1:

( ) (m )
∪ m+1∩ ∪ ∩
B Ai = B Ai ∩ Am+1
i=1 i=1
( ∪ )
= B (∩m i=1 A i ) ∩ (B ∪ Am+1 )
(m )
∩
= (B ∪ Ai ) ∩ (B ∪ Am+1 )
i=1
∩ (
m+1 ∪ )
= B Ai .
i=1

By induction, the equation above holds for any integer n ≥ 1.

We can similarly prove that A ∩ (B ∪ C) = (A ∩ B) ∪ (A ∩ C) and
(n ) n ( ∩
∩ ∪ ∪ )
B Ai = B Ai .
i=1 i=1

Exercise 3. Let A and B be two events in S. Answer the following questions:

1. Are A ∩ B and Ac ∩ B mutually exclusive?

2. Is (A ∩ B) ∪ (Ac ∩ B) = B ?
3. Are A and Ac ∩ B mutually exclusive?
4. Is A ∪ (Ac ∩ B) = A ∩ B ?

Answer:

1. Yes, (A ∩ B) ∩ (Ac ∩ B) = A ∩ Ac ∩ B, however, A ∩ Ac = ∅, so A ∩ Ac ∩ B = ∅.

2. Yes, (A ∩ B) ∪ (Ac ∩ B) = B ∩ (A ∪ Ac ) = B ∩ S = B.
3. Yes, A ∩ Ac ∩ B = ∅ ∩ B = ∅.
4. No, A ∪ (Ac ∩ B) = (A ∪ Ac ) ∩ (A ∪ B) = S ∩ (A ∪ B) = A ∪ B.
 1-5

1.2 Counting and Naive Definition of Probability

1.2.1 Review

Definition 1.5 (Naive Definition of Probability). Let A be an event for an experiment with a finite
sample space S. The naive probability of A is

|A| number of outcomes in A

P (A) = = .
|S| total number of outcomes in S

(We use |A| to denote the size/cardinality of A, i.e., the number of elements in A.)

Definition 1.6 (Permutation). Any ordered sequence of k objects taken without replacement from n
distinct objects is called a permutation of size k of the n objects. The corresponding permutation
number is
n!
n(n − 1) · · · (n − k + 2)(n − k + 1) = , k ≤ n.
(n − k)!
Definition 1.7 (Combination). Given a set of n distinct objects, any unordered subset of size k of the
objects is called a combination. The corresponding combination number is
( )
n n(n − 1) · · · (n − k + 1) n!
= = , k ≤ n.
k k! (n − k)!k!

1.2.2 Exercise

Exercise 4. The College of Science Student Council has one representative from each of the five science
departments (biology, chemistry, statistics, mathematics, physics). In how many ways can

1. Both a council president and a vice president be selected?

2. A president, a vice president, and a secretary be selected?
3. Two members be selected for the Dean’s Council?

Answer:

1. In this case, we select 2 ordered samples without replacement from 5 distinct representatives.
By the definition of permutation, there are 5 × 4 = 20 ways.
2. In this case, we select 3 ordered samples without replacement from 5 distinct representatives.
By the definition of permutation, there are 5 × 4 × 3 = 60 ways.
3. In this case, we select 2 unordered samples without replacement from 5 distinct representatives.
()
By the definition of combination, there are 52 = 5×4
2!
= 10 ways.
1-6

Exercise 5 (Example 1.8 in Lecture Notes). How many ways are there to permute the letters in the
word LALALAAA? How many ways are there to permute the letters in the word STATISTICS?

Answer: To determine a permutation of LALALAAA, we just need to choose where the 5 A’s go (or,
equivalently, just decide where the 3 L’s go). So there are
( ) ( )
8 8 8·7·6
= = = 56 permutations.
5 3 3!

To determine a permutation of STATISTICS, we could choose where to put the S’s, then where to
put the T’s (from the remaining positions), then where to put the I’s, then where to put the A (and
then the C is automatically determined). Alternatively, we can start with 10! and then adjust for
overcounting, dividing by 3!3!2! to account for the fact that the S’s can be permuted among themselves
in any way, and likewise for the T’s and I’s. This gives
( )( )( )( )
10 7 4 2 10!
= = 50400 possibilities.
3 3 2 1 3!3!2!

From this example, we should be aware of the difference between distinguishable and indistinguishable
objects. Order does not matter for indistinguishable objects, so we should adjust for overcounting
properly.

Exercise 6. How many ways are there to choose k items from a set of n objects with replacement,
if order doesn’t matter (we only care about how many times each object was chosen, not the order in
which they were chosen)?

Answer: When order does matter, the answer is nk by the multiplication rule, but this problem where
order does not matter is much harder. We will solve it by solving an equivalent problem.

Let us find the number of ways to put k indistinguishable particles into n distinguishable boxes. That
is, swapping the particles in any way is not considered a separate possibility: all that matters are the
counts for how many particles are in each box. To relate this result back to the original question, we
can let each box correspond to one of the n objects and use the particles as “check marks” to tally
how many times each object is selected. For example, if a certain box contains exactly 3 particles,
that means the object corresponding to that box was chosen exactly 3 times. The particles being
indistinguishable corresponds to the fact that we don’t care about the order in which the objects are
chosen.

Putting k indistinguishable particles into n distinguishable boxes is equivalent to dividing k indistin-

guishable particles into n parts. Each valid configuration of the particles can be encoded as a sequence
of walls and particles in a natural way.
 1-7

1.1: Putting k = 7 indistinguishable particles into n = 4 distinguishable boxes.

A valid sequence must start and end with walls, with n − 1 internal walls and k particles between the
starting and ending walls. We only need to choose where to position the k balls (or n − 1 walls) among
the n − 1 walls and k particles (k + n − 1 available positions in total). So the number of possibilities
( ) (n+k−1)
is n+k−1
k
= n−1 .

Exercise 7. A secretary types four letters to four people and addresses the four envelopes. If he inserts
the letters at random, one into each envelope, what is the probability that exactly two letters will go
into the correct envelopes? Exactly three?

Answer: Each way to insert the letters into the envelopes corresponds to a permutation of 4 objects.
So the total number of outcomes is the number of permutations of size k = 4 of the n = 4 objects,
namely, 4! = 4 ∗ 3 ∗ 2 ∗ 1 = 24. Since the secretary inserts the letters at random, it is reasonable to
assume that each possible outcome is equally likely, so we can use the naive definition of probability.

Let A be the event that exactly two letters will go into the correct envelopes. We can count the number
of outcomes in A via two steps. First, we choose which of the letters go into the correct envelopes.
()
This obviously has 42 = 6 possibilities. Second, for the remaining two letters, we choose to place them
into the wrong envelopes, which obviously has only one possibility. By the multiplication rule, there
are 6 outcomes in A. Thus
6 1
P (A) = = .
24 4

Let B be the event that exactly three letters are correctly placed. If three letters are correctly placed,
then so does the remaining one letter. This means that it is impossible to have exactly three letters
inserted into the correct envelopes. Therefore

P (B) = 0.
1-8

Exercise 8. A box in a certain supply room contains four 40 W lightbulbs, five 60 W bulbs, and six
75 W bulbs. Suppose that three bulbs are randomly selected.

1. What is the probability that exactly two of the selected bulbs are rated 75 W?
2. What is the probability that all three of the selected bulbs have the same rating?
3. What is the probability that one bulb of each type is selected?
4. Suppose now that bulbs are to be selected one by one until a 75 W bulb is found. What is the
probability that it is necessary to examine at least six bulbs?

Answer:

1. The probability that exactly two of the selected bulbs are rated 75 W can be calculated by

P (select exactly two 75 W bulbs) = P (select exactly two 75 W bulbs and 1 other bulb)
(6)(9)
15 × 9
= 2(15)1 = = 0.2967
3
455

2. The probability that all three of the selected bulbs have the same rating can be calculated by

P (all 3 are the same rating) = P (all 3 are 40 W or all 3 are 60 W or all 3 are 75 W)
(4 ) ( 5) (6 )
+ 3 + 3 4 + 10 + 20
= 3 (15 ) = = 0.0747
3
455

3. The probability that one bulb of each type is selected can be calculated by
(4)(5)(6)
120
P (one bulb of each type is selected) = 1 (15 ) =
1 1
= 0.2637
3
455

4. It is necessary to examine at least six bulbs if and only if the first five light bulbs were all of the
40 W or 60 W variety. Since there are 9 such bulbs, the chance of this event is

P (it is necessary to examine at least 6 bulbs)

=P (the first 5 light bulbs were all of the 40 W or 60 W variety)
( 9)
126
= (15
5
)= = 0.0420
5
3003