Sample Spaces, Events and Set Operations

• Any action whose outcome is random and results in well defined out-
comes is an experiment.
• The set of all possible outcomes of an experiment is called the sample
space of the experiment and is denoted by S.

Examples
i. Tossing a fair coin 3 times and recording heads or tails for each.

ii. Tossing a fair coin 3 times and recording the number of heads.

iii. Tossing a fair coin and recording the number of tosses until the third
head occurs.

Example: Four WashU students are selected and asked their opinion on the
university’s handling of covid-19 (on a 1-10 scale).

Only the average of the four responses is recorded.

• An event is a subset of the sample space.

Example: One student’s opinion on WashU’s handling of covid-19 is
recorded.

1
 • An event that contains only one outcome is a simple event.
• An event that contains more than one outcome is a compound event.
• S is an event, and ∅ = empty set is an event.
• We say event A has occurred if the outcome of the experiment is in A.
We can use set operations on events to create additional events. Let A and
B be events.
• The union of A and B, A ∪ B, is the event containing all outcomes in
A or B.

• The intersection of A and B, A ∩ B, is the event containing all outcomes

in both A and B.

• The complement of A, Ac , is the event containing all outcomes not in A.

• The difference A − B is the event containing outcomes that are in A

that are not in B.

2
 • A and B are disjoint or mutually exclusive if they have no outcomes in
common.

• A is a subset of B if all outcomes in A are also in B. This is denoted

A ⊆ B.

Note: Unions and intersections can be extended to more than two sets.
i. A1 ∪ A2 ∪ · · · ∪ Ak is the event containing outcomes that are in A1 or A2
or · · · or Ak .

ii. A1 ∩ A2 ∩ · · · ∩ Ak is the event containing outcomes that are in A1 and

A2 and · · · and Ak .

Example: Resistors manufactured by a machine producing 1kΩ resistors are

unacceptable if they are not within 50Ω of the nominal value. Four such
resistors are tested.
a. What is the sample space?

b. Let Ei be the event that the ith resistor tests acceptable. Describe E2 .

3
 c. Are E1 and E2 mutually exclusive?

d. Find E1 ∪ E2 ∪ E3 ∪ E4 .

e. Find E1 ∩ E2 ∩ E3 ∩ E4 .

Set operations
i. Commutative Laws:

A ∪ B = B ∪ A and A ∩ B = B ∩ A

ii. Associative Laws:

(A ∪ B) ∪ C = A ∪ (B ∪ C) and (A ∩ B) ∩ C = A ∩ (B ∩ C)

iii. Distributive Laws:

(A ∪ B) ∩ C = (A ∩ C) ∪ (B ∩ C) and (A ∩ B) ∪ C = (A ∪ C) ∩ (B ∪ C)

iv. DeMorgan’s Laws:

(A ∪ B)c = Ac ∩ B c and (A ∩ B)c = Ac ∪ B c

4
 Equally Likely Outcomes

• The probability of an event E is the likelihood of the occurrence of E.

This is denoted P (E).

• We will start by talking about experiments with a finite number of

equally likely outcomes.
• Let S be a sample space with N outcomes that are equally likely to
occur. Then the probability of each outcome is 1/N .
• If N (E) denotes the number of outcomes in the event E, then
N (E)
P (E) =
N
Example: Suppose we roll 2 dice separately. Find the probability that the
sum of the two sides is 6.

Counting Techniques

The Multiplication Rule: If a task can be completed in k stages and stage i

has ni outcomes, regardless of the outcomes of the previous stages, then the
task has n1 n2 · · · nk outcomes.

Examples:
i. Suppose a coin is tossed 10 times.

ii. Suppose car license plates have two letters followed by a digit, a letter,
a digit, and a letter.

5
 iii. Select a first, second, and third place winner from a group of four final-
ists.

Permutations and Combinations: If the k stages of a task involve sampling

one unit each, without replacement, from the same group of n units, then:
1. If a distinction is made between the outcomes of the stages, we say the
outcomes are ordered. Otherwise we say the outcomes are unordered.
2. The ordered outcomes are called permutations of k units. The number
of permutations of k units selected from a group of n units is denoted
by Pk,n .
3. The unordered outcomes are called combinations of k units. The number
of combinations of k units selected from a group of n units is denoted
n


by k .

• To compute the number of permutations of k units:

n!
Pk,n = n(n − 1)(n − 2) · · · (n − k + 1) =
(n − k)!

• Note: m! = m(m − 1) · · · (2)(1) and 0! = 1.

• The number of permutations of all n units is

Pn,n =

• To compute the number of combinations of k units

 
n Pk,n n!
= =
k Pk,k k!(n − k)!

6
Example: Suppose we select two cards from a deck of 52 cards.
a. How many outcomes are there if the first card will be given to player 1
and the second card will be given to player 2?

b. How many outcomes are there if both cards will be given to player 1?

Example: In a game of poker each player receives five cards dealt from a deck
of 52 cards. A full house refers to a five-card hand consisting of three of a
kind and two of a kind. Find the probability that a randomly dealt five-card
hand is a full house.

We can generalize the idea of combinations:

The number of arrangements of n units into r groups of sizes n1 , n2 , . . . , nr is

 
n n!
=
n1 , n2 , . . . , nr n1 !n2 ! · · · nr !

7
Example: How many ways are there to form groups of sizes 2, 3, and 3 from
8 students?

Example: A communication system consists of 13 antennas arranged in a

line. The system functions as long as no two nonfunctioning antennas are
next to each other. Suppose 5 antennas stop functioning.
a. How many different arrangements of the 5 nonfunctioning antennas re-
sult in the system being functional?

b. If the arrangement of the 5 nonfunctioning antennas is equally likely,

what is the probability the system is functioning?

Probability Mass Function (PMF)

We often start with an experiment with equally likely outcomes, but record
the value of a random variable X.

The outcomes in the sample space for X may not be equally likely.

The probability mass function of a discrete random variable X is a list of

the probabilities p(x) for each value x in the sample space SX of X.

8
Example: A simple random sample of size n = 3 is drawn from a batch of 10
product items. If 4 of the 10 items are defective, find the PMF of the random
variable X = the number of defective items in the sample.
x 0 1 2 3
p(x)

Once we know the PMF we can simulate the experiment using R.

Example: Simulate 5 replications of the above experiment.

> set.seed(3200)
> sample(0:3, size = 5, replace = TRUE, prob = c(0.167, 0.5, 0.3, 0.033))
[1] 1 0 1 2 2

If we simulate 10000 replications:

> x = sample(0:3, size = 10000, replace = TRUE, prob = c(0.167, 0.5, 0.3, 0.033))
> table(x)/10000
x
0 1 2 3
0.1640 0.4977 0.3048 0.0335

9
 Axioms and Properties of Probability

• Previously, we looked at probability for experiments with equally likely

outcomes.
• Now, we consider the more general situation.

Axioms of Probability: For an experiment with sample space S, probability

is a function that assigns a number P (E) to an event E so that the following
axioms hold:
i. 0 ≤ P (E) ≤ 1
ii. P (S) = 1
iii. For any sequence of disjoint events E1 , E2 , . . .
∞ ∞
!
[ X
P Ei = P (Ei )
i=1 i=1

Properties of Probability
• P (∅) = 0 where ∅ is the empty set.
• For any finite collection, E1 , E2 , . . . , Em , of disjoint events

P (E1 ∪ E2 ∪ · · · ∪ Em ) = P (E1 ) + P (E2 ) + · · · + P (Em )

• If A ⊆ B then P (A) ≤ P (B).

• P (Ac ) = 1 − P (A)

Note: Suppose the event E contains the outcomes s1 , s2 , . . .

10
Example: Suppose two dice are rolled. What is the probability the sum is
greater than 2, but less than 5?

Additional Properties: For any events A, B, and C,

• P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

• P (A ∪ B ∪ C) = P (A) + P (B) + P (C) − P (A ∩ B) − P (A ∩ C)

−P (B ∩ C) + P (A ∩ B ∩ C)

Example: In a certain community 60% of families own a dog, 70% own a cat,
and 50% own both a dog and a cat.
• If a family is selected at random, what is the probability the family owns
at least one of the two kinds of pets.

• If a family is selected at random, what is the probability the family owns

a dog but not a cat.

11
 Conditional Probability

Example: An urn contains 5 one-dollar bills and 1 ten-dollar bill. Suppose

two bills are drawn, without replacement.
a. What is the probability the second bill is a ten-dollar bill?

b. Suppose we know the first bill drawn is a ten-dollar bill. What is the
probability the second bill is a ten-dollar bill?

c. Why are these probabilities different?

For any two events A and B with P (A) > 0, the conditional probability of B
given A, denoted by P (B | A), is
P (A ∩ B)
P (B | A) =
P (A)

Note:

Example: Two dice are rolled and their sum is observed to be 7. Given
this information, what is the conditional probability that one of the two die
rolls was a 3?

12
Example: If a randomly selected family owns a dog, what is the probability
that it also owns a cat?

Example: Let A, B, and C be three disjoint events with P (A) = 0.2,

P (B) = 0.3, and P (C) = 0.5. Find P ([A ∪ B c ] | [B ∪ C]).

Properties
• 0 ≤ P (B | A) ≤ 1
• P (A ∪ C | B) = P (A | B) + P (C | B) − P (A ∩ C | B)

Multiplication Rule:
• P (A ∩ B) = P (A)P (B | A)

• P (A ∩ B ∩ C) = P (A)P (B | A)P (C | A ∩ B)

13
Example: Two consecutive traffic lights have been synchronized to make a
run of green lights more likely. In particular, if a driver finds the first light
to be red, the second light will be green with probability 0.9, and if the first
light is green the second will be green with probability 0.7. If the probability
of finding the first light green is 0.6, find the probability that a driver will
find both lights green.

A tree diagram:

Law of Total Probability: a formula for computing the probability of an event

B when B arises in connection with a partition of the sample space.
• P (B) = P (A)P (B | A) + P (Ac )P (B | Ac )

• If A1 , . . . , Ak constitute a partition of the sample space

P (B) = P (A1 )P (B | A1 ) + · · · + P (Ak )P (B | Ak )

14
Example: Find the probability that a driver will find the second traffic light
green.

Bayes’ Theorem

Bayes’ Theorem is often used in the same context as the law of total proba-
bility.
P (A)P (B | A)
P (A | B) =
P (A)P (B | A) + P (Ac )P (B | Ac )

Example: Joe is a randomly chosen member of a large population in which

3% are heroin users. A randomly given drug test correctly identifies users
95% of the time and correctly identifies nonusers 90% of the time. If Joe
tests positive, what is the probability that he is a heroin user?

Bayes’ Theorem can also be extended to the case where A1 , . . . , Ak are a

partition of the sample space.
P (Aj )P (B | Aj )
P (Aj | B) =
P (A1 )P (B | A1 ) + · · · + P (Ak )P (B | Ak )

15
 Independent Events

• Events A and B are independent if knowledge that A occurred does not

change the probability of B occurring (and vice versa).
• Equivalent mathematical definitions:
i. P (A ∩ B) = P (A)P (B)
ii. P (B | A) = P (B)
iii. P (A | B) = P (A)

• If A and B are not independent, we say they are dependent.

• Independent events can arise from independent experiments or indepen-
dent repetitions of the same experiment.

Example: A card is drawn at random from a deck of 52 cards. Let A and

B be the events that the card is a five and the card is a spade, respectively.
Are the events A and B independent?

Properties:
1. If A and B are independent so are A and B c (and so are Ac and B).
2. S and ∅ are independent of every other event.
3. Disjoint events are not independent unless the probability of one of them
is 0.

16
Example: The proportion of female voters who support the exploration of all
alternative forms of energy production is the same as the proportion of all
voters who support the exploration of all alternative forms of energy produc-
tion. For a randomly selected voter, let F denote the event that the voter is
female, and let E denote the event that the voter supports the exploration
of all alternative forms of energy production.
a. Are the events E and F independent?

b. Is the proportion of male voters who support the exploration of all al-
ternative forms of energy production the same as the corresponding pro-
portion female voters?

Example: Suppose we draw one card from a deck of 52 cards. Let F be the
event that a face card is drawn. Let A be the event that an ace is drawn.
Are A and F independent?

Mutual Independence

E1 , . . . , En are mutually independent if for every subset Ei1 , . . . , Eik , k ≤ n,

P (Ei1 ∩ · · · ∩ Eik ) = P (Ei1 ) · · · P (Eik )

If E1 , . . . , En are mutually independent, so are their complements.

17
Example: 20% of a certain type of laser diodes have efficiency below 0.3
mW/mA. For 5 diodes, selected by random sampling from a large population
of such diodes, find the probability of the following events.
a. All five have efficiency above 0.3.

b. Only the second diode selected has efficiency below 0.3.

18