Contents

Contents 1

3 Chapter 3 Probability 2
3.1 Events, Sample Spaces, and Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
3.2 Unions and Intersections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.3 Complementary Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.4 The Additive Rule and Mutually Exclusive Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.5 Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.6 The Multiplicative Rule and Independent Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.7 Some Additional Counting Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.8 Bayes’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1
Chapter 3

Chapter 3 Probability

In this chapter, we assume that the population is known and calculate the chances of obtaining various samples from the
population. Thus, we show that probability is the reverse of statistics: In probability, we use information about the population
to infer the probable nature of the sample.

3.1 Events, Sample Spaces, and Probability

Suppose a coin is tossed once and the up face is recorded. The result we see is called an observation, or measurement, and

the process of making an observation is called an experiment.

An experiment is an act or process of observation that leads to a single outcome that cannot be predicted with certainty.

A sample point is the most basic outcome of an experiment.

Question

What are all possible outcomes of:

1- Tossing a die and observing the number on the up face?

2- Tossing two coins and observing their up faces?

3- List all the sample points for each experiment.

The sample space of an experiment is the collection of all its sample points which can be represented in set notation as

a set containing sample points or a pictorial method.

The following tree diagram shows all possible outcomes for tossing two coins:

Figure 3.1: Tree Diagram for the Coin-Tossing Experiment

2
CHAPTER 3. CHAPTER 3 PROBABILITY 3

Venn diagram is a graphical representation for a sample space as a closed figure, labeled S, containing all possible

sample points.

Figure 3.2: Venn diagrams for the Three Experiments from Table 3-1

Result from the law of large numbers

The probability of a sample point is a number between 0 and 1 that measures the likelihood that the outcome will occur

when the experiment is performed. This number is usually taken to be the relative frequency of the occurrence of a sample

point in a very long series of repetitions of an experiment.

For example, consider tossing a balanced coin a very large number of times. The following figure supports our reasoning.

Figure 3.3: Proportion of heads in N tosses of a coin

Probability Rules for Sample Points

Let pi represent the probability of sample point i. Then

1. All sample point probabilities must lie between 0 and 1 (i.e., 0 ≤ pi ≤ 1)

2. The probabilities of all the sample points within a sample space must sum to 1
P
(i.e., pi = 1)

Event

An event is a specific collection of sample points.

Probability of an Event

The probability of an event A is calculated by summing the probabilities of the sample points in the sample space for A.
CHAPTER 3. CHAPTER 3 PROBABILITY 4

Steps for Calculating Probabilities of Events

1. Define the experiment; that is, describe the process used to make an observation and the type of observation that will

be recorded.

2. List the sample points.

3. Assign probabilities to the sample points.

4. Determine the collection of sample points contained in the event of interest.

5. Sum the sample point probabilities to get the probability of the event.

Example 3.1.1 A fair die is tossed, and the up face is observed. If the face is even, you win $1. Otherwise, you lose $1.

What is the probability that you win?

On the basis of this notion of probability, in the long run, you will win $1 half the time and lose $1 half the time.

Recall that the sample space for this experiment contains six sample points:

S : {1, 2, 3, 4, 5, 6}

1
Since the die is balanced, we assign a probability of 6 to each of the sample points in this sample space.

We define an event A, a collection of sample points which observe a 2, a 4, or a 6.

A : {2, 4, 6}

Since the event A contains three sample points— all with probability 61 —and since no sample points can occur simulta-

neously, we reason that the probability of A is the sum of the probabilities of the sample points in A. Thus, the probability
1 1 1
of A is 6 + 6 + 6 = 21 .

Figure 3.4: Die-toss experiment with event A, Observe an even number

CHAPTER 3. CHAPTER 3 PROBABILITY 5

Example 3.1.2 Consider the experiment of tossing two unbalanced coins. Because the coins are not balanced, their outcomes

(H or T) are not equiprobable. Suppose the correct probabilities associated with the sample points are given in the accompanying

table. [Note: The necessary properties for assigning probabilities to sample points are satisfied.] Consider the events

A : {Observe exactly one head.} B : {Observe at least one head.}

or:

A : {HT, TH} B : {HT, TH, HH}

Calculate the probability of A and the probability of B.

Sample Point Probability

4
HH 9
2
HT 9
2
TH 9
1
TT 9
Total 1

2 2 4
p(A) = p(Observe HT) + p(Observe TH) = + =
9 9 9

Similarly,

4 2 2 8
p(B) = + + =
9 9 9 9

When the sample points run into the thousands or millions, one method of determining the number of sample points for

a complex experiment is to develop a counting system. Start by examining a simple version of the experiment. A second

method of determining the number of sample points for an experiment is to use combinatorial mathematics.

Combinations Rule

Suppose a sample of n elements is to be drawn without replacenent from a set of N elements. Then the number of

different samples possible is denoted by N

n and is equal to

N N!


n = n!(N −n)!
Where: n! = n × (n − 1) × (n − 2) × ...3 × 2 × 1 (n factorial) and 0! = 1.

Question

How many possible simple random samples of size 3 (without replacement), can we have from a population of size 6?
CHAPTER 3. CHAPTER 3 PROBABILITY 6

3.2 Unions and Intersections

An event can often be viewed as a composition of two or more other events. Such events, which are called compound events,

can be formed (composed) in two ways.

The union of two events A and B is the event that occurs if either A or B (or both) occurs on a single performance of

the experiment. We denote the union of events A and B by the symbol A ∪ B. A ∪ B consists of all the sample points that

belong to A or B or both. ( See Figure 3.5a.)

The intersection of two events A and B is the event that occurs if both A and B occur on a single performance of the

experiment. We write A ∩ B for the intersection of A and B. A ∩ B consists of all the sample points belonging to both A and

B. (See Figure 3.5b.)

Figure 3.5: Venn diagrams for union and intersection

Question

Family Planning Perspectives reported on a study of over 200,000 births in New Jersey over a recent two-year period.

The study investigated the link between the mother’s race and the age at which she gave birth (called maternal age). The

percentages of the total number of births in New Jersey, by the maternal age and race classifications, are given in the following

table.

Tow-Waye Table: Percentage of New Jersey Birth Mothers, by Age and Race

Race
Maternal White Black
Age (years)
≤ 17 2% 2%
18-19 3% 2%
20-29 41% 12%
≥ 30 33% 5%

Consider the following event:

A : {A New Jersey birth mother is white.} B : {A New Jersey mother was a teenager when giving birth.}

Find P (A), P (B), P (A ∪ B), and P (A ∩ B).

CHAPTER 3. CHAPTER 3 PROBABILITY 7

3.3 Complementary Events

complementary events:

The complement of an event A is the event that A does not occur—that is, the event consisting of all sample points that

are not in event A. We denote the complement of A by Ac .

Figure 3.6: Venn diagram of complementary events

Note from the figure that all sample points in S are included in either A or Ac and that no sample point is in both A and

Ac .

Rule of Complements

The sum of the probabilities of complementary events equals 1; that is,

P (A) + P (Ac ) = 1

Example 3.3.1 Consider the experiment of tossing fair coins. Define the following event:

A : {Observing at least one head.}

a. Find P (A) if 2 coins are tossed.

b. Find P (A) if 10 coins are tossed.

a. When 2 coins are tossed, we know that the event A : {Observing at least one head.} consists of the sample points

A : {HH, HT, TH}

then

1 1 1 3
p(A) = + + =
4 4 4 4

b. When 10 coins are tossed, we know that the event Ac is:

CHAPTER 3. CHAPTER 3 PROBABILITY 8

1
Ac : {TTTTTTTTTT} and P (Ac ) = 1024 (why?)

Now we use the relationship of complementary events to find P (A):

1 1023
P (A) = 1 − P (Ac ) = 1 − 1024 = 1024 = 0.999

3.4 The Additive Rule and Mutually Exclusive Events

Additive Rule of Probability

The probability of the union of events A and B is the sum of the probability of event A and the probability of event B,

minus the probability of the intersection of events A and B; that is,

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

Example 3.4.1 Hospital records show that 12% of all patients are admitted for surgical treatment, 16% are admitted for

obstetrics, and 2% receive both obstetrics and surgical treatment. If a new patient is admitted to the hospital, what is the

probability that the patient will be admitted for surgery, for obstetrics, or for both?

Consider the following events:

A : {A patient admitted to the hospital receives surgical treatment.}

B : {A patient admitted to the hospital receives obstetrics treatment.}

Then, from the given information,

P (A) = 0.12 and P (B) = 0.16

and the probability of the event that a patient receives both obstetrics and surgical treatment is: P (A ∩ B) = 0.02

The event that a patient admitted to the hospital receives either surgical treatment, obstetrics treatment, or both is the

union (A ∪ B), the probability of which is given by the additive rule of probability:

P (A ∪ B) = P (A) + P (B) − P (A ∩ B)

= 0.12 + 0.16 − 0.02 = 0.26

CHAPTER 3. CHAPTER 3 PROBABILITY 9

Thus, 26% of all patients admitted to the hospital receive either surgical treatment, ob- stetrics treatment, or both.

Mutually Exclusive Events

Events A and B are mutually exclusive if A ∩ B contains no sample points—that is, if A and B have no sample points in

common (i.e. cannot occur simultaneously). For mutually exclusive events,

P (A ∩ B) = 0

Figure 3.7: Venn diagram of mutually exclusive events

Probability of Union of Two Mutually Exclusive Events

If two events A and B are mutually exclusive, the probability of the union of A and B equals the sum of the probability

of A and the probability of B; that is,

P (A ∪ B) = P (A) + P (B)

3.5 Conditional Probability

Often, we have additional knowledge that might affect the outcome of an experiment, so we may need to alter the probability

of an event of interest. A probability that reflects such additional knowledge is called the conditional probability of the event.

Question

Consider a family that have 3 children. calculate the probability of the following events:

a. All of the children are girls.

b. All of the children are boys, but we are given the information that the third children is a boy.

Conditional Probability Formula

To find the conditional probability that event A occurs given that event B occurs, divide the probability that both A and

B occur by the probability that B occurs; that is,

P (A ∩ B
P (A|B) =
P (B)

When P (B) 6= 0.
CHAPTER 3. CHAPTER 3 PROBABILITY 10

Example 3.5.1 Many medical researchers have conducted experiments to examine the relationship between cigarette smoking

and cancer. Consider an individual randomly selected from the adult male population. Let A represent the event that the

individual smokes, and let Ac denote the complement of A (the event that the individual does not smoke). Similarly, let B

represent the event that the individual develops cancer, and let B c be the complement of that event. Then the four sample

points associated with the experiment are shown in Figure 3.6, and their probabilities for a certain section of the United States

are given in following table. Use these sample point probabilities to examine the relationship between smoking and cancer.

Probabilities of Smoking and Developing Cancer

Develops Cancer
Smoker Yes, B No, B c
Yes, A 0.05 0.2
No, Ac 0.03 0.72

Figure 3.8: Sample space for Example 3.5

Note

The example above is an observational study. Observational studies can be used to identify risk facotrs not for claiming

causation.

3.6 The Multiplicative Rule and Independent Events

Multiplicative Rule of Probability

P (A ∩ B) = P (A)P (B|A) ≡ P (A ∩ B) = P (B)P (A|B)

Example 3.6.1 Ambulance response time. Geographical Analysis (Jan. 2010) presented a study of Emergency Medical

Services (EMS) ability to meet the demand for an ambulance. In one example, the researchers presented the following

scenario. An ambulance station has one vehicle and two demand locations, A and B. The probability that the ambulance can

travel to a location in under eight minutes is 0.58 for location A and 0.42 for location B. The probability that the ambulance

is busy at any point in time is 0.3.

CHAPTER 3. CHAPTER 3 PROBABILITY 11

a. Find the probability that EMS can meet demand for an ambulance at location A.

b. Find the probability that EMS can meet demand for an ambulance at location B.

Independent Events

Events A and B are independent events if the occurrence of B does not alter the probability that A has occurred; that is,

events A and B are independent if

P (A|B) = P (A)

When events A and B are independent, it is also true that

P (B|A) = P (B)

Events that are not independent are said to be dependent.

Probability of Intersection of Two Independent Events

If events A and B are independent, then the probability of the intersection of A and B equals the product of the

probabilities of A and B; that is,

P (A ∩ B) = P (A)P (B)

The converse is also true: If P (A ∩ B) = P (A)P (B), then events A and B are independent.

Example 3.6.2 Consider the experiment of tossing a fair die, and let

A = {Observe an odd number.} B = {Observe a number less than or equal to 5.}

Are A and B independent events?

Mutually Exclusive Events

Mutually exclusive events are dependent events and can not occur simultaneously.

P (A|B) = P (B|A) = P (A ∩ B) = 0

3.7 Some Additional Counting Rules

The Multiplicative Rule

You have k sets of elements, n1 in the first set, n2 in the second set, ..., and nk in the kth set. Suppose you wish to form

a sample of k elements by taking one element from each of the k sets. Then the number of different samples that can be

formed is the product

CHAPTER 3. CHAPTER 3 PROBABILITY 12

Figure 3.9: Mutually exclusive events are dependent events

n1n2n3...nk
Permutations Rule

Given a single set of N different elements, you wish to select n elements from the N and arrange them within n positions.

The number of different permutations of the N elements taken n at a time is denoted by PnN and is equal to

N!
PnN = (N −n)!
Partitions Rule

Suppose you wish to partition a single set of N different elements into k sets, with the first set containing n1 elements,

the second containing n2 elements, ..., and the kth set containing nk elements. Then the number of different partitions is

N!
n1 !n2 !n3 !...nk !
Where n1 + n2 + n3 + ... + nk = N .

Exercises :

1- How many possible 3 digit numbers are there?

2- Suppose we have 3 positions and 7 candidates. How many ways we can fill these positions? (each position can be filled

with one candidate)

3- Selecting project teams. Suppose you are managing 10 employees and you need to form three teams to work on different

projects. Assume that each employee may serve on any team. In how many different ways can the teams be formed if the

number of members on each project team are 3, 3, and 4.

3.8 Bayes’s Rule

Bayes’s Rule
CHAPTER 3. CHAPTER 3 PROBABILITY 13

Given k mutually exclusive and exhaustive events, B1 , B2 , ..., Bk such that P (B1 ) + P (B2 ) + P (B3 ) + ... + P (Bk ) = 1

and, given an observed event A, it follows that

P (Bi ∩ A)
P (Bi |A) =
P (A)

P (Bi )P (A|Bi )
=
P (B1 )P (A|B1 ) + P (B2 )P (A|B2 ) + ... + P (Bk )P (A|Bk )

Example 3.8.1 An unmanned monitoring system uses high-tech video equipment and microprocessors to detect intruders.

A prototype system has been developed and is in use outdoors at a weapons munitions plant. The system is designed to detect

intruders with a probability of 0.9. However, the design engineers expect this probability to vary with the weather. The system

automatically records the weather condition each time an intruder is detected. On the basis of a series of controlled tests in

which an intruder was released at the plant under various weather conditions, the following information is available: Given

that the intruder was, in fact, detected by the system, the weather was clear 75% of the time, cloudy 20% of the time, and

raining 5% of the time. When the system failed to detect the intruder, 60% of the days were clear, 30% cloudy, and 10%

rainy. Use this information to find the probability of detecting an intruder, given rainy weather. (Assume that an intruder

has been released at the plant.) Could you find the probability of detecting an intruder, given cloudy weather?

First we extract the information using the following event.

D = {The intruder is detected by the system.} then:

P (D) = 0.9 P (Dc ) = 0.1

P (Clear|D) = 0.75 P (Clear|Dc ) = 0.6

P (Rainy|D) = 0.05 P (Rainy|Dc ) = 0.1

P (Cloudy|D) = 0.2 P (Coudy|Dc ) = 0.3

By using Bayes’s Rule we have:

P (D)P (Rainy|D)
P (D|Rainy) =
P (D)P (Rainy|D) + P (Dc )P (Rainy|Dc )

0.05 × 0.9
= = 0.818
0.05 × 0.9 + 0.1 × 0.1

The following graph is called tree diagram:

CHAPTER 3. CHAPTER 3 PROBABILITY 14

Question

HIV testing and false positives. Bayes’s rule was applied to the problem of HIV testing in The American Statistician

(Aug. 2008). In North America, the probability of a person having HIV is .008. A test for HIV yields either a positive or

negative result. Given that a person has HIV, the probability of a positive test result is .99. (This probability is called the

sensitivity of the test.) Given that a person does not have HIV, the probability of a negative test result is also .99. (This

probability is called the specificity of the test.) The authors of the article are interested in the probability that a person

actually has HIV given that the test is positive. Find the probability of interest for a North American by using Bayes’s rule.

The core content of the slides are from the textbook of this course;

STATISTICS (13th Edition)

by

JAMES McCLAVE and TERRY SINCICH