Chapter 4

Basic Probability
Objectives
 To understand basic probability concepts.

 To understand conditional probability.

 Use Bayes’ theorem to revise probabilities.

 Apply counting rules.

Probability Helps Bridge Descriptive &
Inferential Statistics

 Probability principles are the foundation for:

 Probability distributions.
 Mathematical expectation.
 Binomial and Poisson distributions.
 For example, probability can be used on intent-
to-purchase survey responses and associated
follow-up responses to answer many purchase
behavior questions.
The Sample Space Is The Collection Of
All Possible Outcomes Of A Variable

e.g. All 6 faces of a die:

e.g. All 52 cards of a bridge deck

Each Possible Outcome Of A Variable Is An
Event
 Simple event:
 An event described by a single characteristic.

 e.g., A day in January from all days in 2018.

 Joint event:
 An event described by two or more characteristics.

 e.g. A day in January that is also a Wednesday

from all days in 2018.

 Complement of an event A (denoted A’):
 All events that are not part of event A.

 e.g., All days from 2018 that are not in January.

 Basic Probability Concepts
 Probability – the numerical value representing
the chance, likelihood, or possibility that a certain
event will occur (always between 0 and 1).

 Impossible Event – an event that has no chance

of occurring (probability = 0).

 Certain Event – an event that is sure to occur

(probability = 1).
Mutually Exclusive Events

 Mutually exclusive events:

 Events that cannot occur simultaneously.

Example: Randomly choosing a day from 2018

A = day in January; B = day in February

 Events A and B are mutually exclusive.

Collectively Exhaustive Events

 Collectively exhaustive events:

 One of the events must occur.
 The set of events covers the entire sample space.
Collectively Exhaustive Events

Example: Randomly choose a day from 2018.

A = Weekday; B = Weekend;
C = January; D = Spring;

 Events A, B, C and D are collectively exhaustive

(but not mutually exclusive – a weekday can be in
January or in Spring).
 Events A and B are collectively exhaustive and
also mutually exclusive.
 Three Approaches To Assessing Probability
Of An Event

1. a priori -- based on prior knowledge of the process

X number of ways in which the event occurs
probability of occurrence  
Assuming T total number of possible outcomes
all
outcomes
are
equally
2. empirical probability -- based on observed data
likely number of ways in which the event occurs
probability of occurrence  total number of possible outcomes

3. subjective probability
based on a combination of an individual’s past experience,
personal opinion, and analysis of a particular situation.
 Example of a priori probability

When randomly selecting a day from the year 2018

what is the probability the day is in January?

X number of days in January

Probability of Day In January  
T total number of days in 2018

X 31 days in January 31
 
T 365 days in 2018 365
 Example of empirical probability
Find the probability of selecting a male taking statistics from
the population described in the following table:

Taking Not Total

Stats Taking
Stats
Male 84 145 229
Female 76 134 210
Total 160 279 439

number of males taking stats 84

Probability of male taking stats    0.191
total number of people 439
Subjective Probability Differs From Person
To Person

 What is the probability a new ad campaign is

successful?
 A media development team assigns a 60% probability
of success to its new ad campaign.
 The chief media officer of the company is less
optimistic and assigns a 40% of success to the same
campaign.
Subjective Probability Differs From Person
To Person

 The assignment of a subjective probability is

based on a person’s experiences, opinions, and
analysis of a particular situation.
 Subjective probability is useful in situations when
an empirical or a priori probability cannot be
computed.
Summarizing Sample Spaces

Contingency Table -- M&R Survey Results.

Actually Purchased TV
Planned To Purchase TV Yes No Total
Yes 200 50 250
No 100 650 750
Total 300 700 1,000

Total Number
Of Sample
Space Outcomes.
Summarizing Sample Spaces
Venn Diagram -- M&R Survey Results.

A = Planned to Purchase
A’ = Did not Plan To Purchase
B = Actually Purchased
B’ = Did not Purchase

Actually Purchased TV
Planned To Purchase TV Yes No Total
Yes 200 50 250
No 100 650 750
Total 300 700 1,000
Simple Probability: Definition & Computing
 Simple Probability refers to the probability of a
simple event.
 P(Planned to purchase)
 P(Actually purchased)

number of outcomes satisfying A

P( A ) 
total number of outcomes
Simple Probability: Definition & Computing
number of outcomes satisfying A
P( A ) 
total number of outcomes

P(Purchase) = 250 / 1000

Actually Purchased TV
Planned To Purchase TV Yes No Total
Yes 200 50 250
No 100 650 750
Total 300 700 1,000

P(Actually Purchased) = 300 / 1000

Joint Probability: Definition & Computing

 Joint Probability refers to the probability of an

occurrence of two or more events (joint event).
 ex. P( Plan to Purchase and Purchase ).
 ex. P( No Plan and Purchase ).

number of outcomes satisfying A and B

P( A and B ) 
total number of outcomes
Joint Probability: Definition & Computing
number of outcomes satisfying A and B
P( A and B ) 
total number of outcomes

P(Plan to Purchase and Purchase) = 200 / 1000

Actually Purchased TV
Planned To Purchase TV Yes No Total
Yes 200 50 250
No 100 650 750
Total 300 700 1,000

P(No Plan and Purchase) = 100 / 1000

Computing A Marginal Probability Via Joint
Probabilities
 Computing a marginal (or simple) probability:
P(A)  P(A and B1 )  P(A and B 2 )    P(A and Bk )
 Where B1, B2, …, Bk are k mutually exclusive and
collectively exhaustive events.

P(Planned) = P(Yes and Yes) + P(Yes and No) =

250 / 1000= 200 / 1000 + 50 / 1000

Actually Purchased TV
Planned To Purchase TV Yes No Total
Yes 200 50 250
No 100 650 750
Total 300 700 1,000
Marginal & Joint Probabilities In A
Contingency Table

Event
Event B1 B2 Total
A1 P(A1 and B1) P(A1 and B2) P(A1)
A2 P(A2 and B1) P(A2 and B2) P(A2)

Total P(B1) P(B2) 1

Joint Probabilities Marginal (Simple) Probabilities

 Probability Summary So Far
 Probability is the numerical
measure of the likelihood that an 1 Certain
event will occur.
 The probability of any event must
be between 0 and 1, inclusively.
0 ≤ P(A) ≤ 1 For any event A 0.5
 The sum of the probabilities of all
mutually exclusive and collectively
exhaustive events is 1. 0
Impossible
P(A)  P(B)  P(C)  1
If A, B, and C are mutually exclusive and
collectively exhaustive
General Addition Rule
General Addition Rule

P(A or B) = P(A) + P(B) - P(A and B)

If A and B are mutually exclusive, then

P(A and B) = 0, so the rule can be simplified:

P(A or B) = P(A) + P(B)

For mutually exclusive events A and B
General Addition Rule
Example

P(Planned or Purchased) =
P(Planned) + P(Purchased) – P(Planned and Purchased) =
250 / 1000 + 300 / 1000 – 200 / 1000 = 350 / 1000

Actually Purchased TV
Planned To Purchase TV Yes No Total
Yes 200 50 250
No 100 650 750
Total 300 700 1,000
Computing Conditional Probabilities
 A conditional probability is the probability of one
event, given that another event has occurred:

P(A and B) The conditional

P(A | B)  probability of A given that
P(B) B has occurred.

P(A and B) The conditional

P(B | A)  probability of B given that
P(A) A has occurred.

Where P(A and B) = joint probability of A and B

P(A) = marginal or simple probability of A
P(B) = marginal or simple probability of B
Conditional Probability Example
P(Purchased | Planned)
= P(Purchased and Planned) / P(Planned)
= (200 / 1000) / (250 / 1000) = 200 / 250

Actually Purchased TV
Planned To Purchase TV Yes No Total
Yes 200 50 250
No 100 650 750
Total 300 700 1,000

Since Planned is given we only need to consider

the top row of the table.
A Decision Tree Is An Alternative To A Contingency Table
200
Given Planned e d 250 P(A and B) = 200/1000
r c h as
or Not Planned: Pu
0 0 0 No
1
2 50/ Purc
h ase 50 P(A and B’) = 50/1000
n ned
Pla 250
All Conditional
House- Probabilities
holds 100
Did
Pla Not 750 P(A’ and B) = 100/1000
n e d
750 r c h as
/10
0 Pu
0
No
Purc
h ase 650 P(A’ and B’) = 650/1000
750
 Actually Purchased TV
Planned To Purchase TV Yes No Total
Yes 200 50 250
No 100 650 750
Total 300 700 1,000
Independent Events

 Two events are independent if and only if:

P(A | B)  P(A)

 Events A and B are independent when the

probability of one event is not affected by the fact
that the other event has occurred.
The difference between mutually exclusive and
independent events is:
• Mutually exclusive event can simply be
defined as a situation when two events
cannot occur at same time. 
• Independent event occurs when one event
remains unaffected by the occurrence of
the other event.
Example

Does P(Purchased | Planned) = P(Purchased)?

P(Purchased | Planned) = 200 / 250 = 0.8.

P(Purchased) = 700 / 1000 = 0.7.

Since these two probabilities are not equal, these

two events are dependent.

Actually Purchased TV
Planned To Purchase TV Yes No Total
Yes 200 50 250
No 100 650 750
Total 300 700 1,000
 Multiplication Rules For Two Events
The General Multiplication Rule

P(A|B)  P(A and B) / P(B)

Solving for P(A and B)

P(A and B)= P(A|B) P(B)

Note: If A and B are independent, then P(A | B)  P(A)

and the multiplication rule simplifies to:

P(A and B)  P(A) P(B)

Marginal Probability Using The General
Multiplication Rule
 Marginal probability for event A
P(A)  P(A | B1 ) P(B1 )  P(A | B 2 ) P(B 2 )    P(A | Bk ) P(Bk )

 Where B1, B2, …, Bk are k mutually exclusive and

collectively exhaustive events.
Let A = Planned, B1 = Purchase, & B2 = No Purchase

P(A) = P(A|B1)P(B1) + P(A|B2)P(B2) =

(200/300)(300/1000) + (50/700)(700/1000) = 0.25
Actually Purchased TV
Planned To Purchase TV Yes No Total
Yes 200 50 250
No 100 650 750
Total 300 700 1,000
 Ethical Issues & Probability
 Ethical issues can arise when any statements
related to probability are presented to the public.
 Unintended misinterpretations can occur with
people who are not comfortable with numerical
concepts.
 Advertising quoting probabilities can also be
intentionally misleading.
Bayes’ Theorem
Bayes’ Theorem
 Bayes’ Theorem is used to revise previously
calculated probabilities based on new
information.
 Developed by Thomas Bayes in the 18th Century.
 It is an extension of conditional probability.
• Bayes' theorem describes the probability of an
event, based on prior knowledge of conditions
that might be related to the event.
• For example, if the risk of developing health
problems is known to increase with age, Bayes'
theorem allows the risk to an individual of a
known age to be assessed more accurately by
conditioning it relative to their age, rather than
simply assuming that the individual is typical of
the population as a whole.
Bayes’ Theorem

P(A | B i )P(B i )
P(B i | A) 
P(A | B 1 )P(B 1 )  P(A | B 2 )P(B 2 )      P(A | B k )P(B k )

where:
Bi = ith event of k mutually exclusive and collectively
exhaustive events
A = new event that might impact P(Bi)
 Bayes’ Theorem Example
 A drilling company has estimated a 40% chance of
striking oil for their new well.
 A detailed test has been scheduled for more
information. Historically, 60% of successful wells
have had detailed tests, and 20% of unsuccessful
wells have had detailed tests.
 Given that this well has been scheduled for a
detailed test, what is the probability that the well
will be successful?
Bayes’ Theorem Example (continued)

 Let S = successful well

U = unsuccessful well
 P(S) = 0.4 , P(U) = 0.6 (prior probabilities)
 Define the detailed test event as D
 Conditional probabilities:
P(D|S) = 0.6 P(D|U) = 0.2
 Goal is to find P(S|D)
 Bayes’ Theorem Example (continued)

Apply Bayes’ Theorem:

P(D | S)P(S)
P(S | D) 
P(D | S)P(S)  P(D | U)P(U)
(0.6)(0.4)

(0.6)(0.4)  (0.2)(0.6)
0.24
  0.667
0.24  0.12

So the revised probability of success, given that this well

has been scheduled for a detailed test, is 0.667
 Bayes’ Theorem Example
(continued)

 Given the detailed test, the revised probability

of a successful well has risen to 0.667 from
the original estimate of 0.4

Prior Conditional Joint Revised

Event Prob.
Prob. Prob. Prob.
S (successful) 0.4 0.6 (0.4)(0.6) = 0.24 0.24/0.36 = 0.667
U (unsuccessful) 0.6 0.2 (0.6)(0.2) = 0.12 0.12/0.36 = 0.333

Sum = 0.36
Bayes' Theorem | Hate it or Love it, can't ignore it!

Bayes theorem, the geometry of changing beliefs

Counting Rules
Counting Rules Are Often Useful In
Computing Probabilities

 In many cases, there are a large number of

possible outcomes.

 Counting rules can be used in these cases to

help compute probabilities.
 Counting Rules

 Rules for counting the number of possible

outcomes
 Counting Rule 1:
 If any one of k different mutually exclusive and
collectively exhaustive events can occur on
each of n trials, the number of possible
outcomes is equal to

k n

 Example
 If you roll a fair die 3 times then there are 63 =
216 possible outcomes
 Counting Rule 2:
 If there are k events on the first trial, k events
1 2
on the second trial, … and kn events on the nth
trial, the number of possible outcomes is
(k1)(k2)…(kn)
 Example:
 You want to go to a park, eat at a restaurant, and
see a movie. There are 3 parks, 4 restaurants, and
6 movie choices. How many different possible
combinations are there?
 Answer: (3)(4)(6) = 72 different possibilities
 Counting Rule 3:
 The number of ways that n items can be

arranged in order is
n! = (n)(n – 1)…(1)

 Example:
 You have 5 books to put on a bookshelf.

How many different ways can these books be

placed on the shelf?

 Answer: 5! = (5)(4)(3)(2)(1) = 120 different

possibilities.
N! ：  n factorial; factorial n

https://www.youtube.com/watch?
v=oT64b5euTfs&ab_channel=MathandScience
 Counting Rule 4:
 Permutations: The number of ways of arranging X

objects selected from n objects in order is

n!
n Px 
(n  X)!
 Example:
 You have 5 books and are going to put 3 on a

bookshelf. How many different ways can the

books be ordered on the bookshelf?
n! 5! 120
n Px     60
 Answer:
(n  X)! (5  3)! 2
Permutations: an ordering of events when
the order matters

nPr
No. of permutations of “n” elements taken “r”
at a time

nPn=n!
 n!
n Px 
 Counting Rule 5:
(n  X)!
 Combinations: The number of ways of selecting
X objects from n objects, irrespective of order, is

n!
C
n x 
X!(n  X)!
Counting Rules (continued)
 Example:
 You have 5 books and are going to select three

are to read. How many different combinations

are there, ignoring the order in which they are
selected?

 Answer:
n! 5! 120
n Cx     10
X!(n  X)! 3! (5  3)! (6)(2)
different possibilities
Chapter Summary

1. Using basic probability concepts.

2. Using conditional probability.

3. Using Bayes’ theorem to revise probabilities.
4. Using counting rules.