HO CHI MINH CITY OPEN UNIVERSITY

ADVANCED STUDY PROGRAM

T.S: Nguyễn Thị Thanh Thủy

 CHAPTER OUTLINE

I Introduction

II Organizing and visualizing events

III Joint and marginal probabilities

IV Probabilities summary
I. INTRODUCTION
Basic probability concepts
⚫ Probability – the chance that an uncertain event
will occur (always between 0 and 1)

⚫ Event – each possible outcome of a variable

⚫ Simple event – an event that can be described

by a single characteristic

⚫ Sample space – the collection of all possible

events
 I. INTRODUCTION
Assessing probability
There are three approaches to assessing the probability of
an uncertain event:
1. a priori -- based on prior knowledge of the
process
probability of occurrence = X = number of ways in which the event
Assuming
all
occurs T total number of possible outcomes
outcomes 2. empirical probability
are equally
likely number of ways in which the event
probability of occurrence = total number of possible outcomes
occurs

3. subjective probability
based on a combination of an individual’s past experience,
personal opinion, and analysis of a particular situation
I. INTRODUCTION
Example of a priori probability

When randomly selecting a day from the year 2015

what is the probability the day is in January?

number of days in
X
Probability of Day In January  T January
total number of days in
 2015

X 31days in
T 

January

31 365 days in 2015
I. INTRODUCTION
Example of empirical probability

Find the probability of selecting a male taking statistics from the

population described in the following table:

Taking Stats Not Taking Total

Stats
Male 84 145 229
Female 76 134 210
Total 160 279 439

number of males taking stats 84

Probability of male taking stats   
0.191
total number of people
439
I. INTRODUCTION
Sample Space
The sample space is the collection of all possible events
E.g. All 6 faces of a die:

E.g. All 52 cards of a bridge deck:

I. INTRODUCTION
Events
⚫ Simple event
◦ An outcome from a sample space with one
characteristic
◦ Ex: A red card from a deck of cards
◦ Example: A day in January from all days in
2024
I. INTRODUCTION
Events
⚫ Complement of an event A (denoted A’)
◦ All outcomes that are not part of event A
◦ Ex: All cards that are not diamonds
⚫ Joint event
◦ Involves two or more characteristics
simultaneously
◦ Ex. An ace that is also red from a
deck of cards
◦ Example: A day in January that is also
a Wednesday from all days in 2024
II. ORGANIZING AND VISUSALIZING EVENTS
2.1. Contingency tables and tree diagrams
⚫ Contingency tables
Ace Not Ace Total

Black 2 24 26
Red 2 24 26

Total 4 48 52

Sample
Space
⚫ Tree diagrams 2

Sample
Space
Full Deck 24
of 52 Cards
2

24
II. ORGANIZING AND VISUSALIZING EVENTS

⚫ Contingency tables -- for all days in

2015

⚫ Decision Trees
4 Number
Sample Of
27 Sample
Space
All Days Space
48
In 2015 Outcomes

28
II. ORGANIZING AND VISUALIZING EVENTS
2.2.Visualizing events

⚫ Venn Diagrams
◦ Let A = aces
A ∩ B = ace and red
◦ Let B = red cards (intersection)

B
A U B = ace or red
(union)
 II. ORGANIZING AND VISUSALIZING EVENTS

⚫ Venn diagram for all days in 2015

Sample Space (All Days That Are In January and

Are Wednesdays

January
Days

Wednesdays
II. ORGANIZING AND VISUSALIZING EVENTS
2.3.Simple probability
⚫ Simple probability refers to the probability of a
simple event.
◦ ex. P(Jan.)
◦ ex. P(Wed.)

P(Wed.) = 52 / 365
Jan Not Total
Wed 4 48 52
Not Web 27 286 313
Total 31 334 365

P(Jan.) = 31 / 365
II. ORGANIZING AND VISUSALIZING EVENTS
2.4.Joint probability
⚫ Joint probability refers to the probability of an
occurrence of two or more events (joint event).
◦ ex. P(Jan. and Wed.)
◦ ex. P(Not Jan. and Not Wed.)

Jan Not Total

Wed 4 48 52
Not Web 27 286 313 P(Not Jan. and Not
Wed.)
Total 31 334 365 = 286 / 365

P(Jan and Web) = 4 / 365

II. ORGANIZING AND VISUSALIZING EVENTS
2.5.Mutually exclusive events

⚫ Mutually exclusive events

◦ Events that cannot occur together

Example:

A = queen of diamonds; B = queen of clubs

◦ Events A and B are mutually

exclusive
II. ORGANIZING AND VISUSALIZING EVENTS
2.6.Collectively exhaustive events
⚫ Collectivelyexhaustive events
◦ One of the events must occur
◦ The set of events covers the entire sample
space

Example:

A = aces; B = black cards;

C = diamonds; D = hearts

◦ Events A, B, C and D are collectively exhaustive (but

not mutually exclusive – an ace may also be a heart)
◦ Events B, C and D are collectively exhaustive and
also mutually exclusive
II. ORGANIZING AND VISUSALIZING EVENTS
2.6.Collectively exhaustive events

Example: Randomly choose a day from 2015

A = Weekday; B = Weekend;
C = January; D = Spring;

◦ Events A, B, C and D are collectively exhaustive (but

not mutually exclusive – a weekday can be in January
or in Spring)

◦ Events A and B are collectively exhaustive and also

mutually exclusive
III. JOINT AND MARGINAL PROBABILITIES
Computing joint and marginal probabilities
⚫ The probability of a joint event, A and B:

number of outcomes satisfying A and

P(A and B) 
B total number of elementary
outcomes

⚫ Computing
P(A)  P(A andaBmarginal (or Bsimple)
1)  P(A and 2 )    P(A and

Bk probability:
)
🞄 Where B1, B2, …, Bk are k mutually exclusive
and collectively exhaustive events
III. JOINT AND MARGINAL PROBABILITIES
Joint probability
Example

Jan Not Total

Wed 4 48 52
Not Web 27 286 313
Total 31 334 365
III. JOINT AND MARGINAL PROBABILITIES
Marginal probability
Example

Jan Not Total

Wed 4 48 52
Not Web 27 286 313
Total 31 334 365

P(Jan.) = 31 / 365
P (Not)= 334/365
III. JOINT AND MARGINAL PROBABILITIES
Marginal and joint probability in a contingency
table

Event
Event B1 B2 Total

A1 P(A1 and B1) P(A1 and B2) P(A1)

A2 P(A2 and B1) P(A2 and B2) P(A2)

Total P(B1) P(B2) 1

Joint Probabilities Marginal (Simple) Probabilities

IV. PROBABILITIES
Probability summary
⚫ Probability is the numerical measure
of the likelihood that an event will Certain
1
occur
⚫ The probability of any event must be
between 0 and 1, inclusively
0 ≤ P(A) ≤ 1 For any event A
0.5
⚫ The sum of the probabilities of all
mutually exclusive and collectively
exhaustive events is 1

P(A)  P(B)  P(C)  1

0 Impossible
If A, B, and C are mutually exclusive and
collectively exhaustive
IV. PROBABILITIES

General Addition Rule:

P(A or B) = P(A) + P(B) - P(A and B)

If A and B are mutually exclusive, then

P(A and B) = 0, so the rule can be simplified:

P(A or B) = P(A) + P(B)

For mutually exclusive events A and B
IV. PROBABILITIES
General Addition Rule Example

P(Jan. or Wed.) = P(Jan.) + P(Wed.) - P(Jan. and Wed.)

= 31/365 + 52/365 - 4/365 =

79/365Don’t count
the four
Jan Not Total Wednesdays
Wed 4 48 52 in January
twice!
Not Web 27 286 313

Total 31 334 365

IV. PROBABILITIES
General Addition Rule Example

At a local language school, 40% of the students are learning

Spanish, 20% of the students are learning German, and 8% of
the students are learning both Spanish and German.
What is the probability that a randomly selected student is
learning Spanish or German?

Solution:
P(Spanish orGerman)=P(Spanish)+P(German)
−P(Spanish
and German)
=0.4+0.2−0.08=0.52
IV. PROBABILITIES
Computing conditional probabilities
⚫ A conditional probability is the probability of one event,
given that another event has occurred:

The conditional
P(A and B)
P(A | B)  P(B) probability of A
given that B has
occurred

The conditional
P(A and B) probability of B
P(B | A)  P(A) given that A has
occurred

Where P(A and B) = joint probability of A and B

P(A) = marginal or simple probability of A
P(B) = marginal or simple probability of B
IV. PROBABILITIES
Conditional probability
Example

Of the cars on a used car lot, 70% have air conditioning (AC)
and 40% have a GPS. 20% of the cars have both.

⚫ What is the probability that a car has a GPS,

given that it has AC?

So ,we want to find P(GPS | AC)

IV. PROBABILITIES
Conditional probability
Example
Of the cars on a used car lot, 70% have air conditioning
(AC) and 40% have a GPS and 20% of the cars have both.

GPS Not GPS Total

AC 0.2 0.5 0.7

Not AC 0.2 0.1 0.3

Total 0.4 0.6 1..0

P(GPSand AC) 0.2

P(GPS | AC)    0.2857
P(AC)
IV. PROBABILITIES
Using decision trees
.2
Given AC or no
.7 P(AC and GPS) = 0.2

AC:

P(AC and GPS’) = 0.5

.5
.7
Conditional
All Probabilities
Cars
.2
.3 P(AC’ and GPS) = 0.2

.1 P(AC’ and GPS’) = 0.1

.3
IV. PROBABILITIES
Using decision trees
(continued)
.2
.4 P(GPS and AC) = 0.2

Given GPS or
no GPS:
P(GPS and AC’) = 0.2
.2
.4
Conditional
All Probabilities
Cars
.5
.6 P(GPS’ and AC) = 0.5

.1 P(GPS’ and AC’) = 0.1

.6
IV. PROBABILITIES
Independence
⚫ Two events are independent if and only if:

P(A | B) 
P(A)
⚫ Events A and B are independent when the probability of
one event is not affected by the fact that the other event
has occurred
IV. PROBABILITIES
Multiplication rules
⚫ Multiplication rule for two events A and
B:

P(A and B) P(A | B)P(B)

Note: If A and B are independent, then P(A | B)  P(A)

and the multiplication rule simplifies to

P(A and B)  P(A)P(B)

IV. PROBABILITIES
Bayes’ Theorem
⚫ Bayes’ Theorem is used to revise previously
calculated probabilities based on new
information.

⚫ Developed by Thomas Bayes in 18th

the Century.

⚫ It is an extension of conditional probability.

IV. PROBABILITIES

Bayes’ Theorem (conditional probability)

P(A | B i )P(Bi )
P(B | A) 
i
P(A | B )P(B )  P(A | B )P(B )
 P(A | 1B )P(B ) 1 2 2
k k

⚫ where:
Bi = ith event of k mutually exclusive and collectively
exhaustive events
IV. PROBABILITIES
Bayes’
Theorem
Example
⚫A drilling company has estimated a 40% chance of
striking oil for their new well.
⚫A detailed test has been scheduled for more
information. Historically, 60% of successful wells
have had detailed tests, and 20% of unsuccessful
wells have had detailed tests.
⚫ Given that this well has been scheduled for a
detailed test, what is the probability
that the well will be successful?
IV. PROBABILITIES

Bayes’ Theorem
⚫ Let S = successful well
U = unsuccessful well
⚫ P(S) = 0.4 , P(U) = 0.6 (prior probabilities)
⚫ Define the detailed test event as D
⚫ Conditional probabilities:
P(D|S) = 0.6 P(D|U) = 0.2
⚫ Goal is to find P(S|D)
IV. PROBABILITIES

Bayes’ Theorem
Apply Bayes’ Theorem:

P(D | S)P(S)
P(S |D)
 P(D | S)P(S)  P(D |
U)P(U)
(0.6)(0.4)

(0.6)(0.4)
0.24  (0.2)
 
(0.6)
0.24  0.667
0.12
So the revised probability of success, given that this well
has been scheduled for a detailed test, is 0.667
IV. PROBABILITIES

⚫ Given the detailed test, the revised probability

of a successful well has risen to 0.667 from the
original estimate of 0.4

Prior Conditional Joint Revised

Event
Prob.
Prob. Prob. Prob.
S (successful) 0.4 0.6 (0.4)(0.6) = 0.24 0.24/0.36 = 0.667
U (unsuccessful) 0.6 0.2 (0.6)(0.2) = 0.12 0.12/0.36 = 0.333

Sum = 0.36
IV. PROBABILITIES
Counting rules are often useful in computing
probabilities
⚫ Inmany cases, there are a large number of
possible outcomes.

⚫ Countingrules can be used in these cases to

help compute probabilities.
IV. PROBABILITIES
Counting rules
⚫ Rules for counting the number of possible outcomes
⚫ Counting Rule 1:
◦ If any one of k different mutually exclusive and
collectively exhaustive events can occur on each of
n trials, the number of possible outcomes is
equal to
n
k

◦ Example
🞄 If you roll a fair die 3 times then there are 63 = 216
possible outcomes
IV. PROBABILITIES

Counting rules
⚫ Counting rule 2:
◦ If there are k1 events on the first trial, k2 events on
the second trial, … and kn events on the nth
trial, the number of possible outcomes is

…
◦ Example: (k1)(k2) (kn)
🞄 You want to go to a park, eat at a restaurant, and see a
movie. There are 3 parks, 4 restaurants, and 6 movie
choices. How many different possible combinations are
there?
🞄 Answer: (3)(4)(6) = 72 different possibilities
IV. PROBABILITIES
Counting rules
⚫ Counting rule 3:
◦ The number of ways that n items can be arranged in
order is …
n! = (n)(n – 1) (1)

◦ The formula for factorial

◦ Example:
🞄 You have five books to put on a bookshelf.
How many different ways can these books be
placed on the shelf?

🞄 Answer: 5! = (5)(4)(3)(2)(1) = 120

different possibilities
IV. PROBABILITIES
Counting Rules
⚫ Counting rule 4:
◦ Permutations: The number of ways of arranging X
objects selected from n objects in order is

n!
n Px
 (n  X)!
◦ Example:
🞄 You have five books and are going to put three on a
bookshelf. How many different ways can the books be
ordered on the bookshelf?

🞄 Answer:
n! 5!
n Px  
120
 (n  X)! (5  3)! 2  60 different
possibilities
IV. PROBABILITIES
Counting rules
⚫ CountingRule 5:
◦ Combinations: The number of ways of selecting
X objects from n objects, irrespective of order,
is
n!
n Cx
 X!(n  X)!

◦ Example:
🞄 You have five books and are going to select
three are to read. How many different
combinations are there, ignoring the order in
which they are selected?
🞄 Answer: n!
nC   120  10 different

X!(n5! X)! 3!(5  3)! (6) possibilities
x
(2)