PROBABILITY

Random Experiments
Probability
Rules of Probability
 Random Experiment

Sample Space

• A random experiment is an observational process whose

results cannot be known in advance.
• The set of all outcomes (S) is the sample space for the
experiment.
• A sample space with a countable number of outcomes is
discrete.
 Random Experiment
Sample Space
• For a single roll of a die, the sample space is:
S = {1, 2, 3, 4, 5, 6}
• When two dice are rolled, the sample space is the
following pairs:

S= {(1,1), (1,2), (1,3), (1,4), (1,5), (1,6),

(2,1), (2,2), (2,3), (2,4), (2,5), (2,6),
(3,1), (3,2), (3,3), (3,4), (3,5), (3,6),
(4,1), (4,2), (4,3), (4,4), (4,5), (4,6),
(5,1), (5,2), (5,3), (5,4), (5,5), (5,6),
(6,1), (6,2), (6,3), (6,4), (6,5), (6,6)}
 Random Experiment
Sample Space
Consider the sample space to describe a randomly chosen
United Airlines employee by:
2 genders
21 job classifications
6 home bases (major hubs)
4 education levels
There are: 2 x 21 x 6 x 4 = 1008 possible outcomes

It would be impractical to enumerate this

sample space
 Multiple Step Experiment

If an experiment can be described as a sequence of k steps

with,
n1 possible outcomes on the first step,
n2 possible outcomes on the second step, and so on,
then, the total number of experimental outcomes is
(n1)· (n2)· (n3)· · · (nk)
 Tree Diagram
Experimental
Step 1 Step 2 Outcome
Second Coin (Sample Point)
First Coin
(H,H)

(H,T)

(T,H)

(T,T)
.
Kentucky Power & Light (KP&L) Project
KP&L is starting a new project designed to increase the
generating capacity of one of its plants in northern Kentucky.
The project is divided in to two sequential stages –
Stage 1 – Design
Stage 2 – Construction
An analysis of similar construction projects in the past,
revealed possible completion times for the design stage of 2, 3
or 4 months and possible completion times for the
construction stage of 6, 7 or 8 months.
The Management sets a goal of 10 months for the completion
of the entire project.
 Experimental
Step 2 Outcome Total Project
Step 1 - Design Construction (Sample Point) Completion Time
(2 , 6) 8 months

. 7 mo.

8 mo.
(2 , 7) 9 months

(2 , 8) 10 months
2 mo.
9 months
6 mo. (3 , 6)

. 3 mo.
. 7 mo. (3 , 7) 10 months
8 mo.
11 months
(3 , 8)
4 mo.
(4 , 6) 10 months
6 mo.

. 7 mo.
8 mo.
(4 , 7) 11 months

(4 , 8) 12 months
 Counting Rule for Combinations

To count the number of experimental outcomes when n objects

are to be selected from a set of N objects.
Number of Combinations of N Objects Taken n at a Time
N!
N
Cn 
n!( N  n)!

where: N! = N(N - 1)(N - 2) . . . (2)(1)

n! = n(n - 1)(n - 2) . . . (2)(1)
0! = 1
 Counting Rule for Permutations

To count the number of experimental outcomes when n objects

are to be selected from a set of N objects, where the order of
selection is important.
Number of Permutations of N Objects Taken n at a Time
N!
N
Pn 
( N  n)!
where: N! = N(N - 1)(N - 2) . . . (2)(1)
n! = n(n - 1)(n - 2) . . . (2)(1)
0! = 1
 Random Experiment
Sample Space
If the outcome is a continuous measurement, the sample space
can be described by a rule.
For example, the sample space for the length of a randomly
chosen cell phone call would be
S = {all X such that X > 0}
or written as S = {X | X > 0}
The sample space to describe a randomly chosen student’s GPA
would be
S = {X | 0.00 < X < 5.00}
 Random Experiment

Events
• An event is any subset of outcomes in the sample space.
• A simple event or elementary event, is a single outcome.
• A discrete sample space S consists of all the simple events
(Ei)
S = {E1, E2, …, En}
 Random Experiment
Events
• Consider the random experiment of tossing a balanced
coin. What is the sample space?
S = {H, T}
• What are the chances of observing a H or T?
• These two elementary events are equally likely
• When you buy a lottery ticket, the sample space
S = {win, lose} has only two events.
• Are these two events equally likely to occur?
 Random Experiment
Events
A compound event consists of two or more simple events. For
example, in a sample space of 7 simple events, we could define
the compound events
A = {Music, DVD, VHS}
B = {Newspapers, Magazines}
 Random Experiment
Events
 Many different compound events could be defined.
 Compound events can be described by a rule.
 For example, the compound event
A = “rolling a seven” on a roll of two dice consists
of 6 simple events:
S = {(1,6), (2,5), (3,4), (4,3), (5,2), (6,1)}
 Probability

• The probability of an event is a number that measures the

relative likelihood that the event will occur.
• The probability of event A [denoted P(A)], must lie within the
interval from 0 to 1:
0 < P(A) < 1

If P(A) = 0, then the If P(A) = 1, then the event

event cannot occur. is certain to occur.
 Probability
• In a discrete sample space, the probabilities of all simple
events must sum to unity:

P(S) = P(E1) + P(E2) + … + P(En) = 1

• For example, if the following number of purchases were

made by

Credit card: 32% P(Credit card) = 0.32

Debit card: 20% Probability P(Debit card) = 0.20
Cash: 30% P(Cash) = 0.30
Cheque: 18% P(Cheque) = 0.18
Sum = 100% Sum = 1.00
 Probability
What is Probability?
Three approaches to probability:

Approach Example
There is a 2 percent chance of twins in
Empirical a randomly chosen birth

There is a 50 % probability of heads on

Classical a coin flip
There is a 75 % chance that England will
Subjective adopt the Euro currency by 2013
 Assigning Probabilities…

Classical approach
- based on equally likely events.

Relative frequency approach (empirical)

- assigning probabilities based on
experimentation or historical data.

Subjective approach
- assigning probabilities based on the assignor’s
(subjective) judgment.
 Classical Approach…

If an experiment has n possible outcomes, this method would

assign a probability of 1/n to each outcome. It is necessary to
determine the number of possible outcomes.

Experiment : Rolling a die

Outcomes : {1, 2, 3, 4, 5, 6}
Probabilities : Each sample point has a 1/6
chance of occurring.
 Classical Approach…

Experiment : Rolling two dice and observing the total

Outcomes : {2, 3, …, 12}

1 2 3 4 5 6
P(2) = 1/36 1 2 3 4 5 6 7
2 3 4 5 6 7 8
P(6) = 5/36
3 4 5 6 7 8 9
P(10) = 3/36 4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12
 Relative Frequency Approach…

Bits & Bytes Computer Shop tracks the number of desktop computer
systems it sells over a month (30 days):

Desktops
For example, # of Days
Sold
12 days out of 30 0 1
3 desktops were sold. 1 2
2 10
From this we can construct
3 12
the probabilities of an event
4 5
(i.e. the # of desktop sold on a given day)…
 Relative Frequency Approach…

Desktops Sold # of Days Desktops Sold

0 1 1/30 = 0.03
1 2 2/30 = 0.07
2 10 10/30 = 0.33
3 12 12/30 = 0.40
4 5 5/30 = 0.17
∑ = 1.00

“There is a 40% chance Bits & Bytes will sell 3 desktops on

any given day”
 Subjective Approach…
“In the subjective approach we define probability as the
degree of belief that we hold in the occurrence of an
event”
E.g. weather forecasting’s “P.O.P.”
“Probability of Precipitation” (or P.O.P.) is defined
in different ways by different forecasters, but
basically it’s a subjective probability based on past
observations combined with current weather
conditions.

POP 60% – based on current conditions, there is a 60%

chance of rain (say).
 Interpreting Probability…
No matter which method is used to assign probabilities all
will be interpreted in the relative frequency approach.

For example, a government lottery game where 6 numbers

(of 49) are picked. The classical approach would predict the
probability for any one number being picked as
1/49=0.0204.

We interpret this to mean that in the long run each

number will be picked 2.04% of the time.
 Joint, Marginal, Conditional Probability…
We study methods to determine probabilities of events that
result from combining other events in various ways.
There are several types of combinations and relationships
between events:
– Complement event
– Intersection of events
– Union of events
– Mutually exclusive events
– Dependent and independent events
 Complement of an Event…
The complement of event A is defined to be the event
consisting of all sample points that are “not in A”.

Complement of A is denoted by Ac

The Venn diagram below illustrates the concept of a

complement.

A
P(A) + P(Ac ) = 1
Ac
 Complement of an Event…
For example, the rectangle stores all the possible tosses of 2 dice
{(1,1), 1,2),… (6,6)}.
Let A = tosses totaling 7 {(1,6), (2, 5), (3,4), (4,3), (5,2), (6,1)}

P(Total = 7) + P(Total not equal to 7) = 1

A Ac
 Intersection of Two Events…
The intersection of events A and B is the set of all sample
points that are in both A and B.

The intersection is denoted: A and B

The joint probability of A and B

A B
is the probability of the intersection of A and B,
i.e. P(A and B)
 Intersection of Two Events…

For example,
let A = tosses where first toss is 1
{(1,1), (1,2), (1,3), (1,4), (1,5), (1,6)}

and B = tosses where the second toss is 5

{(1,5), (2,5), (3,5), (4,5), (5,5), (6,5)}

The intersection is {(1,5)}. The joint probability of A

and B is the probability of the intersection of A and B,
i.e. P(A and B) = 1/36
 Union of Two Events…

The union of two events A and B, is the event containing

all sample points that are in A or B or both:

Union of A and B is denoted: A or B

A B
 Union of Two Events…

For example, let A = tosses where first toss is 1

{(1,1), (1,2), (1,3), (1,4), (1,5), (1,6)}

and B is the tosses that the second toss is 5

{(1,5), (2,5), (3,5), (4,5), (5,5), (6,5)}

Union of A and B is {(1,1), (1,2), (1,3), (1,4), (1,5), (1,6)}

(2,5), (3,5), (4,5), (5,5), (6,5)}
 Mutually Exclusive Events…
When two events are mutually exclusive (that is the two events
cannot occur together), their joint probability is 0, hence:

A B

Mutually exclusive; no points in common…

For example A = tosses totaling 7 and B = tosses totaling 11
 Basic Relationships of Probability…

Complement of Event Union of Events

A Ac A B

Intersection of Events Mutually Exclusive Events

A B A B
Why are some mutual fund managers more successful than
others? One possible factor is where the manager earned his or
her MBA. The following table compares mutual fund performance
against the ranking of the school where the fund manager earned
their MBA:

Mutual fund Mutual fund

outperforms the doesn’t outperform
market the market
Top 20 MBA program 0.11 0.29
Not top 20 MBA program 0.06 0.54

E.g. This is the probability that a mutual fund

outperforms AND the manager was in a top-
20 MBA program; it’s a joint probability.
Alternatively, we could introduce shorthand notation to represent
the events:
A1 = Fund manager graduated from a top-20 MBA program
A2 = Fund manager did not graduate from a top-20 MBA program
B1 = Fund outperforms the market
B2 = Fund does not outperform the market

B1 B2
A1 0.11 0.29
A2 0.06 0.54

E.g., P(A2 and B1) = 0.06

= the probability a fund outperforms the market
and the manager isn’t from a top-20 school.
 Marginal Probabilities…
Marginal probabilities are computed by adding across rows and
down columns; that is they are calculated in the margins of the
table:
P(A2) = 0.06 + 0.54
“what’s the probability a fund manager isn’t from a top school?”

B1 B2 P(Ai)
A1 0.11 0.29 0.40
A2 0.06 0.54 0.60
P(Bj) 0.17 0.83 1.00

P(B1) = 0.11 + 0.06 BOTH margins must add to 1

“what’s the probability a fund (useful error check)
outperforms the market?”
 Conditional Probability…

Conditional probability is used to determine how two events

are related; that is, we can determine the probability of one
event given the occurrence of another related event.

Conditional probabilities are written as P(A | B) and read as

“the probability of A given B” and is calculated as:
 Conditional Probability…
Again, the probability of an event given that another event has
occurred is called a conditional probability…

Note how “A given B” and “B given A” are related…

 Conditional Probability…
What’s the probability that a fund will outperform the market
given that the manager graduated from a top-20 MBA
program?
Recall:
A1 = Fund manager graduated from a top-20 MBA program
A2 = Fund manager did not graduate from a top-20 MBA program
B1 = Fund outperforms the market
B2 = Fund does not outperform the market

Thus, we want to know “what is P(B1 | A1) ?”

 Conditional Probability…
We want to calculate P(B1 | A1)

B1 B2 P(Ai)
A1 0.11 0.29 0.40
A2 0.06 0.54 0.60
P(Bj) 0.17 0.83 1.00

Thus, there is a 27.5% chance that that a fund will

outperform the market given that the manager graduated
from a top-20 MBA program.
 Independence…
One of the objectives of calculating conditional probability is
to determine whether two events are related.

In particular, we would like to know whether they are

independent, that is, if the probability of one event is not
affected by the occurrence of the other event.

Two events A and B are said to be independent if

P(A|B) = P(A)
or
P(B|A) = P(B)
 Independence…
For example, we saw that

P(B1 | A1) = 0.275

The marginal probability for B1 is: P(B1) = 0.17

Since P(B1|A1) ≠ P(B1), B1 and A1 are not independent

events.

Stated another way, they are dependent. That is, the

probability of one event (B1) is affected by the
occurrence of the other event (A1).
 Union…

We stated earlier that the union of two events is denoted as:

A or B. We can use this concept to answer questions like:

Determine the probability that a fund outperforms the market

or the manager graduated from a top-20 MBA program.
 Union…
Determine the probability that a fund outperforms (B1)
or the manager graduated from a top-20 MBA program (A1).
A1 or B1 occurs whenever:
A1 and B1 occurs, A1 and B2 occurs, or A2 and B1 occurs…

B1 B2 P(Ai)
A1 0.11 0.29 0.40
A2 0.06 0.54 0.60
P(Bj) 0.17 0.83 1.00

P(A1 or B1) = 0.11 + 0.06 + 0.29 = 0.46

 Union…
Determine the probability that a fund outperforms (B1)
or the manager graduated from a top-20 MBA program (A1).

B1

B1 B2 P(Ai)
A1 A1 .11 .29 .40
A2 .06 .54 .60

P(Bj) .17 .83 1.00

P(A1 or B1) = 0.11 + 0.06 + 0.29 = 0.46

 Alternatively…
Take 100% and subtract off “when doesn’t A1 or B1
occur”?
i.e. at A2 and B2
B1
B1 B2 P(Ai)
A1 A1 0.11 0.29 .40
A2 0.06 0.54 .60
(Bj) 0.17 0.83 1.00

P(A1 or B1) = 1 – P(A2 and B2) = 1 – 0.54 = 0.46

 Probability Rules and Trees…

We introduce three rules that enable us to calculate the

probability of more complex events from the probability of
simpler events…

The Complement Rule,

The Multiplication Rule, and

The Addition Rule

 Complement Rule…
As we saw earlier with the complement event, the
complement rule gives us the probability of an event NOT
occurring. That is:

P(AC) = 1 – P(A)

For example, in the simple roll of a die, the probability of

the number “1” being rolled is 1/6. The probability that
some number other than “1” will be rolled is 1 – 1/6 = 5/6.
 Multiplication Rule…
The multiplication rule is used to calculate the joint probability
of two events. It is based on the formula for conditional
probability defined earlier:

If we multiply both sides of the equation by P(B) we have:

P(A and B) = P(A | B)•P(B)

Likewise, P(A and B) = P(B | A) • P(A)

If A and B are independent events, then P(A and B) = P(A)•P(B)

A graduate statistics course has seven male and three female
students. The professor wants to select two students at
random to help her conduct a research project. What is the
probability that the two students chosen are female?

Let A represent the event that the first student is female

P(A) = 3/10 = 0.30

What about the second student?

A graduate statistics course has seven male and three
female students. The professor wants to select two
students at random to help her conduct a research project.
What is the probability that the two students chosen are
female?

Let B represent the event that the second student is female

P(B | A) = 2/9 = 0.22

That is, the probability of choosing a female student given

that the first student chosen is a female is 2 (females) / 9
(remaining students) = 2/9
A graduate statistics course has seven male and three female
students. The professor wants to select two students at
random to help her conduct a research project. What is the
probability that the two students chosen are female?

Thus, we want to answer the question: what is P(A and B) ?

P(A and B) = P(A)•P(B|A) = (3/10)(2/9) = 6/90 = 0.067

“There is a 6.7% chance that the professor will choose two

female students from her grad class of 10.”
The professor who teaches the course is suffering from the flu
and will be unavailable for two classes. The professor’s
replacement will teach the next two classes. His style is to
select one student at random and pick on him or her to
answer questions during that class. What is the probability
that the two students chosen are female?

Let A represent the event that the first student is female

P(A) = 3/10 = 0.30

What about the second student?

Let B represent the event that the second student is female

P(B | A) = 3/10 = 0.30

That is, the probability of choosing a female student given that

the first student chosen is unchanged since the student
selected in the first class can be chosen in the second class.

P(A and B) = P(A)•P(B|A) = (3/10)(3/10) = 9/100 = 0.090

 Addition Rule…
Recall: the addition rule is used to compute the probability of
event A or B or both A and B occurring; i.e. the union of A and B.

P(A or B) = P(A) + P(B) – P(A and B)

A B = A + B –
If A and B are
mutually exclusive,
P(A or B) = P(A) + P(B) – P(A and B) then this term
goes to zero
In a large city, two newspapers are published, the Sun and the
Post. The circulation departments report that 22% of the city’s
households have a subscription to the Sun and 35% subscribe
to the Post. A survey reveals that 6% of all households
subscribe to both newspapers. What proportion of the city’s
households subscribe to either newspaper?

That is, what is the probability of selecting a household at

random that subscribes to the Sun or the Post or both?

i.e. what is P(Sun or Post) ?

In a large city, two newspapers are published, the Sun and the Post.
The circulation departments report that 22% of the city’s households
have a subscription to the Sun and 35% subscribe to the Post. A
survey reveals that 6% of all households subscribe to both
newspapers. What proportion of the city’s households subscribe to
either newspaper?

P(Sun or Post) = P(Sun) + P(Post) – P(Sun and Post)

= 0.22 + 0.35 – 0.06 = 0.51

“There is a 51% probability that a randomly selected household

subscribes to one or the other or both papers”
 Conditional Probability
Example: Payment Method and Purchase Quantity
A small grocery store would like to know if the number of items
purchased by a customer is independent of the type of
payment method the customer chooses to use. Why would this
information be useful to the store manager? The manager
collected a random sample of 368 customer transactions.
 Conditional Probability

Example: Payment Method and Purchase Quantity

Here is the contingency table of frequencies:
 Conditional Probability

Payment Method and Purchase Quantity

• Calculate the marginal probability that a customer will use
cash to make the payment.

• Let C be the event cash.

P(C) = 126/368 = 0.3424

• Now, is this probability the same if we condition on number

of items purchased?
 Conditional Probability
Example: Payment Method and Purchase Quantity
P(C | 1-5) = 30/88 = 0.3409
P(C | 6-10) = 46/135 = 0.3407
P(C | 10-20) = 31/89 = 0.3483
P(C | 20+) = 19/56 = 0.3393

• P(C) = 0.3424, so what do you conclude about independence?

• Based on this, the manager might decide to offer a cash-only
lane that is not restricted to the number of items purchased