Probability Theory

Week 6

1
Part 1:
• Applying Counting Rules to Probability Problems:
• Rule for Multiple-step Experiments
• Counting Rule for Permutations
• Counting Rule for Combinations

If event A is getting HHT when you toss a coin 3 times, what is

the probability of A? 2
 Counting Rules:
I. Rule for Multiple-Step Experiments
 If an experiment consists of a sequence of k steps
in which there are n1 possible results for the first step,
n2 possible results for the second step, and so on,
then the total number of experimental outcomes is
given by (n1)(n2) . . . (nk).

 Example: in the morning you are choosing an outfit. You

have 3 pants, 5 shirts and 4 shoes. How many different
outfits can you put together? 3*5*4=60
3
Example: X X X

In a sequence of three random digits, what is the

probability that the code is 976?
 

4
Example: X X X

In a sequence of three random digits, what is the

probability that the code is 976?
 

Using multi-step approach: 10 ways to select a number at step 1,

10 ways to select a number at step 2 and 10 ways to select
a number at step 3. So in total: 10x10x10=1,000 ways

5
 II. Counting Rule for Permutations
 1. Number of permutations: take k objects out of n objects (order matters!)

 
Can think of
this as
forming
lines
 Example: In how many ways can we arrange 3 people?
6 ways ABC, ACB, BAC, BCA, CAB, CBA

Counting permutation (order matters):

  3 3! 3 ∗ 2∗ 1 6
𝑃 =
3 = = =6
(3 −3 )! 0! 1
6
 Example:
In a sequence of three random digits, what is the probability
that the digits are all distinct (different)?
 

 𝑀 =¿ 𝑜𝑓 𝑤𝑎𝑦𝑠 𝑤𝑒 𝑐𝑎𝑛 𝑠𝑒𝑙𝑒𝑐𝑡 3 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 𝑛𝑢𝑚𝑏𝑒𝑟𝑠 𝑜𝑢𝑡 𝑜𝑓 10 𝑝𝑜𝑠𝑠𝑖𝑏𝑙𝑒 𝑛𝑢𝑚𝑏𝑒𝑟𝑠

Should we use:
 𝑁 =¿ 𝑜𝑓 𝑎𝑙𝑙 𝑝𝑜𝑠𝑠𝑖𝑏𝑙𝑒 𝑐𝑜𝑚𝑏𝑖𝑛𝑎𝑡𝑖𝑜𝑛𝑠 1. Multiplication rule
2. Combinations
3. Permutations
to find M?
7
 Example:
In a sequence of three random digits, what is the probability
that the digits are all distinct?
 

 𝑀 = ¿ 𝑜𝑓 𝑤𝑎𝑦𝑠 𝑤𝑒 𝑐𝑎𝑛 𝑠𝑒𝑙𝑒𝑐𝑡 3 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 𝑛𝑢𝑚𝑏𝑒𝑟𝑠 𝑜𝑢𝑡 𝑜𝑓 10= 𝑃10 = 10 !

3 =10 ∗9 ∗ 8= 720
( 10 −3 ) !
There are 720 ways we can choose 3 different numbers out
of 10 possible numbers (where 345 is different from 435).
  N

Using multiplication rule: 10 ways to select a number at step 1, 10

ways to select a number at step 2 and 10 ways to select a number at step 3. 8
 Example:
In a sequence of three random digits, what is the probability
that the digits are all distinct?

9
 III. Counting Rule for Combinations
Number of combinations: take k objects out of n objects (order doesn’t matter!)

 
Can think of
this as
forming
group
Example: In how many ways can form a group of 3 people
from 3 people?
ABC=ACB=BAC=BCA=CAB=CBA 1 way (all these are
same group)

Counting combination (order doesn’t matter):

  3 3! 3 ∗2 ∗1 6
𝐶 = 3 = = =1
( 3− 3 ) ! 3 ! 0!3! 6
10
Example:
Suppose, we are randomly selecting 2 out of 5 candidates, and
we know that 2 of the candidates are good, and 3 are bad.

1. Find the number of ways we can select:

Exactly one good applicant out of 2 people = event A

2. What is the probability of observing event A?

 

11
Example: Suppose, we are randomly selecting 2 out of 5 candidates, and
we know that 2 of the candidates are good, and 3 are bad
(Candidates: G1, G2, B1, B2, B3)

1. Find the number of ways of selecting exactly one good applicant

(event A). This means selecting one good applicant and one bad
applicant (GB). Let’s find how in many ways we can do that:

Step 1: select 1 out of 2 good candidates

2 possible ways of doing that

Step 2: select 1 out of 3 bad candidates

3 possible ways of doing that

In total, there are 2x3=6 ways of selecting 1 good and 1 bad candidate
12
Example:
Suppose, we are randomly selecting 2 out of 5 candidates, and we
know that 2 of the candidates are good, and 3 are bad.

2. What is the probability of observing event A (1 good and 1 bad?

 

  5!
5
𝑁 =𝐶 = 2 =10
2!3!

13
 Geometric way of solving
probability problems:
Romeo and Juliet problem
Romeo and Juliet want to meet up for a date tomorrow at 1pm,
but the problem is that they don’t have phones and they aren’t
necessarily punctual. The possible times they can arrive
(individually) are at: 1pm, 1:15pm, 1:30pm, 1:45pm and 2pm [So,
one can arrive at 1pm and another at 1:30pm for example. All
pairs are equally likely].

Each person is willing to wait for another person at most 15

minutes. If the other person doesn’t show up in 15 minutes, then
the person who arrived first will leave.

What is the probability that they meet?

28
 Romeo and Juliet problem: graphical solution

The number of possible

outcomes is 5x5=25.

Out of 25 outcomes, they

meet in 13.

So, the probability of meeting

= 13/25

Chap 4-15
Part II: Chapter 4 continued
Some Basic Relationships of Probability
Complement of an Event

Union of Two Events

Intersection of Two Events

Mutually Exclusive Events

Complement of an Event

The complement of event A is defined to be the event

consisting of all sample points that are not in A.

The complement of A is denoted by Ac.

Sample
Event A Ac Space S

Venn
Diagram
In case of rolling a dice: if event A is
rolling 2 or 4, what is complement of A?
Which of the following statements is always true?

a) -1 ≤ P(Ei) ≤ 0.999

b) P(A) = 1 - P(Ac)

c) P(A) + P(B) = 1

d) ∑P ≥ 1

18
Which of the following statements is always true?

a) -1 ≤ P(Ei) ≤ 0.999

b) P(A) = 1 - P(Ac)

c) P(A) + P(B) = 1

d) ∑P ≥ 1

19
 Union of Two Events
The union of events A and B is the event containing
all sample points that are in A or B or both.

The union of events A and B is denoted by A B

Sample
Space S

AUB
Event A is getting a 1 or 3 when rolling a
die. Event B is getting 4. What is A U B?
 Intersection of Two Events

The intersection of events A and B is the set of all

sample points that are in both A and B.

The intersection of events A and B is denoted by A 

EventA Sample
In case of rolling a Space S
die: event A – rolling Event B
3 or less. Event B –
rolling 5 or less. What
is A  ?
Intersection of A and B
Mutually Exclusive Events
Two events are said to be mutually exclusive if the
events have no sample points in common.

Two events are mutually exclusive if, when one event

occurs, the other cannot occur.

Sample
Event A Event B Space S
Addition rule: 2 events
NOT mutually exclusive
S

A AUB B

Why are we
subtracting
intersection of A
and B?

When is this
equal to zero?
P(AUB)=P(A)+P(B)
For mutually
exclusive events 23
Conditional Probability

The probability of an event given that another event

has occurred is called a conditional probability.

The conditional probability of A given B is denoted

by P(A|B). Opposite is P(B|A).

A conditional probability is computed as follows :

P( A  B ) P(A  B)
P( A|B)  P(B | A) 
P( B) P(A)
 Understanding conditional probability formula

Of the cars on a used car lot, 70% have air conditioning (AC)
and 40% have a CD player (CD). 20% of the cars have both.
We are also given all these probabilities in the table.
Q1: what do
circled numbers CD No CD Total
represent? AC 0.2 0.5 0.7
Interpret these
numbers. No AC 0.2 0.1 0.3
Total 0.4 0.6 1.0
Q2: Given that a
car has a CD,
what is the
probability that
is has AC? 8
 Understanding conditional probability formula

Of the cars on a used car lot, 70% have air conditioning

(AC) and 40% have a CD player (CD). 20% of the cars
have both.
𝑃(
  𝐴𝐶 ∩𝐶𝐷)
Q1: what do CD No CD Total
circled numbers 0.2 0.5 0.7
AC
represent?
Interpret these No AC 0.2 0.1 0.3
numbers.
Total 0.4 0.6 1.0
Q2: Given that a
car has a CD,   𝑃( 𝐴𝐶 ∩ 𝐶𝐷) 0.2
𝑃 ( 𝐴𝐶 |𝐶𝐷 ¿= = = 0.5
what is the 𝑃(𝐶𝐷) 0.4
probability that
is has AC? 8
Conditional Probability for more than 2 events

A conditional probability is computed as follows :

P( A  B ) P(A  B)
P( A|B)  P(B | A) 
P( B) P(A)

𝑃 ¿
 
Independent Events

If the probability of event A is not changed by the

existence of event B, we would say that events A
and B are independent.

Two events A and B are independent if:

P(A|B) = P(A) or P(B|A) = P(B)

You can also think of independence in this way:

knowing something about one event doesn’t tell me
anything about the probability of another event.
 Independent Events

If two events A and B are independent:

P(A|B) = P(A) or P(B|A) = P(B)

What happens to conditional probability formulas?

P( A  B ) P(A  B)
P( A|B)  P(B | A) 
P( B) P(A)

Two events are independent if and only if:

P(A  B)  P(A) P(B)

 Probability formulas we covered so far:

P(AUB)=P(A)+P(B) If A and B are mutually exclusive

If A and B are NOT mutually exclusive

P( A  B) P(A  B)
P( A|B)  P(B | A)  Conditional probability
P( B ) P(A)

Based on conditional probability

P(A|B) = P(A) P(B|A) = P(B) P(A  B)  P(A) P(B) Independent events

Problem 1: If A and B are mutually exclusive
events with P(A) = 0.4 and P(B) = 0.5, then P(A ∩
B) =

a) 0.00
b) 0.10
c) 0.90
d) 0.20

31
If A and B are mutually exclusive events with
P(A) = 0.4 and P(B) = 0.5, then P(A ∩ B) =

a) 0.00
b) 0.10
c) 0.90
d) 0.20

32
 Problem 2: if P(A) = 0.4, P(B | A) = 0.35,
P(A ∪ B) = 0.69, then P(B) =

a) 0.14
P( A  B) P(AUB
b) 0.43 P( A|B)  ) =P(A)+
P( B )
P( B)
c) 0.75
d) 0.59 ( A) P(A  B)
e be |B) = P P(B | A) 
o f t h
ld l? P(A P(A)
ich as co sefuu
h
W ul ly u
o r m al
f e n ti
p o t
33
 Problem: if P(A) = 0.4, P(B | A) = 0.35, P(A ∪ B) =
0.69, find P(B)

𝑺𝒕𝒆𝒑 𝟏: 𝑃 ( 𝐴 ∩ 𝐵 )=𝑃 ( 𝐵| 𝐴 ) ∗ 𝑃 ( 𝐴 )=0.35 ∗ 0.4=0.14

 

𝑺𝒕𝒆𝒑 𝟐 : 𝑃 ( 𝐵 )=𝑃 ( 𝐴 𝑈 𝐵 ) − 𝑃 ( 𝐴 ) + 𝑃 ( 𝐴 ∩ 𝐵 )=0.69− 0.4+0.14=0.43

 
 Are Mutually Exclusive Events Independent?
S
B
Are A and B
A
independent?

Independence means that having information about one

event, doesn’t affect probability of another event [P(A|
B)=P(A)]
If we know that event B happened does that tell us anything
about event A?
 Are Mutually Exclusive Events Independent?
S
B
Are A and B
A
independent?

Independence means that having information about one

event, doesn’t affect probability of another event [P(A|
B)=P(A)]
If we know that: P(A)>0, P(B)>0 and P(A∩B)=0 and that event
B happened does that tell us anything about event A?
P(A|B)=0; P(A)>0; so A and B are not independent
Disjoint doesn’t mean independent
Are Mutually Exclusive Events Independent?

Do not confuse the notion of mutually exclusive

events with that of independent events.

Two events with nonzero probabilities cannot be

both mutually exclusive and independent.

If one mutually exclusive event is known to occur,

the other cannot occur.; thus, the probability of the
other event occurring is reduced to zero (and they
are therefore dependent).

Two events that are not mutually exclusive, might

or might not be independent.
End of Chapter 4