Basic Probability

Sanjay Rastogi, IIFT, New Delhi 1

 Important Terms
• Probability – the chance that an uncertain
event will occur (always between 0 and 1)
• Event – Each possible outcome of a variable
• Sample Space – the collection of all possible
events

Sanjay Rastogi, IIFT, New Delhi 2

 Classical Probability
• Number of outcomes leading
to the event divided by the n
P( E )  e
total number of outcomes N
possible Where:
• Each outcome is equally likely N  total number of outcomes
• Determined a priori -- before
performing the experiment
ne
 number of outcomes in E

• Applicable to games of chance

• Objective -- everyone correctly
using the method assigns an
identical probability

Sanjay Rastogi, IIFT, New Delhi 3

 Relative Frequency Probability
• Based on historical
data
• Computed after P( E )  n
e

performing the N
experiment Where:
• Number of times an N  total number of trials
event occurred divided
by the number of trials n e
 number of outcomes
• Objective -- everyone producing E
correctly using the
method assigns an
identical probability

Sanjay Rastogi, IIFT, New Delhi 4

 Subjective Probability
• Comes from a person’s intuition or reasoning
• Subjective -- different individuals may (correctly)
assign different numeric probabilities to the same
event
• Degree of belief
• Useful for unique (single-trial) experiments
– New product introduction
– Initial public offering of common stock
– Site selection decisions
– Sporting events

Sanjay Rastogi, IIFT, New Delhi 5

 Sample Space
The Sample Space is the collection of all
possible events
e.g. All 6 faces of a die:

e.g. All 52 cards of a bridge deck:

Sanjay Rastogi, IIFT, New Delhi 6

 Events
• Simple event
– An outcome from a sample space with one characteristic
– e.g., A red card from a deck of cards
• Complement of an event A (denoted A’)
– All outcomes that are not part of event A
– e.g., All cards that are not diamonds
• Joint event
– Involves two or more characteristics simultaneously
– e.g., An ace that is also red from a deck of cards

Sanjay Rastogi, IIFT, New Delhi 7

 Visualizing Events
• Contingency Tables
Ace Not Ace Total

Black 2 24 26
Red 2 24 26

Total 4 48 52

• Tree Diagrams Sample

Ac e 2 Space
Sample
Space k C a rd
Full Deck Blac Not an Ace 24
of 52 Cards Ac e
Red C 2
ard
NoDelhi
Sanjay Rastogi, IIFT, New t an 8
Ace 24
 Visualizing Events
• Venn Diagrams
– Let A = aces
– Let B = red cards A ∩ B = ace and red

A U B = ace or red B

Sanjay Rastogi, IIFT, New Delhi 9

 Mutually Exclusive Events
• Mutually exclusive events
– Events that cannot occur together

example:

A = queen of diamonds; B = queen of clubs

– Events A and B are mutually exclusive

Sanjay Rastogi, IIFT, New Delhi 10
 Collectively Exhaustive Events
• Collectively exhaustive events
– One of the events must occur
– The set of events covers the entire sample space

example:
A = aces; B = black cards;
C = diamonds; D = hearts

– Events A, B, C and D are collectively exhaustive (but

not mutually exclusive – an ace may also be a heart)
– Events B, C and D are collectively exhaustive and also
mutually exclusive
Sanjay Rastogi, IIFT, New Delhi 11
 Probability
• Probability is the numerical measure
of the likelihood that an event will
occur 1 Certain
• The probability of any event must be
between 0 and 1, inclusively
0 ≤ P(A) ≤ 1 For any event A
0.5
• The sum of the probabilities of all
mutually exclusive and collectively
exhaustive events is 1
P(A)  P(B)  P(C)  1
0 Impossible
If A, B, and C are mutually exclusive and
collectively exhaustive
Sanjay Rastogi, IIFT, New Delhi 12
 Computing Joint and
Marginal Probabilities
• The probability of a joint event, A and B:
number of outcomes satisfying A and B
P ( A and B) 
total number of elementary outcomes

• Computing a marginal (or simple) probability:

P(A)  P(A and B1 )  P(A and B 2 )    P(A and Bk )

• Where B1, B2, …, Bk are k mutually exclusive and collectively

exhaustive events

Sanjay Rastogi, IIFT, New Delhi 13

Joint Probability Example

P(Red and Ace)

number of cards that are red and ace 2
 
total number of cards 52

Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
Sanjay Rastogi, IIFT, New Delhi 14
 Marginal Probability Example
P(Ace)
2 2 4
 P( Ace and Re d)  P( Ace and Black )   
52 52 52

Color
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
Sanjay Rastogi, IIFT, New Delhi 15
 Joint Probabilities Using
Contingency Table

Event
Event B1 B2 Total
A1 P(A1 and B1) P(A1 and B2) P(A1)
A2 P(A2 and B1) P(A2 and B2) P(A2)

Total P(B1) P(B2) 1

Joint Probabilities Marginal (Simple) Probabilities

Sanjay Rastogi, IIFT, New Delhi 16

 General Addition Rule

General Addition Rule:

P(A or B) = P(A) + P(B) - P(A and B)

If A and B are mutually exclusive, then

P(A and B) = 0, so the rule can be simplified:
P(A or B) = P(A) + P(B)
For mutually exclusive events A and B

Sanjay Rastogi, IIFT, New Delhi 17

 General Addition Rule Example

P(Red or Ace) = P(Red) +P(Ace) - P(Red and Ace)

= 26/52 + 4/52 - 2/52 = 28/52

Don’t count
the two red
Color aces twice!
Type Red Black Total
Ace 2 2 4
Non-Ace 24 24 48
Total 26 26 52
Sanjay Rastogi, IIFT, New Delhi 18
 Computing Conditional
Probabilities
• A conditional probability is the probability of one
event, given that another event has occurred:
P(A and B) The conditional
P(A | B)  probability of A given
P(B) that B has occurred

P(A and B) The conditional

P(B | A)  probability of B given
P(A) that A has occurred

Where P(A and B) = joint probability of A and B

P(A) = marginal probability of A
P(B) = marginal probability of B
Sanjay Rastogi, IIFT, New Delhi 19
 Conditional Probability Example
• Of the cars on a used car lot, 70% have air
conditioning (AC) and 40% have a CD player
(CD). 20% of the cars have both.

• What is the probability that a car has a CD player,

given that it has AC ?

i.e., we want to find P(CD | AC)

Sanjay Rastogi, IIFT, New Delhi 20

 Conditional Probability Example
(continued)
• Of the cars on a used car lot, 70% have air
conditioning (AC) and 40% have a CD player (CD).
20% of the cars have both.

CD No CD Total
AC 0.2 0.5 0.7
No AC 0.2 0.1 0.3
Total 0.4 0.6 1.0

P(CD and AC) 0.2

P(CD | AC)    0.2857
P(AC) 0.7
Sanjay Rastogi, IIFT, New Delhi 21
 Conditional Probability Example
(continued)
• Given AC, we only consider the top row (70% of the cars). Of
these, 20% have a CD player. 20% of 70% is about 28.57%.

CD No CD Total
AC 0.2 0.5 0.7
No AC 0.2 0.1 0.3
Total 0.4 0.6 1.0

P(CD and AC) 0.2

P(CD | AC)    0.2857
P(AC) 0.7
Sanjay Rastogi, IIFT, New Delhi 22
 Using Decision Trees
.2
D .7 P(AC and CD) = 0.2
Given AC or Has
C
no AC: 0 .7
C)= D oe
P( A s
have not .5 P(AC and CD’) = 0.5
s AC CD
Ha .7
All
Cars
Doe .2
hav s not .3
eA
C P(A C D P(AC’ and CD) = 0.2
C’) Has
=0
.3
D oe
s
have not .1 P(AC’ and CD’) = 0.1
CD
Sanjay Rastogi, IIFT, New Delhi 23
.3
 Using Decision Trees
(continued)
.2
.4 P(CD and AC) = 0.2
Given CD or a s AC
H
no CD: 0 .4
D ) = D oe
P(C s
have not .2 P(CD and AC’) = 0.2
s CD AC
Ha .4
All
Cars
Doe .5
hav s not .6
eC AC P(CD’ and AC) = 0.5
D P(C Has
D’)
=0
.6
D oe
s
have not .1 P(CD’ and AC’) = 0.1
AC
Sanjay Rastogi, IIFT, New Delhi 24
.6
 Statistical Independence
• Two events are independent if and
only if:
P(A | B)  P(A)

• Events A and B are independent when the

probability of one event is not affected by the
other event
Sanjay Rastogi, IIFT, New Delhi 25
 Multiplication Rules
• Multiplication rule for two events A and B:

P(A and B)  P(A | B) P(B)

Note: If A and B are independent, then P(A | B)  P(A)

and the multiplication rule simplifies to
P(A and B)  P(A) P(B)
Sanjay Rastogi, IIFT, New Delhi 26
 Marginal Probability
• Marginal probability for event A:

P(A)  P(A | B1 ) P(B1 )  P(A | B2 ) P(B2 )    P(A | Bk ) P(Bk )

– Where B1, B2, …, Bk are k mutually exclusive and

collectively exhaustive events

Sanjay Rastogi, IIFT, New Delhi 27

 Bayes’ Theorem
Bayes’ theorem is used to revise previously calculated
probabilities after new information is obtained
P(A | B i )P(B i )
P(B i | A) 
P(A | B 1 )P(B 1 )  P(A | B 2 )P(B 2 )    P(A | B k )P(B k )
• where:
Bi = ith event of k mutually exclusive and collectively
exhaustive events
A = new event that might impact P(Bi)

Sanjay Rastogi, IIFT, New Delhi 28

 Bayes’ Theorem Example
• A drilling company has estimated a 40% chance of
striking oil for their new well.
• A detailed test has been scheduled for more
information. Historically, 60% of successful wells
have had detailed tests, and 20% of unsuccessful
wells have had detailed tests.
• Given that this well has been scheduled for a
detailed test, what is the probability
that the well will be successful?
Sanjay Rastogi, IIFT, New Delhi 29
 Bayes’ Theorem Example
(continued)
• Let S = successful well
U = unsuccessful well
• P(S) = 0.4 , P(U) = 0.6 (prior probabilities)
• Define the detailed test event as D
• Conditional probabilities:
P(D|S) = 0.6 P(D|U) = 0.2
• Goal is to find P(S|D)
Sanjay Rastogi, IIFT, New Delhi 30
 Bayes’ Theorem Example
(continued)

Apply Bayes’ Theorem:

P(D | S)P(S)
P(S | D) 
P(D | S)P(S)  P(D | U)P(U)
(0.6)(0.4)

(0.6)(0.4)  (0.2)(0.6)
0.24
  0.667
0.24  0.12
So the revised probability of success, given that this
well has been scheduled for a detailed test, is 0.667
Sanjay Rastogi, IIFT, New Delhi 31
 Bayes’ Theorem Example
(continued)

• Given the detailed test, the revised

probability of a successful well has risen to
0.667 from the original estimate of 0.4

Prior Conditional Joint Revised

Event Prob.
Prob. Prob. Prob.
S (successful) 0.4 0.6 (0.4)(0.6) = 0.24/0.36 = 0.667
0.24
U (unsuccessful) 0.6 0.2 (0.6)(0.2) = 0.12/0.36 = 0.333
Sum
0.12= 0.36

Sanjay Rastogi, IIFT, New Delhi 32