Probability

Ravindra S. Gokhale IIM Indore

It is a truth very certain that, when it is not in our power to determine what is true, we ought to follow what is most probable
- Rene Descartes

Probability Models
Concerned with the study of random (or chance) phenomena Random experiment: An experiment that can result in different outcomes, even though it is repeated in the same manner every

time
Managerial Implication: Due to this random nature additional capacities for any setup are required

Although random, certain statistical regularities are to be captured to form the model Model: An abstraction of real world problem

Sample Space
A set of all possible outcomes of a random experiment Denoted by S Examples:

Tossing of a coin: S = {Head, Tail}

Throwing of a dice: S = {1, 2, 3, 4, 5, 6} Marks of a student in a 20 mark paper with only integer negative marking

possible: S = {-20, -19, , -1, 0, 1, , 19, 20}

Number of people arriving at a bank in a day: S = {0, 1, 2, } Inspection of parts till one defective part is found: S = {d, gd, ggd, gggd, } Temperature of a place with a knowledge that it ranges between 10 degrees and 50 degrees: S = {any value between 10 to 50}

Speed of a train at a given time, with no other additional information: S = {any value between 0 to infinity}
4

Sample Space (cont)

Discrete sample space: One that contains either finite or countable infinite set of outcomes

Out of the previous examples, which ones are discrete sample spaces???

Continuous sample space: One that contains an interval of real

numbers. The interval can be either finite or infinite

Events
A collection of certain sample points A subset of the sample space Denoted by E Examples:

Getting an odd number in dice throwing experiment S = {1, 2, 3, 4, 5, 6}; E = {1, 3, 5} Getting a defective part within the first three inspected parts S = {d, gd, ggd, gggd, ggggd, }; E = {d, gd, ggd} Event of a component not failing before time t: S = [0, ); E = [t, )

Events (cont)
Complement of an event E1: E1 = S E1 Intersection of two events: E3 = E1 Union of two events: E4 = E1 U E2 U Two events A and B are mutually exclusive if: A B= U E2

Union and Intersection can be extended to more than two events

Two events A and B are mutually exclusive and exhaustive if:

A

B = and A U B = S

Graphical Representation of Events

Venn Diagram S A B A
A

S B U
B

Disjoint events A and B

S A
AUB

B D

Mutually exclusive and exhaustive events: A, B, C, and D

Graphical Representation of Events (cont)

Tree Diagram

Defective

NotDefective

Head

Tail

Defective

NotDefective

Head

Tail Head

Tail

Defective

NotDefective

Probability
Probability of an event represents the relative likelihood that performance of the experiment will result in occurrence of that event

P(A) = Probability of the event A in the sample space

Examples:

A = Getting an odd number in dice throwing experiment

P(A) = 0.5
B = Getting two heads in two successive tosses of a coin P(B) = 0.25

10

Concept of Sample and Population

A population is the entire collection units in which we are interested

A sample is a subset of a population

The objective of inferential statistics is to make an inference about a population of interest based on the information obtained from the sample from that population

11

Probability in the Context of Sample and Population

Proportion Sample

Probability

Population

12

The Approaches for Determining Probability

Classical Approach: probability is based on the idea that certain occurrences are equally likely (e.g. throwing of dice)

Relative Frequency Approach: probability based on long term relative frequency --- requires a large amount of historical data

Subjective Probability: probability based on ones judgement (e.g. how may days will it take for a consignment to reach from Indore to Delhi?)

13

The Three Axioms of Probability

For any event A, 0 <= P(A) <= 1 P(S) = 1 If A and B are mutually exclusive events then, P(A U B) = P(A) + P(B)

Some subsequent results:

P(E1) = 1 P(E1) If event E1 is contained in event E2, then

P(E1) <= P(E2)

14

Addition Rules
If A and B are any two events then, P(A U B) = P(A) + P(B) P(A U B)

If A, B and C are any three events then,

P(A U B U C) = P(A) + P(B) + P(C) P(A B) P(A C) P(B C) + P(A

U U
B

C)

15

An Example

16

An Example
What is the sample space? Which are the simple events? What is the probability of each simple event?

What is the probability that your slice will have tomato or

mushroom (or both)?

What is the probability that your slice will have tomato and mushroom? What is the probability that your slice will not have tomato?

17

An Example
You have pulled out a slice of pizza that has tomato on it. What is the probability that your slice will have mushrooms?

This is called as Conditional Probability

18

Conditional Probability
The conditional probability of A given B is:

if P(B) 0; else it is undefined

P(A|B) = P(A

B) / P(B)

19

Multiplication Rule
The conditional probability rule can be rewritten to get the multiplication rule: U P(A B) = P(A|B) x P(B) = P(B|A) x P(A)

20

Example
A consulting firm is bidding for two jobs, one with each of two large multinational corporations. The company executives estimate that the probability of obtaining the consulting job with

firm A, event A, is 0.45. The executives also feel that if the

company should get the job with firm A, then there is a 0.90 probability that firm B will also give the company the consulting job. What are the companys chances of getting both jobs?

21

Example
In a standard government run lottery in Europe and North America, you choose 6 out of 49 numbers (1 through 49). You win the biggest prize if these 6 are drawn. (The prize money is divided

between all those who choose the lucky numbers. If no one wins,
then most of the prize money is put back into next week's lottery). You are offered two tickets A = (1,2,3,4,5,6) or B = (39,36,32,21,14, and 3). Do you prefer A, B, or are you indifferent between the two?

22

An Example

23

Independence
Two events are said to be independent if occurrence of one has no effect on the chances for the occurrence of the other.

Using the Statistically Independent Pizza, are events mushroom

and tomato independent?

24

Independence
Two events A and B are independent if any one of the following equivalent statements are true: P(A|B) = P(A)

P(B|A) = P(B)
U P(A B) = P(A) x P(B)

If a list of events is mutually independent, the probability of their

intersection is the product of their probabilities

25

More about Statistical Independence

Assume that biologically, the probability of giving birth to a girl is 0.5 and the probability of giving birth to a boy is 0.5

A couple has two girls. What is the chance of delivering a baby girl
for the third time?

Does this imply statistical independence?

26

More about Statistical Independence

In many practical business scenarios, the assumption of statistical independence may not be valid. Loan default example: A bank has assigned probabilities of default to three different loans: P(Loan 1 defaults) = p1 P(Loan 2 defaults) = p2

P(Loan 3 defaults) = p3
If the three loans are from three different companies, but in the same business, example suppliers of auto parts. Loan repayment problem with one of the customers is very likely an indication of similar problems with remaining two customers. Is the assumption of independence valid in such a case?
27

More about Statistical Independence

Dependence may be built in purposely Advertising campaign example: The aim of any advertising campaign is to attract potential customers and boost up the sales Suppose, a company has launched a new advertisement for a product Event A = Potential customer sees an advertisement for a product Event B = Potential customer buys the product You can estimate P(B) from the historical data What can you say about the estimate for P(B|A)? Is the independence of events desirable in such a case?

28

More about Statistical Independence

Consider the statement: If two events are mutually exclusive then they are

independent.

The above statement is FALSE

In fact, the exact reverse is true, that is,

If two events are mutually exclusive then they can never be

independent.

29

More about Statistical Independence

Conditional Independence: Two events A and B are said to be conditionally independent, given another event C with P(C) > 0, if P(A B |C) = P(A|C) x P(B |C)

Independence does not imply conditional independence, and

vice versa. Hint example:

A = {1st toss result is a tail},

B = {2nd toss result is a tail}, C = {the two tosses have different results}.
30

More about Statistical Independence

Independence of three events: If we have a collection of three events, A1, A2, and A3, independence amounts to satisfying all the four conditions:
P(A1 A2) = P(A1) x P(A2), P(A1 A3) = P(A1) x P(A3), P(A2 A3) = P(A2) x P(A3),

P(A1 A2 A3) = P(A1) x P(A2) x P(A3).

only then the three events are said to be independent events.

First three conditions simply assert that any two events are independent, a property known as pairwise independence. The fourth condition is also important and does not automatically follow from the first three. Conversely, the fourth condition does not imply the first three
31

More about Statistical Independence

Independence of three events: Hint examples: A1 = {1st toss result is a tail}, A2 = {2nd toss result is a tail}, A3 = {the two tosses have different results} Exercise: Verify that the first three conditions (pairwise independence) are satisfied, but the fourth is not satisfied. A1 = {Roll of first dice is 1, 2, or 3}, A2 = {Roll of first dice is 3, 4, or 5}, A3 = {Sum of rolls of first dice and second dice is 9} Exercise: Verify that the fourth condition is satisfied, but the pairwise independence is not satisfied.
32

Joint and Marginal Probabilities

Example: Does education really give you a better income? Annual household income and education in US 2000 census

Education

Poor (<$25K) 10.51 10.93 6.36 2.23 0.62 0.16 0.09 30.9

Lower Middle ($25K to ($50K to $50K) $75K) 4.26 10.57 8.36 4.16 1.26 0.22 0.21 29.04 1.32 5.84 5.84 4.15 1.36 0.29 0.29 19.09

Upper (>$75K) 0.68 3.87 5.23 6.57 2.84 0.99 0.80 20.98

Row total

No HS diploma High school grad. Some college Bachelors deg. Masters deg. Professional deg. Doctorate Column total

16.77 31.21 25.79 17.11 6.08 1.66 1.39 100

All cells show percentages Sample size is 98 million households Source: Statistics for Business Decision Making and Analysis by Stine and Foster 33

Law of Total Probability

For any two events A and B:

= P(B) x P(A|B) + P(Bc) x P(A|Bc)

In general, Given k mutually exclusive and exhaustive events S1, S2, Sk and any other event A, the probability of event A can be expressed as:

= P(S1) x P(A|S1) + P(S2) x P(A|S2) + + P(sk) x P(A|Sk)

P(A) = P(A

S1) + P(A

P(A) = P(A

B) + P(A

Bc)

S2) + + P(A

Sk )

34

An Example towards Bayes Rule

A petroleum company uses a specialized equipment for sensing the presence of oil wells in a region, which can then be drilled to extract the oil. The machine can accurately sense the presence of

oil for 95% of the times. However, it also gives a false-positive

result (that is incorrectly showing the presence of oil when in fact there is none) for 25% of the times. It is estimated that 10% of the region under consideration actually has oil underneath. For a certain survey, if the machine has given a positive result for a certain area under consideration within the region, what is the probability that the oil is actually present in that area?

35

Bayes Rule

36

Bayes Rule
P(Si) is called as the prior probability of Si because it does not take into account any other behaviour and is based on historical data

In practice, Bayes Rule can be applied to re-evaluate (or update) the prior belief of probabilities of S1, S2, , Sk, when new information is obtained (as a result of event A happening)

37

Bayes Rule Some More Examples

ABC Plastics specializes in manufacturing of plastic bottles. For this purpose, it has two production facilities in two different cities say City 1 and City 2. The plant in City 1 contributes to 60% of the total annual

output, and that in City 2 the remaining 40%. The central quality
assurance reported that 5% of the bottles produced in City 1 plant were defective. Likewise, 10% of the bottles from the City 2 plant were defective. The Controller of Operations for ABC Plastics wants to apportion

the cost of poor quality fairly between the two plants. How can that be
done?

38

Bayes Rule Some More Examples

A startup software development firm has prepared a customized software application for a large business organization. The sales manager of the startup firm has a hunch that the probability of selling the application is

60%.
Usually in this kind of process, after making an initial presentation to the potential client by the vendor, the client organization most often asks the vendor to make a repeat presentation to the higher authorities from the

client side. Historically suppose for this particular client, 70% of the
successful vendors had been asked to do a second presentation. However, 50% of the unsuccessful vendors were also asked to do the second presentation. Suppose the startup firm in question has been asked to do a second presentation. What is the revised estimate of the probability of selling the application using the additional information in the previous paragraph?
39

Bayes Rule Some More Examples

You are a physician. You think it is quite likely that one of your patients has strep throat, but you are not sure. You take some swabs from the throat and send them to a lab for testing. The test is (like nearly all lab tests) not perfect. If the patient has strep throat, then 70% of the time the lab says YES. but 30% of the time it says NO. If the patient does not have strep throat, then 90% of the time the lab says NO. But 10% of the time it says YES. You send five successive swabs to the lab, from the same patient. You get back these results, in order; YNYNY. What do you conclude among the following? 1. These results are worthless 2. It is likely that the patient does not have the strep throat.

3. It is slightly more likely than not, that patient does have the strep throat.
4. It is very much more likely than not, that patient does have the strep throat.

40

Law of Total Probability An Interesting Example

In the Monty Hall three doors problem, a contestant on a game show is asked to choose between three doors. Behind two of the doors, there is

sand. Behind the third door, there is the prize, a car. Once the contestant
has picked one of the doors, the host of the game show (who knows where the prize is) opens one of remaining doors and shows that there is sand behind it. The host then gives the contestant a chance to switch to

the remaining unopened door. What should the contestant do, continue
with the same door or switch to the remaining door?

41

Additional Examples
Samples of emission from three suppliers are classified for conformance to air-quality specifications. The results from 100 samples are summarized as follows:

Confirms
Supplier 1 Supplier 2 Supplier 3 22 25 30

Not confirms
8 5 10

Let A denote the event that the sample is from supplier 2 and let B denote the event that the sample confirms the specifications. If a sample is selected at random, determine the following probabilities:

(a) P(A),

(b) P(B),
B), (e) P(A U B),

(c) P(A),
(f) P(A U B)

(d) P(A

42

Additional Examples
A lot contains 15 castings from a local supplier and 25 castings from a supplier from the next state. Two castings are selected randomly without replacement from the lot of 40. Let event A be

that the first casting is selected from the local supplier and the
event B be that the second casting is selected from the local supplier. Determine the following:

P(A

P(A U B)

P(A) P(B|A) B)

43

Additional Examples
Software to detect fraud in customer phone cards tracks the number of metropolitan areas where the calls originate each day. It is found that 1% of legitimate callers originate calls from more

than one metropolitan areas in a single day. As against these,

30% of the fraudulent users originate calls from more than one metropolitan areas in a single day. The proportion of fraudulent users is 0.01%. If a same user originates calls from more than one metropolitan area in a single day, what is the probability that the user is fraudulent?

44

Additional Examples
A business group owns three five-star hotels (say, A, B, and C) in India. By studying the past behavior of the revenue obtained from the three hotels month by month, it has been observed that the probability of

increase in revenue of either B or C or both of them is 0.5. If As

revenue increases in a given month, the probability of increase in Bs revenue is 0.7, the probability of increase in Cs revenue is 0.6, and the probability of increase in both B and Cs revenue is 0.5. However if As

revenue does not increase in a given month, the probability of increase

in Bs revenue is 0.2, the probability of increase in Cs revenue is 0.3, and the probability of increase in both B and Cs revenue is 0.1. What is the probability that the revenue of all the three hotels, A, B, and C

increases in a given month.

45