ELEMENTARY PROBABILITY THEORY

PRIVATUS NTARE
November 9, 2024

1
1 Introduction
Definition 1.1. Probability is the measure of likelihood that an event will occur.

It is an important part of mathematics and statistics. It is the basis of inferential statistics.

In inferential statistics we make decision under condition of uncertainty. Probability theory is
used to evaluate the uncertainty involved in those decision. For example, estimating next year’s
revenue for a business company is based on many assumptions, some of which may happen to be
true and others may not. Probability theory will help us to make decision under such conditions
of imperfect information and uncertainty. Discussing probability an probability distributions
together with descriptive statistics will help us make decisions (inference) about the population
based on the information( data)obtained from samples.

2 Probability Concepts
The term probability can be defined in two major perspectives; the empirical and theoretical
concepts.

2.1 Empirical Perspectives

Based on this, probability is defined as the relative frequency; that is the quotient of the total
number of occurences of the situation to the total number of the times an experiment was
repeated. The number of repetition should be large enough to justify the concepts. That is ,
if fx is the number of times a situation x has occurred out of n repeated trials, then we define
the probability of an event x to occur as
!
fx
P (X = x) = x→∞
lim (1)
n

Sometimes it is difficult to obtain probabilities from the practical experiment. In this case we
can use the previous experiences to assign the probability to a given event under study. This
kind of probability is known as subjective probability.

2.2 Theoretical Concept

Definition of some important terms:

Definition 2.1. An experiment is a planned operation carried out under controlled condi-
tions. eg flipping the fair coin twice.

We can simply say that the experiment is the process that, when performed results in one
and only one of many observations. These observations are called the outcome of experiment.

Definition 2.2. Outcome is a result of an experiment.

The collection of all outcomes for an experiment is called the sample space.

Definition 2.3. A sample space is the set consisting of all sample points.

It is denoted by S. Consider case of tossing the fair coin twice with H representing the
heads shown up and T representing the number of tails. In this case S is given by

S = {HH, HT, T H, T T }

2
Definition 2.4. A sample point is an individual element of the sample space.
It is denoted by e. From the above experiment e1 = {HH} , e2 = {HT }

Table 1: Examples of experiments, Outcomes and Sample Space

Experiment Outcomes Sample Space
Toss a coin once Head, Tail S = {Head, Tail}
Toss a coin twice HH, HT, TH, TT S = {HH, HT, T H, T T }
Roll a die once 1,2,3,4,5,6 S = {1, 2, 3, 4, 5, 6}
Take a test Pass, Fail S = {Pass, Fail}
Play a lottery Win, Lose S = {Win, Lose}
Select a worker Male, Female S = {Male, Female}

CASE 1: Equally Likely Outcomes

Two or more outcomes (or event) that have the same probability of occurrence are said to
be equally likely outcomes(or events). Let E be any events whose members is a subset of
S with equally likely points. In this case , the probability of event E is given by

n(E)
P (E) = (2)
n(S)

where n(E) and n(S) are the number of elements in E and S respectively.
CASE 2: Not Equally Likely Outcomes
In this case the probability of a given event E, can be obtained by adding the probabilities
of individual points in E.
The sample space for the experiments can be illustrated by using the Venn diagram or a
tree diagram. In a tree diagram each outcome is represented by the branch of tree. Both
tree and Venn diagram help us to understand the probability concepts by presenting them
visually.
Example 1.
Draw the tree diagram for the experiment of tossing a coin twice.
Solution

Figure 1: Tree diagram for the coin tossed twice

3
Example 2.
A football team has to play two matches to qualify for the second round. There is 0.7 chance
that it will win the first match an 0.8 chance of winning the second match. Fin the probability
that it will win

(a) Only one match

(a) At least one match

Solution Let

W1 =a team wins the first match

L1 =a team loses the first match
W2 =a team wins the second match
L2 =a team loses the second match

From the given information, we find that

P (W1 ) = 0.7, P (L1 ) = 0.3, P (W2 ) = 0.8, P (L2 ) = 0.2
The sample space can be obtained from the tree diagram as shown below

The sample space S = {W1 W2 , W1 L2 , L1 W2 , L1 L2 }

(a) If A is the event that a team wins only one match, then

A = {W1 L2 , L1 W2 }
P (A) =P (W1 )P (L2 ) + P (L1 )P (W2 )
P (A) =0.72 × 0.2 + 0.3 × 0.8 = 0.38

(b) If B is the event that a team wins one match , then

B = {W1 W2 , W1 L2 , L1 W2 }
P (B) =P (W1 )P (W2 ) + P (W1 )P (L2 ) + P (L1 )P (W2 )
P (B) =0.7 × 0.7 × 0.72 × 0.2 + 0.3 × 0.8 = 0.94

4
2.3 Axioms of Probabilities
Probability which gives the likelihood of occurence of an event, is denoted by P . The probability
that a simple event Ei will occur is denoted by P (Ei ), and the probability that a compound
event A will occur is denoted by P (A).

1. The probability of an event always lie in range 0 to 1.

Whether it is a simple or compound event, the probability of an event is never less than
0 or greater than 1.Mathematically can be written as

0 ≤ P (Ei ) ≤ 1
0 ≤ P (A) ≤ 1

An event that cannot occur has the zero probability ; such an event is called an impossi-
ble event.An event that is certain to occur has the probability equal to 1 is called sure
event. That is

For an impossible event M :P (M ) = 0

For a sure event C :P (C) = 1

2. The sum of the probabilities of all simple events (or final outcomes) for an
experiment is always 1
For an experiment,

P (Ei ) = P (E1 ) + P (E2 ) + P (E3 ) · · · = 1

X

From this property, for the experiment of tossing a coin once

P (H) + P (T ) = 1

For the professional football game

P (win) + P (loss) + P (tie) = 1

2.4 Rules of Probabilities

(a) Addition Rule
This rule states that if A and B are two events, then

P (A ∪ B) = P (A) + P (B) − P (A ∩ B). (3)

(b) The Multiplication Rule

If A and B are two events defined on a sample space, then:

P (AandB) ≡ P (A ∩ B) = P (B)P (A|B). (4)

P (A ∩ B)
This also is written as P (A|B) = .
P (B)
If A and B are independent, becomes P (A ∩ B) = P (A)P (B).

5
 (c) Complement Rule
′
Let A denotes that an event A can not occur. Then ,
′
P (A ) = 1 − P (A). (5)

2.5 Contingency Tables

Definition 2.5. A contingency table provides a way of portraying data that can facilitate
calculating probabilities
The contingency table helps in determining conditional probabilities quite easily. The table
displays sample values in relation to two different variables that may be dependent or contin-
gent on one another. Later on, we discuss the marginal andd conditional probability and use
contingency tables to calculate the probabilities.

2.6 Marginal, Conditional and Joint Probability

Probabilities may be either marginal, joint or conditional. Understanding their differences
and how to manipulate among them is key to success in understanding the foundations of
statistics.

2.6.1 Marginal Probability

Definition 2.6. Marginal probability is the probability of a single event without consideration
of any other event. Marginal probability is also called simple probability.
Is simply the probability of an event irrespective of the outcome of another variable.
Marginal probability may be thought of as an unconditional probability since is not de-
pending on another event.

Suppose 100 students from CBE were asked whether they like or hate football. Table 2
gives a two-way classification of the responses of these 100 students.

Table 2: Two-Way Classification of Students Responses

Like Hate
Male 15 45
Female 4 36

Table 2 shows the distribution of 100 students based on two variables or characteristics:
gender (male or female) and opinion (in like or hate). Such a table is called a contingency
table. In Table 2, each box that contains a number is called a cell. Notice that there are
four cells. Each cell gives the frequency for two characteristics. For example, 15 students in
this group possess two characteristics: “male” and “in like of football.” We can interpret the
numbers in other cells the same way.
By adding the row totals and the column totals to Table 2, we write Table 3.

Table 3: Two-Way Classification of Students Responses with Totals

Like Hate Total
Male 15 45 60
Female 4 36 40
Total 19 81 100

6
 Suppose one student is selected at random from these 100 Students. This student may be
classified either on the basis of gender alone or on the basis of opinion. If only one character-
istic is considered at a time, the students selected can be a male, a female, Like, or hate. The
probability of each of these four characteristics or events is called marginal probability or
simple probability because they are not conditioned on another event.

For Table 3, the four marginal probabilities are calculated as follows:

Number of male 60
P (male) = = = 0.6
Total number of students 100
Number of female 40
P (female) = = = 0.4
Total number of students 100
Number of students who like 19
P (Like) = = = 0.19
Total number of students 100
Number of students who hate 81
P (Hate) = = = 0.81
Total number of students 100
Now suppose that one student is selected at random from these 100 students. Also, assume
it is known that this (selected) students is a male. In other words, the event that the student
selected is a male has already occurred. What is the probability that the student selected likes
football? This probability is written as P (Like|Male). This probability, P (Like|Male) is called
the conditional probability of “Like” given that the event “male” has already happened. It
is read as “the probability that the student selected is in like of football given that this students
is a male”.

2.6.2 Conditional Probability

Definition 2.7. Conditional probability is the probability that an event will occur given that
another event has already occurred.

The conditional probability of A given B is written P (A|B). P (A|B) is read as “the

probability of A given that B has already occurred”. Such events events have the property of
depending one another.
Example of conditional events is like; enrolment of students in basic certificate and perfor-
mance in form four exam. The formula to calculate P (A|B) is given by

P (A ∩ B)
P (A|B) = , P (B) ≥ 0, (6)
P (B)
Similarly,
P (B ∩ A)
P (B|A) = , P (A) ≥ 0, (7)
P (A)

Example 3. The probability that a regularly scheduled flight departs on time is P (B) = 0.83;
the probability that it arrives on time is P (A) = 0.82; and the probability that it departs and
arrives on time is P (B ∩ A) = 0.78.Find the probability that a plane:

(a) arrives on time given that it departed on time,

(b) departed on time given that it has arrived on time,

(c) arrives on time, given that it did not depart on time.

Solution

7
 (a) The probability that a plane arrives on time given that it departed on time is

P (A ∩ B) 0.78
P (A|B) = = = 0.94
P (B) 0.83

(b) The probability that a plane departed on time given that it has arrived on time is

P (B ∩ A) 0.78
P (B|A) = = = 0.95
P (A) 0.82

(c) The probability that a plane arrives on time, given that it did not depart on time. P (A|B ′ )

P (A ∩ B ′ ) P (A) − P (A ∩ B) 0.82 − 0.78

P (A|B ′ ) = = = = 0.24.
P (B ′ ) 1 − P (B) 1 − 0.83

The notion of conditional probability provides the chance of re-evaluating the idea of
probability of an event in the light of additional information, that when you get a new
information about an event how you change your probability (opinion). The probability
P (A|B) is an updating of P (A) based on the knowledge that event B has occurred.

2.6.3 Joint Probability

Joint probability is a statistical measure that calculates the likelihood of two events occurring
together and at the same point in time.

Definition 2.8. Joint Probability: The probability of the intersection of two events is called
their joint probability. The joint probability of two events A and B It is written as P (A ∩ B)
or P (AB)

The joint probability of two events is obtained by multiplying the marginal probability of one
event by the conditional probability of the second event. This rule is called the multiplication
rule.

Definition 2.9. Multiplication Rule to Find Joint Probability: The probability of the
intersection of two events A and B is given by

P (A ∩ B) = P (B)P (A|B).

Example 4. Table 4 gives the classification of CBE staffs by gender and degree.

Table 4: Classification of CBE staffs by Gender and Education

Graduate (G) Postgraduate (T ) Total
Male (M ) 7 20 27
Female (F ) 4 9 13
Total 11 29 40

If one of these staff member is selected at random for membership on the research committee,
what is the probability that this staff is

(a) a female and a graduate?

(b) a male and graduate?

8
 (c) a male and postgraduate?

(d) a female and postgraduate ?

Solution

(a) From Table 4, the probability that a female is selected is

Number of female 13
P (F ) = =
Total number of staffs 40
To calculate the probability P (G|F ), we know that F has already occurred. Consequently,
the staff selected is one of the 13 females. In the table, there are 4 graduates staffs among
13 female staff. Hence, the conditional probability of G given F is
4
P (G|F ) =
13
The joint probability of F and G is
13 4
  
P (F ∩ G) = P (F )P (G|F ) = = 0.1
40 13
The probability that a randomly selected staff is a female and graduate is 0.1

(b)
27 7
  
P (M ∩ G) = P (M )P (G|M ) = = 0.175
40 27
The probability that a randomly selected staff is a male and graduate is 0.175

(c)
27 20
  
P (M ∩ T ) = P (M )P (T |M ) = = 0.5
40 27
The probability that a randomly selected staff is a male and postgraduate is 0.5

(d)
13 9
  
P (F ∩ T ) = P (F )P (T |F ) = = 0.225
40 13
The probability that a randomly selected staff is a female and postgraduate is 0.225

Alternatively: Example 4 can be solved by using the tree diagram. All four joint
probabilities for this example are shown in Figure 2.

9
 Figure 2: Tree diagram for joint probabilities.

2.7 Mutually Exclusive Events

Two events A and B are said to be mutually exclusive if they can not occur at the same time.
Thus the occurrence of one event excludes the occurrence of the other event or events.

Definition 2.10. Mutually Exclusive Events: Events that cannot occur together are said
to be mutually exclusive events.

This means that A and B do not share any outcomes and P (A ∩ B) = 0 and hence

P (A ∪ B) = P (A) + P (B). (8)

Example 5. Suppose the sample space S = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10} .

Let

A = {1, 2, 3, 4, 5}
B = {4, 5, 6, 7, 8}
C = {7, 9}

i. Are events A and B mutually exclusive?

ii. Are events A and C mutually exclusive?

Solution
2
i. A ∩ B = {4, 5} , P (A ∩ B) = and is not equal to zero. Therefore, A and B are not
10
mutually exclusive.

ii. A and C do not have any numbers in common so P (A ∩ C) = 0. Therefore, A and C are
mutually exclusive.

10
2.8 Independent Versus Dependent Events
In the case of two independent events, the occurrence of one event does not change the
probability of the occurrence of the other event.

Definition 2.11. Two events are said to be independent if the occurrence of one does not
affect the probability of the occurrence of the other

Two events A and B are independent if the knowledge that one occurred does not affect the
chance the other occurs. Two events are independent if the following are true:

• P (A|B) = P (A)

• P (B|A) = P (B)

• P (A ∩ B) = P (A)P (B)

To show two events are independent, you must show only one of the above condition. If two
events are not independent, then we say that they are dependent. In probability notation,
the two events are dependent if either P (A|B) ̸= P (A) or P (B|A) ̸= P (B).
Example 6. Refer to the information on 100 students given in Table 3 in Subsection 2.6.1.
Are events “female (F )” and “Like (A)” independent?
Solution
Events F and A will be independent if P (F |A) = P (F ) Otherwise they will be dependent.
Using the information given in Table 3 we compute the following two probabilities:
4 40
P (F |A) = = 0.2105 and P (F ) = = 0.4
19 100
Because these two probabilities are not equal, the two events are dependent. Here, depen-
dence of events means that the respective percentages of males who like and hate football are
different from the respective percentages of females who are in like and hate football.
In this example, the dependence of A and F can also be proved by showing that the
probabilities P (A) ̸= P (A|F ).

2.9 Complementary Events

Two mutually exclusive events that taken together include all the outcomes for an experiment
are called complementary events. Note that two complementary events are always mutually
exclusive.

Definition 2.12. The complement of event A, denoted by Ā is the event that includes all the
outcomes for an experiment that are not in A.

Figure 3: Venn diagram of two complementary events.

11
 Because two complementary events, taken together, include all the outcomes for an experi-
ment and because the sum of the probabilities of all outcomes is 1, that gives

P (A) + P (Ā) = 1 (9)

Simply, if we know the probability of an event, we can find the probability of its complementary
event by subtracting the given probability from 1.
Example 7. In a group of 200 BBA students, 400 have been cheated at least once in exams.
If one BBA student is randomly selected from this group, what are the two complementary
events for this experiment, and what are their probabilities?
Solution
The two complementary events for this experiment are

A =the selected student has been cheated at least once in exam

Ā =the selected student has never been cheated in exam

400 1600
P (A) = = 0.2 and P (Ā) = = 0.8
2000 2000
Also we can observe that the sum of these two probabilities P (A) + P (Ā) = 0.2 + 0.8 = 1 which
indicated that the two events are complementary.

2.10 EXERCISES
1. Explain the meaning of the intersection of two events. Give one example.

2. What is meant by the joint probability of two or more events? Give one example

3. CBE research team has 150 members. 75 of the members are lecturers. 47 of the members
are assistant lectures. The remainder are tutorial assistants. 40 of the lectures teach four
times a week. 30 of the assistant lectures teach four times a week. 10 of the tutorial
assistants teach four times a week. Suppose one member of the research team is chosen
randomly.

(a) What is the probability that the member is a tutorial assistants?

(b) What is the probability that the member teach four times a week?
(c) What is the probability that the member is an lecturer and teach four times a week?
(d) What is the probability that a member is an lecturer and an assistant lecturer? Are
being lecturer and assistant lecturer mutually exclusive? Why or why not?

4. Find the joint probability of A and B for the following.

(a) P (A) = 0.4 and P (B|A) = 0.25

(b) P (B) = 0.65 and P (A|B) = 0.36
(c) P (B) = 0.59 and P (A|B) = 0.77
(d) P (A) = 0.28 and P (B|A) = 0.35

5. A fair, six-sided die is rolled. Describe the sample space S, identify each of the following
events with a subset of S and compute its probability (an outcome is the number of dots
that show up).

12
 a. Event T = the outcome is two
b. Event A = the outcome is an even number.
c. Event B = the outcome is less than four.
d. The complement of A.
e. A GIVEN B
g. A AND B
h. A OR B
6. Suppose a study of speeding violations (SV) and drivers who use cell phones produced
the following fictional data:
SV No SV Total
Cell phone user 25 280
Not a Cell phone user 45 405
Total

a. Find P(Person is a car phone user).

b. Find P(person had no speed violation in the last year).
c. Find P(Person had no speed violation in the last year AND was a cell phone user).
d. Find P(Person is a cell phone user OR person had no speed violation in the last
year).
e. Find P(Person is a cell phone user GIVEN person had a speed violation in the last
year).

7. 2000 randomly selected students were asked whether or not they have ever cheated on
the exams. The following table gives a two-way classification of the responses obtained.

Table 5: Two-Way Classification of Students Responses

Have cheated Have never cheated
Male 500 700
Female 300 500

Suppose one student is selected at random from these 2000 students. Find the probabil-
ities that
a. has never cheated on the exam and is a male
b. has cheated on the exam and is a female
c. Mention what other joint probabilities you can calculate for this table and then find
those. You may draw a tree diagram to find these probabilities.
8. In the box, there are 11 balls. Three balls are red (R) and eight balls are blue (B). Draw
two balls, one at a time, with replacement. Use a tree diagram to
a. calculate P(RR).
b. calculate P(RB OR BR)
c. calculate P(R on 1st draw AND B on 2nd draw).
d. calculate P(R on 2nd draw GIVEN B on 1st draw)
e. Repeat the above example with no replacement

13