Chapter 3

Introduction to
Probability
Probability is the study of random or nondeterministic
 

experiments.

If a die is tossed in the air, then it is certain that the die will
come down, but it is not certain that, say, a 6 will appear.
However, suppose we repeat this experiment of tossing a die;
let be the number of successes, i.e. the number of times a 6
appears, and let be the number of tosses. Then it has been
empirically observed that the ratio , called the relative
frequency, becomes stable in the long run, i.e. approaches a
limit. This stability is the basis of probability theory.
 Historically, probability theory began with the study of
 

games of chance, such as roulette and cards. The

probability of an event was defined as follows: if can
occur in ways out of a total of equally likely ways, then

Example
 

In tossing a die an even number can occur in 3 ways

out of 6 “equally likely” ways; hence
 This classical definition of probability is essentially
circular since the idea of “equally likely” is the same as
that of “with equal probability” which has not been
defined. The modern treatment of probability theory is
purely axiomatic. This means that the probabilities of
our events can be perfectly arbitrary, except that they
must satisfy certain axioms (to be discussed later). The
classical theory will correspond to the special case of
so-called equiprobable spaces.
SAMPLE SPACE AND EVENTS
 

Sample space the set of all possible outcomes of some given

experiment.
Sample point or sample a particular outcome or an element in .
Event a set of outcomes or, in other words, a subset of the
sample space .
Elementary event the event consisting of a single sample .
Impossible event he empty set .
Certain or sure event the itself.
We can combine events to form new events using the various
 

set operations:
i. is the event that occurs iff occurs or occurs (or both);
ii. is the event that occurs iff occurs and occurs;
iii. is the complement of , is the event that occurs iff does
not occur.
Two events and are called mutually exclusive if they are
 

disjoint, i.e. if . In other words, and are mutually exclusive if

they cannot occur simultaneously.
Example 3.1
 

Toss a die and observe the number that appears on top. Then
the sample space consists of the six possible numbers:

Let be the event that an even number occurs, that an odd

 

number occurs and that a prime number occurs

 
- the event that an even or a prime number occurs;
- the event that an odd prime number occurs;
- the event that a prime number does not occur.

, Thus and are mutually exclusive. in other words, an even number and
an odd number cannot occur simultaneously.
Example 3.2
 

Toss a coin 3 times and observe the sequence of heads (H)

and tails (T) that appears. The sample space S consists of
eight elements:

Let be the event that two or more heads appear

 

consecutively, and that all the tosses are the same:

Then is the elementary event in which only heads appear.

The event that 5 heads appear is the empty set .

Example 3.3
 

Toss a coin until a head appears and then count the number of
times the coin was tossed. The sample space of this
experiment is

Here refers to the case when a head never appears and so the
coin is tossed an infinite number of times. This is an example of
a sample space which is countably infinite.
Example 3.4
 

Let a pencil drop, head first, into a

rectangular box and note the point on the
bottom of the box that the pencil first
touches. Here consists of all the points
on the bottom of the box. Let the
rectangular area on the right represent
these points. Let and be the events that
the pencil drops into the corresponding
areas illustrated on the right.
This is an example of a sample space
which is not finite nor even countably
infinite, i.e. which is uncountable.
AXIOMS OF PROBABILITY
 

Let S be a sample space, let be the class of events, and let

be a real-valued function defined on 6. Then is called a
probability function, and is called the probability of the event if
the following axioms hold:
[] For every event , .
[] .
[] If and are mutually exclusive events, then

[] If is a sequence of mutually exclusive events, then

Theorem 3.1: If is the empty set, then .
 

Proof:
 

Let be any set; then and are disjoint and

by []
Theorem 3.2: If is the complement of an event , then .
 

Proof:
 

The sample space can be decomposed into the mutually

exclusive events and ; that is

by []
by []
Theorem 3.3: If , then .
 

Proof:
 

If , then can be decomposed into the mutually exclusive

events and . Thus
by []
is not empty, thus
Clearly
Theorem 3.4: If
 

Proof:
 

can be decomposed into the mutually exclusive events and

that is,

by []
or
Theorem 3.5: If
 

Proof:
 

can be decomposed into the mutually

exclusive events and that is,

by []
by T 3.4

Corollary 3.6: For any events ,

 
 
FINITE PROBABILITY SPACES
Let be a finite sample space; say, . A finite probability
space is obtained by assigning to each point a real number ,
called the probability of , satisfying the following properties:
i. each is nonnegative,
ii. the sum of the is one, .
The of any event , is then defined to be the sum of the
probabilities of the points in . For notational convenience we
write
Example 3.5: Let three coins be tossed and the number of
 

heads observed; then the sample space is

We obtain a probability space by the following assignment

Let A be the event that at least one head appears and let B be
the event that all heads or all tails appear:
Example 3.6: Three horses A, B and C are in a race; A is twice as
likely to win as B and B is twice as likely to win as C. What are
their respective probabilities of winning, i.e. P(A),P(B) and P(C)?
Solution:
 

Let , then and

Example 3.6: Three horses A, B and C are in a race; A is twice as
likely to win as B and B is twice as likely to win as C. What are
their respective probabilities of winning, i.e. P(A),P(B) and P(C)?
Answer:
 

What is the probability that B or C wins?

Answer:
 
FINITE EQUIPROBABLE SPACES
 

Frequently, the physical characteristics of an experiment suggest that

the various outcomes of the sample space be assigned equal
probabilities. Such a finite probability space , where each sample point
has the same probability, will be called an equiprobable or uniform
space. In particular, if contains points then the probability of each point
is . Furthermore, if an event contains points then its probability is . In
other words,
Example 3.7: Let a card be selected a t random from an ordinary deck
 

of 52 cards. Let A = {the card is a spade} and B = {the card is a face

card, i.e. a jack, queen or king}. Compute P(A), P(B) and .

 
Answers:
Example 3.8: Let 2 items be chosen at random from a lot containing
12 items of which 4 are defective. Let A = {both items are defective}
and B = {both items are non-defective}. Find P(A) and P(B).

Solution:
 

can occur in ways

can occur in ways
can occur in ways  
What is the probability that at least one item is defective?

Let C be the event that at least one item is defective, then

 

NOTE: The odds that an event with probability p occurs is defined

to be the ratio p : (1-p).

Thus, the odds that that at least one item is defective is or 19:14.
 
Example 3.9: (Classical Birthday Problem.) We seek the probability p
that n people have distinct birthdays.

(In solving this problem, we ignore leap years and assume that a
person’s birthday can fall on any day with the same probability)

Since there are n people and 365 different days, there are ways in
 

which the n people can have their birthdays. On the other hand, if the n
persons are to have distinct birthdays, then the first person can be born
on any of the 365 days, the second person can be born on the
remaining 364 days, the third person can be born on the remaining 363
days, etc. Thus, there are
ways the n persons can have distinct birthdays.
Example 3.9: (Classical Birthday Problem.) We seek the probability p
that n people have distinct birthdays.

Accordingly,
 

It can be shown that for , ; in other words, amongst 23 or more

 

people, it is more likely that at least two of them have the same
birthday than that they all have distinct birthdays.
INFINITE SAMPLE SPACES
 

Now suppose S is a countably infinite sample space; say . As

in the finite case, we obtain a probability space by assigning
to each a real number , called
its probability, such that

The probability P(A) of any event A is then the sum of the

probabilities of its points.
Example 3.10: Consider the sample space of the experiment of
 

tossing a coin till a head appears; here denotes the number of times
the coin is tossed. A probability space is obtained by setting
, ,…, ,…,

 
The only uncountable sample spaces which we will consider here
are those with some finite geometrical measurement such as length,
area or volume, and in which a point is selected a t random. The
probability of an event , i.e. that the selected point belongs
to , is then the ratio of to ; that is,
or or
Such a probability space is said t o be uniform.
Example 3.11: On the real line , points and are selected at random
 

such that and , as shown below. Find the probability that the distance
between and is greater than 3.
End of Chapter 3 discussion

More solved Problems

will be posted in your GCR
Chapter 4

Conditional
Probability and
Independence
CONDITIONAL PROBABILITY
 

Let be an arbitrary event in a sample space with . The probability

that an event occurs once has occurred or, in other words, the
conditional probability of given , written , is defined as follows:

As seen in the adjoining Venn diagram, in a certain sense measures

the relative probability of with respect to the reduced space .
 
In particular, if is a finite equiprobable space and denotes the
number of elements in an event , then

That is
 

Theorem 4.1: Let S be a finite equiprobable space with events A and E.

Then

Or
Example 4.1: Let a pair of f a i r dice be tossed. If the sum is 6, find
 

the probability that one of the dice is a 2.

In other words, if E = {sum is 6} = {(1,5), (2,4), (3,3), (4,2),(5,1)}
and A = {a 2 appears on at least one die}, find .
Solution:
 

E = {(1,5), (2,4), (3,3), (4,2),(5,1)} and (2,4), (4,2)

On the other hand, since A consists of eleven elements,

 

A = {(2,1), (2,2), (2,3),(2,4),(2,5),(2,6),(1,2),(3,2),(4,2), (5,2),(6, 2)}

and S consists of 36 elements,
.
Example 4.2: A couple has two children. Find the probability p that
 

both children are boys if (i) we are given that the younger child is a
boy, (ii) we are given that (at least) one of the children is a boy.

The sample space for the sex of two children is with probability for
each point. (Here the sequence of each point corresponds to the
sequence of births.)

(i) The reduced sample space consists of two elements, hence p = .

(ii) The reduced sample space consists of three elements, ; hence p = .
MULTIPLICATION THEOREM FOR CONDITIONAL
 

PROBABILITY
If we cross multiply the above equation defining conditional
probability and use the fact that , we obtain the following useful
formula.
Theorem 4.2:
 

This theorem can be extended by induction as follows:

Corollary 4.3: For any events

Example 4.3: A lot contains 12 items of which 4 are defective. Three
 

items are drawn at random from the lot one after the other. Find the
probability p that all three are nondefective.
The probability that the first item is nondefective is since 8 of 12
items are nondefective. If the first item is nondefective, then the
probability that the next item is nondefective is since only 7 of the
remaining 11 items are nondefective.
If the first two items are nondefective, then the probability that the
last item is nondefective is since only 6 of the remaining 10 items are
now nondefective. Thus by the multiplication theorem,
FINITE STOCHASTIC PROCESSES AND TREE DIAGRAMS
A (finite) sequence of experiments in which each experiment has a
finite number of outcomes with given probabilities is called a (finite)
stochastic process.
A convenient way of describing such a process and computing the
probability of any event is by a tree diagram; the multiplication
theorem of the previous section is used to compute the probability that
the result represented by any given path of the tree does occur.
Example 4.4: We are given three boxes as follows: Box I has 10 light
bulbs of which 4 are defective. Box II has 6 light bulbs of which 1 is
defective. Box III has 8 light bulbs of which 3 are defective. We select
a box at random and then draw a bulb at random. What is the
probability p that the bulb is defective?
Example 4.4: We are given three boxes as follows: Box I has 10 light
bulbs of which 4 are defective. Box II has 6 light bulbs of which 1 is
defective. Box III has 8 light bulbs of which 3 are defective. We select
a box at random and then draw a bulb at random. What is the
probability p that the bulb is defective?
  1 2 1 1 1 3
𝑝= ∙ + ∙ + ∙
3 5 3 6 3 8
  2 1 1
𝑝= + +
15 18 8
  48+20+45 113
𝑝= =
360 360
Example 4.5: A coin, weighted so that P(H) = 2/8 and P(T) = 1/8, is
tossed. If heads appears, then a number is selected at random from
the numbers 1 through 9; if tails appears, then a number is selected
at random from the numbers 1 through 5. Find the probability p that
an even number is selected.
  2 4 1 2
𝑝=𝑃 ( 𝐸 )= ∙ + ∙
3 9 3 5
  8 2
𝑝= +
27 15
 
40 +18 58
𝑝= =
135 135
PARTITIONS AND BAYES' THEOREM
 

Suppose the events , form a partition of a sample space ; that is, the
events are mutually exclusive and their union is . Now let be any
other event. Then

where the are also mutually exclusive. Accordingly,

Thus, by multiplication theorem,

On the other hand, for any , the conditional probability of given is
 

defined by

In this equation we use previoes equation to replace P(B) and use to

 

replace , thus obtaining

Bayes' Theorem 4.4: Suppose , is a partition of and is any event.

Then for any ,
Example 4.6: Three machines A , B and C produce respectively 50%,
30% and 20% of the total number of items of a factory. The
percentages of defective output of these machines are 3%, 4% and
5%. If an item is selected at random, find the probability that the item
is defective.
Let X be the event that an item is defective. Then
 
Example 4.7: Consider the factory in the preceding example.
 

Suppose an item is selected at random and is found to be defective.

Find the probability that the item was produced by machine A ; that is,
find .

By Bayes’ theorem
 

 
. 5(.03) 0.015 1000 15
𝑃 ( 𝐴|𝐵 ) = = ∙ =
.5 ( .03 ) +.3 ( .04 )+.2(.05) 0.037 1000 37
INDEPENDENCE
 

An event B is said to be independent of an event A if the probability

that B occurs is not influenced by whether A has or has not occurred.
In other words, if the probability of B equals the conditional
probability of B given A: . Now substituting P(B) for P(B|A ) in the
multiplications theorem , we obtain

We use the above equation as our formal definition of independence.

Definition:
 

Events A and B are independent if ; otherwise, they are dependent.

Example 4.8: Let a fair coin be tossed three times; we obtain the
 

equiprobable space S = {HHH, HHT, HTH, HT'I', THH, THT, TTH,

TTT}. Consider the events A = {first toss is heads}, B = {second toss
is heads} and C = {exactly two heads are tossed in a row}

,
,
,
Then ,
,
Example 4.8: Let a fair coin be tossed three times; we obtain the
 

equiprobable space S = {HHH, HHT, HTH, HT'I', THH, THT, TTH,

TTT}. Consider the events A = {first toss is heads}, B = {second toss
is heads} and C = {exactly two heads are tossed in a row}

, ,
, ,

, A and B are independent

, A and C are independent
, A and B are dependent
Example 4.9: The probability that A hits a target is 1/4 and the
 

probability that B hits it is 2/5. What is the probability that the target
will be hit if A and B each shoot at the target?

; and , and .

, because A and B are clearly independent . Thus

Three events A , B and C are independent if:
 

i. and i.e. if the events are pairwise independent, and

ii. .

Example 4.10: Let a pair of fair coins be tossed; here is an

 

equiprobable space. Consider the events

A = {heads on the first coin} = {HH, HT}
B = {heads on the second coin} = {HH, TH}
C = {heads on exactly one coin} = {HT, TH}
Determine if A, B and C are independent or not.
  

A = {HH, HT}
B = {HH, TH}
C = {HT, TH}
 
and
, ,

, ,
, 1st condition is satisfied.
. 2nd condition is not satisfied. Thus, events A, B and C are not
independent.
INDEPENDENT OR REPEATED TRIALS
We have previously discussed probability spaces which were
associated with an experiment repeated a finite number of times, as the
tossing of a coin three times. This concept of repetition is formalized
as follows:
Definition:
 

Let be a finite probability space. By or trials, we mean the probability

space consisting of ordered -tuples of elements of with the
probability of an -tuple defined to be the product of the probabilities of
its components:
Example 4.11: Whenever three horses a, b and c race together, their
 

respective probabilities of winning are ½, 1/3 and 1/6. In other words,

S = {a, b, c } with P(a) = 1/2, P(b)= 1/3 and P(c) = 1/6. If the horses
race twice, then the sample space of the 2 repeated trials is
T = {aa, ab, ac, ba, bb, bc, ca, cb, cc}

For notational convenience, we have written ac for the ordered pair (a,
c). The probability of each point in T is
, , , , , , , ,

Thus, the probability of c winning the 1st race and a winning the 2nd
race is P(ca)=1/12
From another point of view, a
repeated trials process is a
stochastic process whose tree
diagram has the following
properties: (i) every branch point
has the same outcomes; (ii) the
probability is the same for each
branch leading to the same
outcome. For example, the tree
diagram of the repeated trials
process of the preceding
experiment is as shown in the
adjoining figure.
End of Chapter 4 discussion

More solved Problems

will be posted in your GCR