PROB & STAT for COMP SCI 1.

PROBABILITY BASICS © Whalen

“We balance probabilities and choose the most likely. It is the scientific use of the imagination.”
—Sherlock Holmes, The Hound of the Baskervilles

What are the Chances?

We talk about chance all the time. What are the chances of getting a job? Of winning a game? Of
meeting someone? Of rain tomorrow? Of getting to work on time?

For day-to-day purposes, we may only need an intuitive understanding of chance, or equivalently,
probability. To use probability in math and science, however, we need a rigorous definition. This turns
out to be a highly non-trivial task, and there are multiple theories that mathematicians and scientists
have developed.

Three interpretations of probability

First interpretation:

Probability’s origin is in the context of gambling games, particularly in France in the 1600s. Some of the
early masters include Abraham de Moivre, Blaise Pascal, Pierre de Fermat, and Pierre Simon, marquis
de Laplace.

Probability rules were developed and explained using problems involving dice, cards, spinners, coin
tosses, and random draws from a bucket (or urn). In these problems, probability is computed by
reducing events to equally-likely outcomes.

It is well-suited for game theory and events with some kind of symmetry. It is powerless to assign
probabilities to other events, such as the probability that there was ever life on Mars or that you will
meet a famous person this year, or even if it will rain tomorrow.

Second interpretation:

Extending the classical interpretation for use in statistics, probability is interpreted as the relative
frequency of an event. If we can repeat a process many times, how many of those times turned out a
certain way? For instance, if the chance of rain tomorrow is 30%, that means in our past records 30
days out of 100 like tomorrow had rain. This interpretation is fine as long as an experiment is
repeatable, but like classical probability it is powerless to assign chances to outcomes of non-
repeatable experiments.

Third interpretation:

In fact, many interesting questions cannot be answered by repeatable experiments. (Of course, an
experiment or observation may give evidence one way or another, but may not be repeatable.)
Probability may be interpreted as a degree of belief, where we assign probability values based on
evidence (and perhaps our own beliefs).

Page 1 of 10
PROB & STAT for COMP SCI 1. PROBABILITY BASICS © Whalen

Each interpretation has advantages and disadvantages. Fortunately, the mathematical axioms and
rules work regardless of your interpretation.

Fun and Games

To illustrate the probability rules, we will use classical examples.

COIN—a coin is flipped and lands with one of two sides facing up. One side is called “heads” and the
other is called “tails.” Usually a coin is assumed to be “fair,” meaning heads and tails are equally likely.

DIE (plural: DICE)—a cube with six faces, labelled 1, 2, 3, 4, 5, and 6. Usually a die is assumed to be fair.

CARDS—a set of cards is a “deck.” For this class we will assume a standard poker deck. A deck of cards
has 4 suits: hearts, clubs, diamonds, and spades. There are 13 cards in each suit: 2 through 10, jack,
queen, king, and ace. So there are 4 × 13 = 52 cards. Usually a deck is assumed to be “well-shuffled,”
implying if we draw a card from the top of the deck, each card is equally likely to be there.

Range
Mathematically, probability is a function that maps an event to a number between 0 and 1
(equivalently, a percentage between 0% and 100%).

An event is an outcome, or set of outcomes, of some experiment. “Experiment” is a generic term. It

could mean something as simple as rolling a die or drawing a card.

If an event is impossible, its probability is 0. If an event is certain, its probability is 1. To assign a

probability value to an event, we can use the classical formula:

The probability of an event is

the number of possible outcomes where the event happens
the total number of possible outcomes of the experiment
assuming that each outcome is equally likely.

Multiplication Rule and Independence

Suppose you toss a coin twice. If the first toss lands heads, does that affect the second toss? If the first
toss lands tails, does that affect the second toss? No: the outcome of the first toss has no impact on
the outcome of the second toss.

Two events are independent if one of them happening has no effect on the
chance of the other happening.

We can generalize this to any list of events. Saying events are independent means that any one of
them happening makes no difference to the chances of any of the others happening. (In other words,
any two of them are independent.)

Page 2 of 10
PROB & STAT for COMP SCI 1. PROBABILITY BASICS © Whalen

The probability that two events both happen uses multiplication.

To find the probability that two events both happen, first decide if they are independent.
If they are, multiply their probabilities.

Examples:
(a) A coin is tossed twice. What is the probability that it lands heads both times?

(b) A die is rolled twice. What is the chance it lands on five, then on two?

(c) A coin is tossed three times. What is the probability that it lands heads, then tails, then heads?

(d) A coin is tossed 𝑛 times. Find a formula for the probability that it lands heads on all 𝑛 tosses.

(e) Roll a die and draw a card from a well-shuffled deck. What is the probability you roll a 3 and draw
the queen of diamonds?

(f) A deck is well-shuffled. Draw the top two cards. We want the probability that the first card is the ace
of spades and the second card is the king of spades. The probability of getting the ace of spades is
1/52, and the probability of getting the king of spades is 1/52. We want the probability of both.

1 1
True or false: The probability is 52 × 52

Page 3 of 10
PROB & STAT for COMP SCI 1. PROBABILITY BASICS © Whalen

When the events are not independent (that is, if one happens, it affects the chances that the other
happens), we still multiply. But we have to account for the dependence.

The probability that two events both happen is the probability that the first will happen, multiplied
by the probability that the second will happen given that the first happened.

We can generalize this to any sequence of events. For example, the probability that three events all
happen is:
the probability that the first will happen,
times the probability that the second will happen given the first happened,
times the probability that the third will happen, given that the first and second both happened.

Examples:
(a) Draw four cards from a well-shuffled deck. Find the probability that all four are aces.

(b) Shuffle a deck, draw a card, and put it back. Shuffle again, draw a card, and put it back. What is the
probability that both cards will be a queen?

(c) An urn has 4 blue marbles and 8 red marbles. Three marbles are drawn randomly without
replacement. What is the probability that all three are red?

(d) Repeat part c, but this time the marbles are drawn with replacement.

Page 4 of 10
PROB & STAT for COMP SCI 1. PROBABILITY BASICS © Whalen

Addition Rule and Mutual Exclusiveness

Suppose you roll a die. The outcome could be both a 2 and “even.” But, it cannot be both 2 and 3, or
both “even” and “odd.”

Two events are mutually exclusive if they cannot both happen. That is, the occurrence of one
prevents the occurrence of the other.

To find the probability that at least one of two events happens, we use addition.

To find the probability that at least one of two events will happen, check whether they are mutually
exclusive. If they are, add the probabilities.

Examples:

(a) Draw one card from a well-shuffled deck. The chance it will be a spade is 1/4. The chance it will be a
club is 1/4.

1 1
True or false: the chance it will be a club or a spade is 4 + 4

(b) Roll a die. What is the chance of rolling 1 or 2?

(c) Roll a die. What is the chance of rolling an odd number?

(d) An urn has 2 green, 3 blue, and 4 red marbles. If you draw a marble at random, what is the chance
it is green or red?

(e) Roll two dice.

1 1
True or false: the chance of getting a 5 on the first die or a 5 on the second die is 6 + 6

Page 5 of 10
PROB & STAT for COMP SCI 1. PROBABILITY BASICS © Whalen

Complement Rule and DeMorgan’s Laws

The probability of drawing the ace of spades is 1/52. Of course, the probability of the opposite event is
51/52; that is, the probability that you do not draw the ace of spades. If the chance of winning a game
is 40%, then the chance of losing the game is 60%.

The probability of an event equals 1 minus the probability of the opposite event.

The complement rule is often used in conjunction with De Morgan’s Laws, which describe the opposite
of compound events.

The opposite of “at least one event happens” is “none of the events happen.”
The opposite of “all of the events happen” is “at least one event does not happen.”

Examples:
(a) Roll two dice. Find the probability that at least one will be a 5.

(b) (Related to the paradox of the Chevalier de Méré) In a certain gambling game, a pair of dice is rolled
24 times. You win if you roll “double 1s” at least once. (Double 1s is the event that you roll a 1 on both
dice simultaneously.) What is the probability that you win?

Page 6 of 10
PROB & STAT for COMP SCI 1. PROBABILITY BASICS © Whalen

Reliability
The basic probability rules can be used for computing reliability of a system. A system consists of
components working together (for example, the parts of a computer, or people on a team, or multiple
tests run, or a car engine).

Reliability is the probability that a system does its job.

It is usually reasonable to assume that the components function independently—if one component
fails, this has no impact on the other components failing (though the system as a whole might fail). For
example, if your keyboard fails, that does not affect the chances that the other components of your
computer fail. (But you cannot use your computer until you replace the keyboard.)

If components are connected in sequence, then all of the components must work
for the connection to do its job.
If components are connected in parallel, then the connection can do its job so
long as at least one of the components work.

Example: A backup hard drive is an example of a parallel component. At least one of the hard drives
must work for the data to be safe. Suppose the hard drive’s reliability is 0.99 and the backup’s
reliability is 0.95. What is the reliability of this system?

Example: A scan for a virus detects the virus with a probability of 0.9. If three independent scans are
performed, what is the probability that at least one of them detects the virus?

Page 7 of 10
PROB & STAT for COMP SCI 1. PROBABILITY BASICS © Whalen

Example: A system is a sequence of three components, where the center component itself consists of
three parallel components. See the diagram:

The components function independently, and each of the 5 individual components have reliability 𝑅.
Find the formula for the reliability of the system in terms of 𝑅.

Page 8 of 10
PROB & STAT for COMP SCI 1. PROBABILITY BASICS © Whalen

Notation Station
The mathematical way to write about the chance of an event is
𝑃(𝐴)
where 𝐴 is some event.

Events can be combined like sets:

 Set union represents “or.” For 𝐴 ∪ 𝐵 to happen, at least one of 𝐴 or 𝐵 must happen.

 Set intersection represents “and.” For 𝐴 ∩ 𝐵 to happen, both 𝐴 and 𝐵 have to happen.

 Set complement represents “not.” Symbolically, a complement may be 𝐴𝑐 or 𝐴̅. For 𝐴̅ to

happen, it must be that 𝐴 does not happen.

It is just fine to write the words “or” and “and” instead of the symbols ∪ and ∩ unless you are writing
for a math audience.

Example: Let 𝑄 represent drawing a queen, and let 𝐻 represent drawing a heart, from a well-shuffled
deck of cards. Interpret and compute:

𝑃(𝑄 ∩ 𝐻) =

𝑃(𝑄 ∪ 𝐻) =

𝑃(𝑄̅ ) =

Page 9 of 10
PROB & STAT for COMP SCI 1. PROBABILITY BASICS © Whalen

The Birthday Problem

A famous probability problem asks, “If a bunch of people are in the room, what is the chance that two
people in the room have the same birthday?” The answer may be surprising. We assume that each day
of the year is equally likely to be a random person’s birthday, we ignore leap day, and we assume
birthdays are independent.

As a warm-up, in a room full of 2 people, what is the chance two people have the same birthday?

How about a room full of 3 people?

How about a room full of 30 people?

Page 10 of 10