Lecture 4&5

Basic Probability Concepts & Discrete Probability Distributions

Random Variable:

A random variable (usually denoted as X) is a real-valued function which assigns numerical values to the
outcomes of a random trial.

Sample Space X
Numerical Values

Example: In tossing A coin, if we define a random variable X as “number of tails”, then, we have S = {Heads,
Tails}, and X = {0, 1}. Graphically, the function X maps each of the outcomes in the sample space to a
numerical value.

X
Heads 0

Tails 1

Example: In tossing two coins, if we define a random variable X as “number of heads, then, we have S = {TT,
TH, HT, HH} and X = {0, 1, 2}

X
T,T
0
T,H
H,T 1

H,H 2

There are two types of random variables:

Discrete Random Variable: A random variable that can only take discrete value.

The following are examples of a discrete random variable:

 Rolling a die, let X be the number of dots showing. Then, X is a discrete random variable.
 Let X be the number of students who will be absent from a class on a particular day for a particular class.
Then, X is a discrete random variable.

1
Continuous Random Variable: A random variable that can take any value in an interval.

The following are examples of a continuous random variable:

 Dropping a stone from a tall building, let X be the exact time at which the stone hits the ground. Then, X is
a continuous random variable.
 Jumping from a place A to another place B, let X be the distance between A and B. Then, X is a continuous
random variable.

Probability Distribution

A probability distribution for a discrete random variable assigns probability to each of the values that the
discrete random variable can assume.

Example:

1. Rolling two dice, let X be the sum of two dice. Then, we have

X P(X) 0.18
2 1/36 0.16
3 2/36 0.14
Probability

4 3/36
0.12
5 4/36
0.10
6 5/36
7 6/36 0.08
8 5/36 0.06
9 4/36 0.04
10 3/36
0.02
11 2/36
12 1/36 0.00
Sum 1 2 3 4 5 6 7 8 9 10 11 12

2. Flipping two coins, let X be the number of heads. Then, 0.6

Probability

0.5
X P(X) 0.4
0 1/4 0.3
1 2/4 0.2
2 1/4
0.1

0
0 1 2

A discrete probability distribution function must satisfy the following rules:

1) P(x) = 1 where x assumes all possible values, and

2) 0  P(x)  1 for every value of x.

Expected Value and Variance of a Discrete Random Variable

The Expected Value of a random variable X, denoted as E(X) or , is defined as

E( X )=μ=∑ x i P ( X=x i )
i
E(X) is simply the “weighted average of X” with the probability associated with each xi being the weight.

2
Examples:

1. Flipping two coins, let X be “number of heads”. Then, E(X) = 1 as shown below:

x P(x) xP(x)
0 1/4 0
1 2/4 1/2
2 1/4 1/2
E(X) = 1

2. Rolling two dice, let X be “sum of two dice”. Then, E(X) = 7 as shown below:

x P(x) xP(x)
2 1/36 2/36
3 2/36 6/36
4 3/36 12/36
5 4/36 20/36
6 5/36 30/36
7 6/36 42/36
8 5/36 40/36
9 4/36 36/36
10 3/36 30/36
11 2/36 22/36
12 1/36 12/36
E(X) = 7

3. Suppose that if I go to a beer bar in Central, I can expect to see some movie stars with the following
probabilities shown in the table. Then, on average, how many movie stars can I expect to meet each time I
go? If we define a random variable X as “number of movie stars met”, then E(X) = 1.65.

x 0 1 2 3 4 5 6 7 8 9 Total
P(x) 0.5 0.15 0.1 0.05 0.05 0.05 0.04 0.03 0.02 0.01 1
xP(x) 0 0.15 0.2 0.15 0.2 0.25 0.24 0.21 0.16 0.09 1.65

The Interpretation of Expected Value: Apparently, in the third example above, we cannot really meet 1.65
movie stars. So, the expected value will make better sense if we interpret it as a “long run average”. That is, if
the probabilities remain the same and if I go there every night for a long time, then, on average, the number of
movie stars I meet per night will be 1.65.

Variance and Standard Deviation of a Random Variable

The variance of a random variable, denoted as V(X) or 2, is defined as

V(X) = 2 = E(X – E(X))2 = E(X - )2

= E(X2) – E(X)2 = x2P(x) – [xP(x)]2 = x2P(x) - 2

As usual, the standard deviation of X, denoted as , is simply the square root of the variance of X.

Example:

1. Rolling two dice, let X be “sum of two dice”

x x2 P(x) xP(x) x2P(x)

2 4 1/36 1/18 1/9
3 9 1/18 1/6 1/2
4 16 1/12 1/3 4/3
5 25 1/9 5/9 25/9

3
 6 36 5/36 5/6 5
7 49 1/6 7/6 49/6
8 64 5/36 10/9 80/9
9 81 1/9 1 9
10 100 1/12 5/6 25/3
11 121 1/18 11/18 121/18
12 144 1/36 1/3 4
Total 7 329/6

So, E(X) = xP(x) = 7 and E(x2) = x2P(x) = 329/6 = 54.83. Hence, V(x) = E(x2) – E(x)2 = 54.83 – 49 =
5.83. The standard deviation, , is then 2.41.

2. For the following random variable X and its associated probability function, verify that E(X) = 5.1, V(X) =
4.69, and  = 2.17.

x 2 3 5 7 8
P(x) 0.1 0.3 0.2 0.2 0.2

Some Rules about E(X) and V(X) [Optional, out of syllabus]

If Y = aX + b, where a ( 0) and b are constants and X is a random variable, then

E(Y) = E(aX + b) = aE(X) + b, and

V(Y) = V(aX + b) = a2V(X).

Examples:

1. Let X be a random variable that can assume one of the values of 1, 2, and 3 with probabilities of 0.25, 0.5
and 0.25 respectively. Let Y = 2X + 3. Compute E(X), V(X), E(Y), and V(Y).

X X2 Y ( = aX + b ) Y2 P(X) xP(x) yP(y) x2P(x) y2P(y)

1 1 5 25 0.25 0.25 1.25 0.25 6.25
2 4 7 49 0.50 1 3.5 2 24.5
3 9 9 81 0.25 0.75 2.25 2.25 20.25
Total 25 0.25 0.25 1.25 0.25 6.25

Apparently, from the calculation shown on the above table, we know that E(X) = 2 and E(Y) = 2E(X) + 3 =
7. V(X) = E(x2) – E(X)2 = 4.5 – 4 = 0.5; and V(Y) = 22V(X) = 4(0.5) = 2. We can verify that V(Y) is indeed
equal to 22V(X) by computing Y with the original definition of V(Y) = E(Y2) – E(Y)2 = 51 – 72 = 2.

2. Let X be a random variable with possible values and associated probabilities defined in the following table.
Let Y = 4X + 1. Compute E(X), V(X), E(Y), and V(Y).

X Y = 4X+1 X2 Y2 P(X) xP(x) yP(y) x2P(x) y2P(y)

2 9 4 81 0.1 0.2 0.9 0.4 8.1
5 21 25 441 0.15 0.75 3.15 3.75 66.15
7 29 49 841 0.25 1.75 7.25 12.25 210.25
8 33 64 1089 0.2 1.6 6.6 12.8 217.8
10 41 100 1681 0.2 2 8.2 20 336.2
12 49 144 2401 0.1 1.2 4.9 14.4 240.1
Total 1 7.5 31 63.6 1078.6

It’s clear that E(X) = 7.5 and E(Y) = 4E(X) + 1 = 31. Also, V(X) = E(X2) – E(X)2 = 63.6 – 7.52 = 7.35, and
V(Y) = E(Y2) – E(Y)2 = 1078.6 – 312 = 117.6 = 42V(X) = 16(7.35) = 117.6.

4
Binomial Random Variable

Binomial Distribution

Binomial distribution is one of the best known discrete probability distributions. Basically, if we have a
sequence of identical trials in which

1. each trial can only result in one of the two possible outcomes – let’s just conveniently label these two
outcomes as “success” and “failure”
2. the probabilities of “success” and “failure” are known and remain unchanged from trial to trial
3. the trials are independent

then, the probability of getting some number of successes in a sequence n trials can be described by the
binomial distribution. Let X be the “number of successes”, then we call X a binomial random variable. The
probability distribution function that describes how likely it is for X to assume each of the n + 1 possible values
(from 0 to n) is called binomial distribution function.

Following are some examples of binomial random variables:

1. Let X be the “number of heads obtained” when we toss a coin 20 times. Then the probability of getting r
heads out of 20 trials can be described by a binomial probability distribution function.
2. Let X be the “number of correct answers” when someone makes blind guess on 20 true-and-false questions.
Then the probability of getting r correct answers out of 20 blind guesses can be described by a binomial
probability distribution function.
3. Let X be the “number of white balls obtained” when we take 40 random samples from an urn containing 15
black and 5 white balls with replacement,. Then the probability of getting r black balls can be described by
a binomial probability distribution function. (Note: If samples are taken without replacement, then X is not
a binomial random variable, and hence, we cannot use binomial probability distribution function to compute
the probability for each of the possible values of X)

If X is a binomial random variable, then the probability that X will assume a particular value x is

n!
P( X =x )=C nx p x (1−p )n−x = p x (1−p )n−x
x !(n−x )!

where n = number of trials,

x = number of “successes” (So, n – x = number of “failures”)
p = probability of “success” and 1 – p = probability of “failure”.

Examples:

1. If we toss a coin 20 times, what is the probability that we will get 5 heads?

20 5 20−5
Let X = number of heads. Then P(X = 5) = C 5 0 . 5 (1−0 .5 ) = 0.0148.

2. In an exam consisting of 20 true-and-false questions, if we take a random guess on each of the 20 questions,
assuming that it is equally likely to guess it right or wrong, what is the probability that we will get 10
correct answers? 9 correct answers? and less than 12 correct answers? What is your conclusion? Should
you guess based on the probabilities computed?

Let X = number of questions correctly guessed. Then, the probabilities for all possible outcomes (from 0 to
20 correct answers) are shown in the following table (n = 20, and p = 1 – p = 0.5):

X P( X =x )=C nx p x (1−p )n−x Cumulative X P( X =x )=C nx p x (1−p )n−x Cumulative

Probability Probability
0 0.0000 0.000001 11 0.1602 0.748278
1 0.0000 0.000020 12 0.1201 0.868412
2 0.0002 0.000201 13 0.0739 0.942341

5
 3 0.0011 0.001288 14 0.0370 0.979305
4 0.0046 0.005909 15 0.0148 0.994091
5 0.0148 0.020695 16 0.0046 0.998712
6 0.0370 0.057659 17 0.0011 0.999799
7 0.0739 0.131588 18 0.0002 0.999980
8 0.1201 0.251722 19 0.0000 0.999999
9 0.1602 0.411901 20 0.0000 1.000000
10 0.1762 0.588099

0.20

0.15

0.10

0.05

0.00
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Notice that the height of the bar indicates the probability.

3. In an exam consisting of 20 multiple choice questions, each question has 5 possible answers. If we take a
random guess on each of the questions, assuming that each of the 5 possible answers of each question is
equally likely to be the correct answer, find the probability that we will get 10 correct answers? 9 correct
answers? Less than 12 correct answers? What is your conclusion? Should you guess based on the
probabilities computed?

Let X = number of questions correctly guessed. Then, the probabilities for all possible outcomes (from 0 to
20 correct answers) are shown in the following table: (n = 20, p = 0.2, and 1 – p = 0.8)

X P( X =x )=C nx p x (1−p )n−x Cumulative X P( X =x )=C nx p x (1−p )n−x Cumulative

Probability Probability
0 0.0115 0.0115 11 0.0005 0.9999
1 0.0576 0.0692 12 0.0001 0.9999
2 0.1369 0.2061 13 0.0000 0.9999
3 0.2054 0.4114 14 0.0000 0.9999
4 0.2182 0.6296 15 0.0000 0.9999
5 0.1746 0.8042 16 0.0000 0.9999
6 0.1091 0.9133 17 0.0000 0.9999
7 0.0545 0.9679 18 0.0000 0.9999
8 0.0222 0.9900 19 0.0000 0.9999
9 0.0074 0.9974 20 0.0000 1.0000
10 0.0020 0.9994

6
 0.25

0.20

0.15

0.10

0.05

0.00
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Note: Most Statistics textbooks provide binomial probability tables that look like the two examples shown
above. Binomial Probability Table is especially useful when we need to calculate cumulative probability.
Alternatively, we can use Excel to calculate the probability of a binomial random variable. The Excel
function to use is “BINOMDIST(Number_s, Trials, Probability_s, Cumulative)”, where Number_s is
number of successes, Trials is number of trials, Probability_s is probability of success, and Cumulative is
“true” is you want to calculate cumulative probability and is “false” otherwise. For example, to calculate
the cumulative probability of getting less than or equal to 3 successes out of 10 trials for a binomial random
variable with P(success) = 0.6, we can enter the function “=BINOMDIST(3, 10, 0.6, TRUE)”. To calculate
the probability of getting exactly 3 successes out of 10 trials, we enter “=BINOMDIST(3, 10, 0.6,
FALSE)”.

4. A machine is known to have 5% defective rate. That is, 5% of the products produced by that machine are
defective and the remaining 95% are good. If we take a random sample of 10 products produced by that
machine, what is the probability that 3 are defective? Less than or equal to 3 are defective?

Let X = number of defective products. Then, the probabilities for all possible outcomes (from 0 to 10
defective items) are shown in the following table: (n = 10, p = 0.05, and 1 – p = 0.95, notice that, here, we
define the event “success” as “getting a defective product)

X P( X =x ) Cumulative 0.7
=C nx p x (1− p)n−x Probability
0 0.5987 0.5987 0.6
1 0.3151 0.9139 0.5
2 0.0746 0.9885
3 0.0105 0.9990 0.4
4 0.0010 0.9999 0.3
5 0.0001 1.0000
6 0.0000 1.0000 0.2
7 0.0000 1.0000 0.1
8 0.0000 1.0000
9 0.0000 1.0000 0
1 2 3 4 5 6 7 8 9 10 11
10 0.0000 1.0000

7
The Expected Value and Variance of a Binomial Random Variable

If X is a binomial random variable with parameters n and p, then E(X) = np and V(X) = np(1 – p).

Example:
5. Verify that, for Example 2, E(X) = np = 20  0.5 = 10 and the V(X) = np(1 – p) = 20  (0.5)  (0.5) = 5.
How do you interpret these results?

Poisson Distribution [Optional]

The Poisson distribution is a discrete probability distribution defined as

μ x⋅e− μ
P( X =x )=
x!

where x = number of occurrences

e = 2.71828, and
 = mean number of occurrences

Some Remarks:

1. Poisson distribution can be a very good approximation of Binomial distribution when n is large. That is,
μ x⋅e−μ
lim n→∞ Cnx p x (1− p )n− x=
x ! , where  = np.
2. Usually, a Poisson random variable deals with the number of times that a certain event occurs over a
specified interval or area. Hence, the Poisson distribution is usually used to describe the probability for a
certain number of occurrences over a specified interval or area.
3. The set of all possible values that a Poisson random variable X can assume is {0,1, 2, … , }

Assumptions of Poisson Random Variable:

1. Events must occur independently from one interval to the other.

2. The probability of observing a single event must be proportional to the size of the interval.
3. The probability of more than one event occurring in a very small interval is negligible.
4. The probability distribution must remain constant from one interval to another.

Applications:

1. Number of customers entering Park’N Shop during a five-minute interval.

2. Number of cars passing over a toll tunnel from 1:00 to 2:00 p.m. on a given day.
3. Number of machine breakdowns during a night shift.

8
Examples:

1. The number of customers arriving at Park’N Shop of CUHK is assumed to be Poisson distributed with a
mean arrival rate of 8 customers for every 10 minutes. What is the probability that there will be five
customers arriving in the next 5 minutes?

Here,  = 8 per 10 minutes. Hence,  = 4 per 5 minutes. Therefore,

45⋅e−4
P( X =5)= =0 .1563
5!

2. For the above question, what is the probability that there will be no more than () two customers arriving
in the next 5 minutes?

P( X ≤2)=P ( X=0)+P( X =1)+P( X =2)

4 0⋅e−4 41⋅e−4 42⋅e−4
= + + =0 . 0183+0 . 0733+0 . 1465=0 . 2381.
0! 1! 2!

Usually, tables are available for computing the probabilities of a Poisson random variable. One can also
use computer software to compute the probabilities very easily. The following table is computed from
Microsoft Excel.

x P(X = x) P(X  x) 7 0.0595 0.9489

0 0.0183 0.0183 8 0.0298 0.9786
1 0.0733 0.0916 9 0.0132 0.9919
2 0.1465 0.2381 10 0.0053 0.9972
3 0.1954 0.4335 11 0.0019 0.9991
4 0.1954 0.6288 12 0.0006 0.9997
5 0.1563 0.7851 13 0.0002 0.9999
6 0.1042 0.8893 14 0.0001 1.0000

3. The number of cars passing through a toll tunnel is Poisson distributed with a mean rate of 20 per minute.
Calculate using the tables provided
a) The probability that there will be 15 cars passing through in the next minute.
b) The probability that there will be no more than 10 cars passing through in the next 30 seconds.
c) The probability that there will be more than 15 cars passing through in the next 30 seconds.

9
 = 20  = 10
x P(X = x) P(X  x) x P(X = x) P(X  x)
0 0.0000 0.0000 0 0.0000 0.0000
1 0.0000 0.0000 1 0.0005 0.0005
2 0.0000 0.0000 2 0.0023 0.0028
3 0.0000 0.0000 3 0.0076 0.0103
4 0.0000 0.0000 4 0.0189 0.0293
5 0.0001 0.0001 5 0.0378 0.0671
6 0.0002 0.0003 6 0.0631 0.1301
7 0.0005 0.0008 7 0.0901 0.2202
8 0.0013 0.0021 8 0.1126 0.3328
9 0.0029 0.0050 9 0.1251 0.4579
10 0.0058 0.0108 10 0.1251 0.5830
11 0.0106 0.0214 11 0.1137 0.6968
12 0.0176 0.0390 12 0.0948 0.7916
13 0.0271 0.0661 13 0.0729 0.8645
14 0.0387 0.1049 14 0.0521 0.9165
15 0.0516 0.1565 15 0.0347 0.9513
16 0.0646 0.2211 16 0.0217 0.9730
17 0.0760 0.2970 17 0.0128 0.9857
18 0.0844 0.3814 18 0.0071 0.9928
19 0.0888 0.4703 19 0.0037 0.9965
20 0.0888 0.5591 20 0.0019 0.9984
21 0.0846 0.6437 21 0.0009 0.9993
22 0.0769 0.7206 22 0.0004 0.9997
23 0.0669 0.7875 23 0.0002 0.9999
24 0.0557 0.8432 24 0.0001 1.0000
25 0.0446 0.8878 25 0.0000 1.0000
26 0.0343 0.9221
27 0.0254 0.9475
28 0.0181 0.9657
29 0.0125 0.9782
30 0.0083 0.9865
31 0.0054 0.9919
32 0.0034 0.9953
33 0.0020 0.9973
34 0.0012 0.9985
35 0.0007 0.9992
36 0.0004 0.9996
37 0.0002 0.9998
38 0.0001 0.9999
39 0.0001 0.9999
40 0.0000 1.0000

The Expected Value and Variance of a Poisson Random Variable

If X is a Poisson random variable, then E(X) =  = mean occurrences, and V(X) = 2 = . That is, the mean and
the variance of a Poisson random variable are the same.

Note: Again, most Statistics textbooks provide Poisson probability table. We can also use Excel to calculate the
probability of a Poisson random variable. The Excel function to use is “POISSON( x, Mean, Cumulative)”,
where x is number of events, Mean is the expected value, Cumulative is “true” if you want to calculate
cumulative probability and is “false” otherwise. For example, for the previous example in which we need to
calculate the probability that there will be 15 cars passing through in the next minute when the number of cars
passing through a toll tunnel is Poisson distributed with a mean rate of 20 per minute, we enter the following
function in Excel: “=POISSON(15, 20, FALSE)”.

10