Chapter 2: Random variables, random vectors

Phan Thi Khanh Van

E-mail: khanhvanphan@hcmut.edu.vn

October 25, 2021

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 1 / 41
Table of Contents

1 Random variables. Cumulative, probability mass functions and

probability density functions
Discrete Random Variables
Continuous random variable

2 Some characteristics of random variables.

Characteristics of discrete random variables.
Characteristics of continuous random variables.

3 Discrete random vectors and their characteristics

4 Conditional expectation

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 2 / 41
Random experiment
An experiment that results in different outcomes, even when repeated
under the same conditions every time is called a random experiment.

Flip a coin: 2 possible outcomes: H, T

Roll a die: 6 possible outcomes: 1, 2, 3, 4, 5, 6
Count the number of calls dropped due to errors over a particular
time period: N = 1, 2, 3...., countably infinite number of outcomes.
Investigate the time difference between 2 messages arriving at a
message center: 0 ≤ t, uncountably infinite number of outcomes.

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 3 / 41
Random variable
Random variable is a function that assigns a real number to each
outcome in the sample space of a random experiment.

Flip 3 coins at the same time:

HHH 3 heads: x =3
HHT 2 heads: x =2
HTH 2 heads: x =2
HTT 1 head: x =1
THH 2 heads: x =2
THT 1 head: x =1
TTH 1 head: x =1
TTT 0 heads: x =0

Random variable: X : Ω → {0, 1, 2, 3}, (discrete).

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 4 / 41
Discrete and Continuous Random Variables
A discrete random variable is a random variable with a finite (or
countably infinite) range.
A continuous random variable is a random variable with an interval
(either finite or infinite) of real numbers for its range.

Weight of a watermelon: Continuous

Marital status: NO
Time taken to get a fire. Continuous
Number of emergency calls in a hour: Discrete
Length of a tulip stem: Continuous
Color of a tulip flower: NO

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 5 / 41
Discrete random variable. Probability Distributions and
Probability Mass Function
Flip 3 coins, consider the number of heads:
x 0 1 2 3
- Probability Distribution
P(X = x) 18 38 38 18
f (0) = P(X = 0) = 18 ,
f (1) = 83 = f (2),
f (3) = 81 .
f (0) + f (1) + f (2) + f (3) = 1
f : Prob. Mass Function.

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 6 / 41
Probability Distributions
The probability distribution of a random variable X is a description of
the probabilities associated with the possible values of X .

Probability Mass Function (PMF)

For a discrete random variable X with possible values x1 , x2 , ..., xn , a
probability mass function is a function such that
f (xi ) ≥ 0
P n
f (xi ) = 1
i=1
f (xi ) = P(X = xi )

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 7 / 41
Camera flash tests
The time to recharge the flash is tested in three cell-phone cameras. The
probability that a camera passes the test is 0.8, and the cameras perform
independently. Consider the number of cameras that pass the test.

P(X = 0) = 0.008
P(X = 1) = 0.096
P(X = 2) = 0.384
P(X = 3) = 0.512

.
(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 8 / 41
Discrete random variables. Cumulative Distribution
Function
Consider the number of cameras that pass the test
x 0 1 2 3
f (x) = P(X = x) 0.008 0.096 0.384 0.512
Consider: F (2.5) = P(X ≤ 2.5) = 0.008 + 0.096 + 0.384 = 0.488

F (x) = P(X ≤ x)
 =

0, if x < 0,

0.008, if 0 ≤ x < 1,



0.104, if 1 ≤ x < 2,

0.488, if 2 ≤ x < 3,





1, if x ≥ 3

- Cumulative Distribution
Function (CDF)
(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 9 / 41
Cumulative Distribution Functions (CDF)
The cumulative distribution function of a discrete random variable X ,
denoted as F (x), is
P
F (x) = P(X ≤ x) = f (xi )
xi ≤x



0, if x < 0,

0.008, if 0 ≤ x < 1,



F (x) = 0.104, if 1 ≤ x < 2,

0.488, if 2 ≤ x < 3,





1, if x ≥3


(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 10 / 41
Now, suppose that we have the CDF F (x), how can we define PMF f (x)?


0, if x < 0,

0.008, if 0 ≤ x < 1,



F (x) = 0.104, if 1 ≤ x < 2,

0.488, if 2 ≤ x < 3,





1, if x ≥3


We have: f (x) = P(X = x) = P(X ≤ x) − P(X < x), therefore:

f (0) = P(X = 0) = P(X ≤ 0) − P(X < 0) = 0.008 − 0 = 0.008;
f (3) = P(X ≤ 3) − P(X < 3) = 1 − 0.488 = 0.512;
f (2.5) = P(X ≤ 2.5) − P(X < 2.5) = 0.488 − 0.488 = 0.

Note: If F (x) is continuous at x, then f (x) = P(X = x) = 0.

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 11 / 41
Properties of Cumulative Distribution Functions
For a discrete random variable X , F (x) satisfies the following properties
P
F (x) = P(X ≤ x) = f (xi ).
xi ≤x
0 ≤ F (x) ≤ 1.
If x ≤ y , then F (x) ≤ F (y ).

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 12 / 41
Continuous random variables: Probability Density
Functions

Probability Density Functions (PDF)

For a continuous random variable X , a probability density function is a
function such that
1 f (x) ≥ 0
R∞
2 f (x)dx = 1
−∞
Rb
3 P(a ≤ x ≤ b) = f (x)dx = area under f (x) from a to b for any a, b.
a

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 13 / 41
Electric Current
Let the continuous random variable X denote the current measured in a
thin copper wire in milliamperes. Assume that the range of X is [4.9, 5.1]
mA, and assume that the probability density function of X is f (x) = 5 for
4.9 ≤ x ≤ 5.1. What is the probability that a current measurement is less
than 5 milliamperes?

Z5
P(X < 5) = f (x)dx
−∞
Z5
= 5dx
4.9
= 0.5.

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 14 / 41
Hole Diameter
X denotes the diameter of a hole drilled in a sheet metal component. The
target diameter is 12.5 millimeters. Historical data show that the
distribution of X can be modeled by a probability density function
f (x) = 20e −20(x−12.5) , for x ≥ 12.5. If a part with a diameter greater than
12.60 mm is scrapped, what proportion of parts is scrapped?

Z∞
P(X > 12.6) = f (x)dx
12.6
Z∞
= 20e −20(x−12.5) dx
12.6
−2
=e .

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 15 / 41
Cumulative Distribution Functions (CDF)
The cumulative distribution function of a continuous random variable X
is
Rx
F (x) = P(X ≤ x) = f (u)du, for −∞ < x < ∞.
−∞

dF (x)
Property: = f (x) (as long as the derivative exists).
dx

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 16 / 41
Probability Density Function (PDF) Cumulative Distribution Function
(CDF)

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 17 / 41
Example
Define the Probability density function
( PDF , given the cumulative
0, if x < 0,
distribution function CDF : F (x) =
1 − e −2x , if x > 0.
(
0, if x < 0,
f (x) =
2xe −2x , if x > 0.

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 18 / 41
(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 19 / 41
Consumer Spending

Consumer spending of a family from 2013 to 2018 is

Year 2013 2014 2015 2016 2017 2018
Spending $54, 720 $51, 095 $60, 000 $55, 524 $70, 373 $60, 000

Mean:
54.72 + 51.095 + ... + 60
µ= = 58.619 (thousand $)
6
Variance:
(54.72 − µ)2 + (51.095 − µ)2 + ... + (60 − µ)2
σ2 =
6
= 32.227 ((thousand $)2 )
Standard deviation:
√
σ = σ 2 = 6.1014 (thousand $)

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 20 / 41
Messages
The number of e-mail messages received per hour has the following
x = ] of messages 10 11 12 13 14 15
distribution:
f (x) = P(X = x) 0.08 0.15 0.3 0.2 0.2 0.07

Expected value (mean/expectation):

P
E (X ) = xf (x) = 10 × 0.08 + 11 × 0.15 + · · · + 15 × 0.07
x
= 12.5 (message).

Expectation (mean/expected value)

The mean or expected value of the discrete random variable X , denoted
as µ or E (X ), is
P P
E (X ) = xf (x) = xP(X = x).
x x

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 21 / 41
 x = ] of messages 10 11 12 13 14 15
f (x) = P(X = x) 0.08 0.15 0.3 0.2 0.2 0.07
Variance:
V (X ) = (x − µ)2 f (x)
P
x
= (10 − 12.5)2 0.08 + (11 − 12.5)2 0.15 + · · · + (15 − 12.5)2 0.07
= 1.85 (message 2 ) measures how far a set of data is spread out.
V (x) = 0: the data values are identical.
Variance
The variance of X , denoted as σ 2 or V (X ), is

V (x) = E (X − µ)2 = (x − µ)2 f (x) = x 2 f (x) − µ2

P P
x x

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 22 / 41
 x = ] of messages 10 11 12 13 14 15
f (x) = P(X = x) 0.08 0.15 0.3 0.2 0.2 0.07
p √
Standard deviation: σ = V (x) = 1.85 = 1.36 (message)

Standard deviation
The standard deviation of X , denoted as σ is
√
σ = σ2.

Standard deviation shows how much variation (dispersion, spread, scatter)

from the mean exists. It represents a ”typical” deviation from the mean.

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 23 / 41
(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 24 / 41
Mean, Variance, Standard Deviation of continuous random variables.
Suppose that X is a continuous random variable with probability density
function f (x).
The mean or expected value of X , denoted as µ or E (X ), is
R∞
µ = E (X ) = xf (x)dx
−∞
The variance of X , denoted as V (X ) or σ 2 is
R∞ R∞ 2
σ 2 = V (x) = (x − µ)2 f (x)dx = x f (x)dx − µ2
−∞ −∞
The standard deviation of X is
√
σ= σ2.

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 25 / 41
Example
The PDF of the weight of packages delivered by a post office is
f (x) = 70/(69x 2 ) for 1 < x < 70 pounds.
(a)) Determine the probability that the weight of a package exceeds 50
pounds.
(b) Determine the mean and variance of weight.
(c) If the shipping cost is $2.50 per pound, what is the average shipping
cost of a package?
R70 70
(a) P(X > 50) = 2
dx = 0.0058
50 69x
R70 70
(b) E (x) = x. dx = 4.3101 (pound)
1 69x 2
R70 70
V (x) = x 2 . dx − 4.31012 = 51.4233 (pound 2 )
1 69x 2
(c) Ecost = 2.5 × 4.3101 = 10.7752 ($)

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 26 / 41
Expectation of a Function of a Random Variable

Expected Value of a Function of a Random Variable

If X is a discrete random variable with probability mass function
f (x),
P
E [h(X )] = h(x)f (x)
x
If X is a continuous random variable with probability mass function
f (x),
R∞
E [h(X )] = h(x)f (x)dx
−∞

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 27 / 41
Properties of Expectation
1 Let a and b be constants, g and h be functions. For any random
variable X :
E [ag (X ) + bh(X )] = aE [g (X )] + bE [h(X )]
2 In particular
E (aX + b) = aE (X ) + b

Properties of Variance
1 Let a and b be constants, g and h be functions. For any random
variable X :
V [ag (X ) + b] = a2 V [g (X )]
2 In particular
V (aX + b) = a2 V (x)

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 28 / 41
Mobile Response Time

The response time is the speed of page downloads and it is critical for a
mobile Web site. Let X denote the number of bars of service, and let Y
denote the response time (to the nearest second) for a particular user and
site. Consider 1000 page downloads, the number of downloads in each
category is given as below
x = ] of Bars of Signal Strength
y = Response time 1 2 3
4 150 100 50
3 20 100 50
2 20 30 200
1 10 20 250

- joint events.
(X , Y ) is a discrete random vector.

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 29 / 41
Mobile Response Time

Divide all data by 1000, we obtain

x = Number of Bars of Signal Strength
y = Response time 1 2 3
4 0.15 0.1 0.05
3 0.02 0.1 0.05
2 0.02 0.03 0.2
1 0.01 0.02 0.25

- joint probability distribution, (2 random variables: bivariate

distribution).
fXY (2, 3) = P(X = 2, Y = 3) = 0.1.
fXY : Joint Probability Mass Function

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 30 / 41
Discrete random vectors (2 random variables)
Let Ω be a sample space. A pair of discrete random variables (X , Y )
defined on Ω is called a discrete random vector.

Joint Probability Mass Function

The joint probability mass function of the discrete random variables X and
Y , denoted as fXY (x, y ), satisfies
1 fXY (x, y ) ≥ 0
PP
2 fXY (x, y ) = 1
X Y
3 fXY (x, y ) = P(X = x, Y = y )

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 31 / 41
 x = Number of Bars of Signal Strength
y = Response time 1 2 3 fY
4 0.15 0.1 0.05 0.3
3 0.02 0.1 0.05 0.17
2 0.02 0.03 0.2 0.25
1 0.01 0.02 0.25 0.28
fX 0.2 0.25 0.55
- Marginal Probability distributions of X and Y .
fX (x = 1) = 0.2, fY (y = 2) = 0.25.
fX , fY : Marginal Probability Mass Functions .
Marginal Distribution
The individual probability distribution of a random variable is called its
marginal probability distribution.

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 32 / 41
Conditional probability distributions
Given that X = 1
x = Number of Bars of Signal Strength
y = Response time 1 2 3 fY
4 0.15 0.1 0.05 0.3
3 0.02 0.1 0.05 0.17
2 0.02 0.03 0.2 0.25
1 0.01 0.02 0.25 0.28
fX 0.2 0.25 0.55
0.15
P(Y = 4|X = 1) = = 0.75.
0.2
0.02
P(Y = 3|X = 1) = = 0.1.
0.2
0.02
P(Y = 2|X = 1) = = 0.1.
0.2
0.01
P(Y = 1|X = 1) = = 0.05.
0.2
P(X = x, Y = y ) fXY (x, y )
fY |x (y ) = = - Conditional PMF.
P(X = x) fX (x)
(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 33 / 41
Conditional probability distributions

Conditional probability distributions of Y given X = x, fY |x (y ):

x = Number of Bars of Signal Strength

y = Response time 1 2 3
4 0.75 0.4 0.091
3 0.1 0.4 0.91
2 0.1 0.12 0.364
1 0.05 0.08 0.454
Total 1 1 1

Conditional probability mass function

For discrete random vector (X , Y ), the conditional probability mass
function of Y given X = x is
P(X = x, Y = y ) fXY (x, y )
fY |x (y ) = = , for fX (x) > 0.
P(X = x) fX (x)

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 34 / 41
Conditional expectation and Variance
The conditional expectation of Y given X = x, denoted as E (Y |x)
or µY |x , is
P
E (Y |x) = yfY |x (y )
y
The conditional variance of Y given X = x, denoted as V (Y |x), is
V (Y |x) = E [(Y − µY |x )2 ] = (y − µY |x )2 .fY |x (y )
P
y
= y 2 .fY |x (y ) − µ2Y |x
P
y

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 35 / 41
Conditional Expectation and Variance

Conditional probability distributions of Y given X = x, fY |x (y ):

x = Number of Bars of Signal Strength

y = Response time 1 2 3
4 0.75 0.4 0.091
3 0.1 0.4 0.91
2 0.1 0.12 0.364
1 0.05 0.08 0.454
Total 1 1 1

E (Y |x = 1) = 4 × 0.75 + 3 × 0.1 + 2 × 0.1 + 1 × 0.05

= 3.55 (second)
V (Y |x = 1) = 42 × 0.75 + 32 × 0.1 + 22 × 0.1 + 0.05 − 3.552
= 0.7475 (second 2 ).

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 36 / 41
Independent random variables

An orthopedic physician’s practice considers the number of errors in a bill

and the number of X-rays listed on the bill. There may or may not be a
relationship between these random variables. Let the random variables X
and Y denote the number of errors and the number of X-rays on a bill,
respectively.
x = Number of errors
y = Number of X-rays 0 1 2 fY
3 0.1275 0.034 0.0085 0.17
2 0.1875 0.05 0.0125 0.25
1 0.21 0.056 0.014 0.28
0 0.225 0.060 0.015 0.3
fX 0.75 0.2 0.05

We consider the conditional probability mass function fY |x of Y given

X =x
(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 37 / 41
Independent random variables

The conditional probability mass function fY |x of Y given X = x

x = Number of errors
y = Number of X-rays 0 1 2 fY
3 0.17 0.17 0.17 0.17
2 0.25 0.25 0.25 0.25
1 0.28 0.28 0.28 0.28
0 0.3 0.3 0.3 0.3
fX 0.75 0.2 0.05

From the table, we have that fY |x (y ) = fY (y ) for any y : X and Y are

independent random variables.

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 38 / 41
Independence
Two random variables X and Y are called independent , if one of the
following equivalent properties is true
1 fXY (x, y ) = fX (x)fY (y ), ∀x, y
2 fY |x (y ) = fY (y ), ∀x, y : fX (x) > 0
3 fX |y (x) = fX (x), ∀x, y : fY (y ) > 0
4 P(X ∈ A, Y ∈ B) = P(X ∈ A)P(Y ∈ B),
for any sets A, B in the range of X , Y , resp.

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 39 / 41
Linear function and random variables

Linear function
Given random variables X1 , X2 , ...Xp and constants c1 , c2 , ...cp ,
Y = c1 X1 + c2 X2 + .... + cp Xp is a linear combination of X1 , X2 , ...Xp .

Mean of a linear function

If Y = c1 X1 + c2 X2 + .... + cp Xp , then

E (Y ) = c1 E (X1 ) + c2 E (X2 ) + ... + cp E (Xp ).

Variance of a linear function

If X1 , X2 ...Xp are independent and Y = c1 X1 + c2 X2 + .... + cp Xp , then

V (Y ) = c12 V (X1 ) + c22 V (X2 ) + ... + cp2 V (Xp ).

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 40 / 41
 Thank you for your attention!

(Phan Thi Khanh Van) Chapter 2: Random variables, random vectors October 25, 2021 41 / 41