Lecture 04.

Continuous Random Variables

(Chapter 5)

Ping Yu

HKU Business School

The University of Hong Kong

Ping Yu (HKU) Continuous Random Variables 1 / 35

Plan of This Lecture

Continuous Random Variables

Expectations for Continuous Random Variables
The Normal Distribution
Normal Distribution Approximation for Binomial Distribution
The Exponential Distribution
Jointly Distributed Continuous Random Variables

Ping Yu (HKU) Continuous Random Variables 2 / 35

 Continuous Random Variables

Continuous Random Variables

Ping Yu (HKU) Continuous Random Variables 3 / 35

 Continuous Random Variables

Cumulative Distribution Function

The cumulative distribution function (cdf), F (x ), for a continuous r.v. expresses

the probability that X does not exceed the value x, as a function of x, i.e.,

F (x ) = P (X x).

- This definition is the same as in the discrete r.v. case, but there F (x ) is a step
function so is not differentiable.
- This definition of cdf implies P (a < X b ) = F (b ) F (a); recall that the
probability of a single value is zero for a continuous r.v., so whether a and b are
included in the interval or not does not affect the result.
The counterpart of pmf for a continuous r.v. is the probability density function (pdf),
which is defined as
d
f (x ) = F (x ) . [figure here]
dx
- Since F (x ) is nondecreasing, f (x ) 0. We denote the area where f (x ) > 0 as
S , called the support
Rb
of X .1 R R
- P (a X b ) = a f (x )dx, ∞∞ f (x )dx = S f (x )dx = 1 and
Rx Rx
F (x ) = ∞ f (x )dx = xm f (x )dx, where xm = inf (S ). [figure here]

1
(**) Usually, S is defined as the closure of this area, but we will not distinguish this difference in this lecture.
Ping Yu (HKU) Continuous Random Variables 4 / 35
 Continuous Random Variables

S-Shaped CDF Implies the P (a X b )

R
Bell-Shaped PDF at Right = ab f (x )dx

R Rx
S f (x )dx = 1 and F (x ) = xm f (x )dx

Ping Yu (HKU) Continuous Random Variables 5 / 35

 Continuous Random Variables

Example: Gasoline Sales

Assume the gasoline sales at a gasoline station is equally likely from 0 to 1,000
gallons during a day; then the gasoline sales follow a uniform (probability)
distribution: 8
< 0, if x < 0,
F (x ) = 0.001x, if 0 x 1000
:
1, if x > 1000,
whose pdf is
0.001, if 0 x 1000
f (x ) =
0, otherwise.
= 0.001 1(0 x 1000),
where 1( ) is the indicator function which equals 1 when the statement in the
parentheses is true and 0 otherwise. [figure here]
In general, the uniform distribution on (a, b ) has the pdf
1
f (x ) = 1(a x b ),
b a
and the cdf
x a
F (x ) = for x 2 [a, b ] .
b a
Ping Yu (HKU) Continuous Random Variables 6 / 35
 Continuous Random Variables

-3
10
1.2

1
1

0.5

0 0
-200 0 500 1000 1200 -200 0 500 1000 1,200

CDF PDF

Degenerate S-shaped cdf and bell-shaped pdf?

We denote a r.v. X with a uniform distribution on (a, b ) as X U (a, b ).

Ping Yu (HKU) Continuous Random Variables 7 / 35

 Continuous Random Variables

Approximate PDF by PMF

Suppose S = (a, b ), where a can be ∞ and b can be ∞. We can partition S into

small subintervals with length ∆, and then approximate the pdf f (x ) by the pmf
Z a+(i +1)∆
1
p a+ i + ∆ = P (a + i∆ < X a + (i + 1) ∆ ) = f (x ) dx,
2 a+i∆

where i = 0, 1, , b∆a 1.2

Figure: PDF of Wage: wage exp N µ, σ 2 with N µ, σ 2 defined below, a = 0, b = ∞

2
i = 0, a + i∆ = a, and i = b a
∆ 1, a + (i + 1) ∆ = a + ∆ ∆ = b.
b a

Ping Yu (HKU) Continuous Random Variables 8 / 35

 Expectations for Continuous Random Variables

Expectations for Continuous Random Variables

Ping Yu (HKU) Continuous Random Variables 9 / 35

 Expectations for Continuous Random Variables

Mean

The mean (or expected value, or expectation) of a continuous r.v. can be defined
through an approximation of a discrete r.v. in the previous slide:
(b a)/∆ 1
µ X : = E [X ] ∑ a + i + 21 ∆ P (a + i∆ < X a + (i + 1) ∆ )
i =0
(b a)/∆ 1
∑ a + i + 21 ∆ f a + i + 21 ∆ ∆
i =0
∆!0 R b
! a xf (x )dx.
R
∑ ! , a + i + 21 ∆ ! x, and ∆ ! dx.
The mean is the center of gravity of a pole (a, b ) with density at x being f (x ).
In general, the mean of any function of X , g (X ), is
Z
E [g (X )] = g (x )f (x ) dx.
S

- Recall that E [g (X )] 6= g (E [X ]) unless g (X ) is linear in X .

Ping Yu (HKU) Continuous Random Variables 10 / 35

 Expectations for Continuous Random Variables

Variance

The variance of X is defined as

h i h i
σ 2X = E (X µ X )2 = E X 2 µ 2X .

- µ X measures the center of the distribution, while σ 2X measures the dispersion or

spread of the distribution.
q
The standard deviation of X , σ X = σ 2X .
Example: For the uniform distribution on (a, b ),
Z b
1 a+b
µX = x dx = ,
a b a 2
Z b 2
1 a+b (b a )2
σ 2X = x2 dx = ,
a b a 2 12
i.e., the mean is the center of the range, and when the range (a, b ) is wider, the
variance is larger.
For W = a + bX with a and b being constant fixed numbers,
µ W = a + bµ X , σ 2W = b2 σ 2X and σ W = jbj σ X .
X µX
- For the z-score of X , Z = σX , µ Z = 0 and σ 2Z = 1.
Ping Yu (HKU) Continuous Random Variables 11 / 35
 The Normal Distribution

The Normal Distribution

Ping Yu (HKU) Continuous Random Variables 12 / 35

 The Normal Distribution

The Normal Distribution

The pdf for a normally distributed r.v. X is

1 (x µ )2
f xjµ, σ 2 = p e 2σ 2 for ∞ < x < ∞, (1)
2πσ 2
where µ 2 R, and σ 2 2 (0, ∞), e is Euler’s number, and π = 3.14159 is
Archimedes’ constant (the ratio of a circle’s circumference to its diameter).
- Since the normal distribution depends only on µ and σ 2 , we denote a r.v. X with
pdf (1) as X N µ, σ 2 .
R
The cdf of the normal distribution, F xjµ, σ 2 = x ∞ f xjµ, σ 2 dx, does not have
an analytic form (i.e., a closed-form expression), but computation of probabilities
based on the normal distribution is direct nowadays.
This distribution has many applications in business and economics, e.g., the
dimensions of parts, the heights and weights of human beings, the test scores, the
stock prices, etc. all roughly follow normal distributions.
As will be discussed in Lecture 5, the distribution of sample mean will converge to
a normal distribution when the sample size gets large.

Ping Yu (HKU) Continuous Random Variables 13 / 35

 The Normal Distribution

continue

Normal Distribution is also called Gaussian Distribution.

Carl F. Gauss (1777-1855), Göttingen3

3
He is also referred to as the "prince of mathematics", known for many things, e.g., the least squares.
Ping Yu (HKU) Continuous Random Variables 14 / 35
 The Normal Distribution

Properties of the Normal Distribution: Mean, Variance and Skewness

For X N µ, σ 2 ,

µ X = µ [easy] and σ 2X = σ 2 [proof not required],

so a normal distribution is determined completely by its mean and variance.

The shape of f xjµ, σ 2 is a symmetric bell-shaped curve centered on the mean

µ, which implies the skewness of X is 0. [figure here]
- Recall from Lecture 1 that the (population) skewness is
h i
E (X µ X )3 µ
SkewX = 3
=: 33 ,
σX σ

and the sample skewness is

3
∑ni=1 (xi x̄ )
skewness = ,
ns3
where µ 3 is the third central moment, and s is the sample standard deviation.

Ping Yu (HKU) Continuous Random Variables 15 / 35

 The Normal Distribution

Figure: f xjµ, σ 2 with Different µ and σ 2 ’s: (a) same σ 2 , different µ’s; (b) same µ = 5, different
σ 2 ’s.

The normal distribution is a symmetric, bell-shaped distribution with a single peak.

Its peak corresponds to the mean, median, and mode of the distribution.

Ping Yu (HKU) Continuous Random Variables 16 / 35

 The Normal Distribution

Properties of the Normal Distribution: Kurtosis

2
The tail of f xjµ, σ 2 is approximately e x which shrinks to zero very quickly.
The (population) kurtosis is often used to measure the heaviness of a distribution’s
tail [why?]: h i
E (X µ X )4 µ
KurtX = =: 44 ,
σ 4X σ
where µ 4 is the fourth central moment.
- It can be shown that the kurtosis of N µ, σ 2 is 3, which is chosen as a
benchmark, i.e., if a distribution’s kurtosis is larger than 3, it is called heavy tailed,
and if less than 3, called light tailed. Heavy-tailed phenomena seem more frequent
than light-tailed ones since the tail of a normal distribution is already very thin.
The sample kurtosis is
4
∑ni=1 (xi x̄ )
kurtosis = 4
.
ns

Ping Yu (HKU) Continuous Random Variables 17 / 35

 The Normal Distribution

The Standard Normal Distribution

If X N (0, 1), then we call X follows the standard normal distribution.

Because Z = X σ µ has mean 0 and variance 1 for X N µ, σ 2 , and the normal
distribution is completely determined by its mean and variance, we conclude that
Z N (0, 1) if X N µ, σ 2 .4
The pdf of N (0, 1) is often denoted as φ ( ), and the cdf is denoted as Φ ( ).
The upper αth quantile of N (0, 1), i.e., the solution to 1 Φ (z ) = α, is denoted as
zα . [The αth quantile of N (0, 1) is the solution to Φ (z ) = α, i.e., Φ 1 (α )]
- By symmetry of Φ ( ), 1 Φ (z ) = Φ ( z ), so zα = Φ 1 (α ). [figure here]
The relationship between F xjµ, σ 2 and Φ (z ) and between f xjµ, σ 2 and
φ (z ):
X µ x µ x µ
F xjµ, σ 2 = P (X x) = P =Φ ,
σ σ σ
d x µ 1 x µ
f xjµ, σ 2 = Φ = φ ,
dx σ σ σ
where the second result is from the chain rule and Φ0 ( ) = φ ( ).
4
(**) Linear transformations maintain normality, but nonlinear ones need not.
Ping Yu (HKU) Continuous Random Variables 18 / 35
 The Normal Distribution

Figure: Normal Density Function with Symmetric Upper and Lower Values

Ping Yu (HKU) Continuous Random Variables 19 / 35

 The Normal Distribution

Normal Probability Plot

Since the normal distribution is most-used, we often need a way to check whether
the data in hand are approximately normally distributed.
Normal probability plots provide an easy way to achieve this goal; Lecture 8 will
provide a more rigorous test.
If the data are indeed from a normal distribution, then the plot will be a straight
line. [figure here]
- The vertical axis has a transformed cumulative normal scale.
- Two dotted lines provide an interval within which data points from a normal
distribution would occur most cases.
If the data are not from a normal distribution, then the plot will deviate from a
straight line. [figure here]

Ping Yu (HKU) Continuous Random Variables 20 / 35

 The Normal Distribution

Figure: Normal Probability Plot for Data Simulated from a Normal Distribution

Ping Yu (HKU) Continuous Random Variables 21 / 35

 The Normal Distribution

Figure: Normal Probability Plot for Data Simulated from a Uniform Distribution

Ping Yu (HKU) Continuous Random Variables 22 / 35

 Normal Distribution Approximation for Binomial Distribution

Normal Distribution Approximation for Binomial

Distribution

Ping Yu (HKU) Continuous Random Variables 23 / 35

 Normal Distribution Approximation for Binomial Distribution

Normal Distribution Approximation for Binomial Distribution

iid
Recall that a binomial r.v. X = ∑ni=1 Xi , where Xi Bernoulli(p ) with iid meaning
"independent and identically distributed".
When np (1 p ) > 5, N (np, np (1 p )) provides a good approximation of
Binomial(n, p ), which is known as the De Moivre-Laplace theorem. [figure here]
- A more rigorous justification when p is fixed and n ! ∞ is provided in Lecture 5.
- Interestingly, we are using a continuous r.v. to approximate a discrete r.v..

iid
Figure: Galton Board (or quincunx, or bean machine): X = ∑ni=1 Xi , where Xi Bernoulli(0.5)
2 f 1, 1g

Ping Yu (HKU) Continuous Random Variables 24 / 35

 Normal Distribution Approximation for Binomial Distribution

Abraham de Moivre (1667-1754), French5 Pierre-Simon Laplace (1749-1827), French6

The phonomenon of de Moivre–Laplace theorem was first observed by de Moivre

in a private manuscript circulated in 1733 and published in 1738 with the title "The
Doctrine of Chances". Later, Laplace formally proved the theorem in 1810.

5
He was exiled to England due to religious persecution, and became a friend of Newton there. To make a
living, he became a private tutor of mathematics.
6
He was referred to as the French Newton. His students include Poisson and Napoleon.
Ping Yu (HKU) Continuous Random Variables 25 / 35
 Normal Distribution Approximation for Binomial Distribution

continue

Because normal distributions are easier to handle, this approximation can simplify
the analysis of some problems. For example, if X is the number of customers after
n people browsed a store’s website, and based on past experiences, the
probability of visiting the store after browsing is p, the manager wants to predict
the probability of the number of customers falling in an interval, say, [a, b ].
- From the normal approximation,
!
a np X np b np
P (a X b ) = P p p p
np (1 p ) np (1 p ) np (1 p )
! !
b np a np
Φ p Φ p .
np (1 p ) np (1 p )

The normal approximation can also be applied to the proportion (or percentage)
r.v., P = X /n:
approx. np np (1 p ) P (1 P )
P N , N P, ,
n n2 n
where the last approximation is from using P to substitute p, i.e., p is estimated
rather than known a priori.
Ping Yu (HKU) Continuous Random Variables 26 / 35
 Normal Distribution Approximation for Binomial Distribution

Summary of Approximations of Binomial(n, p )

Conditions Approximating Distributions

n large and p 0.05 such that np 7 Poisson(np )
N large and Nn small S
Hypergeometric(n, N, S ) with N p
n large such that np (1 p ) > 5 N (np, np (1 p ))

Ping Yu (HKU) Continuous Random Variables 27 / 35

 The Exponential Distribution

The Exponential Distribution

Ping Yu (HKU) Continuous Random Variables 28 / 35

 The Exponential Distribution

The Exponential Distribution

The pdf for an exponentially distributed r.v. T is
λt
f (tjλ ) = λ e 1(t > 0),
denoted as T Exponential(λ ), i.e., T can take only positive values, and the
distribution is not symmetric – the right tail is heavier than that of the normal
distribution, and the left tail is thinner. [figure here]
- The cdf of exponential(λ ) is F (tjλ ) = 1 e λ t 1(t > 0), which implies that
the survivor function is P (T > t ) = 1 F (tjλ ) = e λt for t > 0.

Figure: f (tjλ ) with λ = 0.2

Ping Yu (HKU) Continuous Random Variables 29 / 35

 The Exponential Distribution

Properties of the Exponential Distribution

F (tjλ ) can be used to model the waiting time, i.e., the probability that an arrival
will occur during an interval of time t (T is the waiting time before the first arrival),
so is particularly useful for waiting-line, or queuing, problems.
The exponential distribution is closely related to the Poisson distribution: If
iid
T1 , , Tn Exponential(λ ), then
n o
max nj ∑i =1 Ti 1
n
Poisson (λ ) ; [proof not required]

i.e., Poisson(λ ) is the number of arrivals in a unit time.

For T Exponential(λ ),
1 1
and σ 2T = 2 ,
µT =
λ λ
so an exponential distribution is determined completely by its mean.
- µ T = λ1 is expected from the mean of Poisson(λ ) which is λ from Lecture 4.

Ping Yu (HKU) Continuous Random Variables 30 / 35

 The Exponential Distribution

continue

Memorylessness:
P (T > s + t \ T > s )
P (T > s + tjT > s ) =
P (T > s )
P (T > s + t )
=
P (T > s )
e λ (s +t )
= λs
e
λt
= e
= P (T > t ) .

- If T is conditioned on a failure to observe the event over some initial period of

time s, the distribution of the remaining waiting time is the same as the original
unconditional distribution.

Ping Yu (HKU) Continuous Random Variables 31 / 35

 Jointly Distributed Continuous Random Variables

Jointly Distributed Continuous Random Variables

Ping Yu (HKU) Continuous Random Variables 32 / 35

 Jointly Distributed Continuous Random Variables

Jointly Distributed Continuous R.V.’s

This section is parallel to the section on multivariate discrete r.v.’s.

Let X1 , , XK be continuous r.v.’s.
1 Their joint cdf
F (x1 , , xK ) = P (X1 x1 \ \ XK xk ) .
2 The cdf, F (x1 ) , , F (xK ), of individual r.v.’s are called their marginal distributions.
3 The r.v.’s are independent iff for all x1 , , xK ,
F (x1 , , xK ) = F (x1 ) F (xK ) .

The counterparts of the joint and marginal probability distributions for multivariate
discrete r.v.’s are the joint pdf
dn
f ( x1 , , xK ) = F (x1 , , xK )
dx1 dxK
and the marginal pdf
Z Z
d
f (xi ) = F ( xi ) = f (x1 , , xK ) dx1 dxi 1 dxi +1 dxK .
dxi
- The independence of X1 , , XK can be equivalently defined as
f (x1 , , xK ) = f (x1 ) f (xK ) for all x1 , , xK .
Ping Yu (HKU) Continuous Random Variables 33 / 35
 Jointly Distributed Continuous Random Variables

Conditional Mean and Variance

For two continuous r.v.’s (X , Y ), the conditional pdf of Y given X = x is

f (x, y )
f (y jx ) = .
f (x )

The conditional mean of Y given X = x is

Z
µ Y jX =x = E [Y jX = x ] = yf (y jx ) dy .

The conditional variance of Y given X = x is

Z 2
σ 2Y jX =x = Var (Y jX = x ) = y µ Y jX =x f (y jx ) dy .

These concepts can be extended to multivariate continuous r.v.’s in an obvious

way.
The results such as E [ a + bY j X = x ] = a + bE [Y jX = x ] and
Var (a + bY jX = x ) = b2 Var (Y jX = x ) for constants a and b also hold.

Ping Yu (HKU) Continuous Random Variables 34 / 35

 Jointly Distributed Continuous Random Variables

Mean and Variance of (Linear) Functions

For a function of X1 , , XK , g (X1 , , XK ), its mean, E [g (X1 , , XK )], is defined

as Z Z
E [g (X1 , , XK )] = g (x1 , , xK ) f (x1 , , xK ) dx1 dxK .

If W = ∑K
i =1 ai Xi , then
K
µ W = E [W ] = ∑ ai µ i ,
i =1
and
σ 2W = Var (W ) = ∑i =1 ai2 σ 2i + 2 ∑i =1 ∑j>i ai aj σ ij ,
K K 1 K

which reduces to ∑K 2
i =1 σ i if σ ij = 0 for all i 6= j and ai = 1 for all i.
Actually, all the results on mean and variance for discrete r.v.’s apply to continuous
r.v.’s.
The covariance and correlation between two continuous r.v.’s (X , Y ) are similarly
defined as
Cov (X , Y ) = σ XY = E [(X µ X ) (Y µ Y )] = E [XY ] µX µY ,
σ XY
Corr (X , Y ) = ρ XY = .
σX σY

Ping Yu (HKU) Continuous Random Variables 35 / 35