5 Continuous random variables

We deviate from the order in the book for this chapter, so the subsections in
this chapter do not correspond to those in the text.
5.1 Densities of continuous random variable
Recall that in general a random variable X is a function from the sample
space to the real numbers. If the range of X is nite or countable innite,
we say X is a discrete random variable. We now consider random variables
whose range is not countably innite or nite. For example, the range of X
could be an interval, or the entire real line.
For discrete random variables the probability mass function is f
X
(x) =
P(X = x). If we want to compute the probability that X lies in some set,
e.g., an interval [a, b], we sum the pmf:
P(a X b) =

x:axb
f
X
(x)
A special case of this is
P(X b) =

x:xb
f
X
(x)
For continuous random variables, we will have integrals instead of sums.
Denition 1. A random variable X is continuous if there is a non-negative
function f
X
(x), called the probability density function (pdf ) or just density,
such that
P(X t) =
_
t

f
X
(x) dx
Proposition 1. If X is a continuous random variable with density f(x),
then
1. P(X = x) = 0 for any x R.
2. P(a X b) =
_
b
a
f(x) dx
3. For any subset C of R, P(X C) =
_
C
f(x) dx
1
4.
_

f(x) dx = 1
Proof. First we observe that subtracting the two equations
P(X b) =
_
b

f
X
(x) dx, P(X a) =
_
a

f
X
(x) dx
gives
P(X b) P(X a) =
_
b
a
f
X
(x) dx
and we have P(X b) P(X a) = P(a < X b), so
P(a < X b) =
_
b
a
f
X
(x) dx (1)
Now for any n
P(X = x) P(x 1/n < X x) =
_
x
x1/n
f
X
(t) dt
As n , the integral goes to zero, so P(X = x) = 0.
Property 2 now follows from eq. (1) since
P(a X b) = P(a < X b) +P(X = a) = P(a < X b)
Note that since the probability X equals any single real number is zero,
P(a X b), P(a < X b), P(a X < b), and P(a < X < b) are all the
same.
Property 3 is easy if C is a disjoint union of intervals. For more general
sets, it is not clear what
_
C
even means. This is beyond the scope of this
course.
Property 4 is just the fact that P(< X < ) = 1.
Caution Often the range of X is not the entire real line. Outside of the
range of X the density f
X
(x) is zero. So the denition of f
x
(x) will typically
involves cases: in one region it is given by some formula, elsewhere it is simply
0. So integrals over all of R which contain f
X
(x) will reduce to intervals over
a subset of R. If you mistakenly integrate the formula over the entire real
line you will of course get nonsense.
2
5.2 Catalog
As with discrete RVs, two continuous RVs dened on completely dierent
probability spaces can have the same density. And there are certain densities
that come up a lot. So we start a catalog of them.
Uniform: (two parameters a, b R with a < b) The uniform density on
[a, b] is
f(x) =
_
1
ba
, if a x b
0, otherwise
We have seen the uniform before. Previously we said that to compute the
probability X is in some subinterval [c, d] of [a, b] you take the length of that
subinterval divided by the length of [a, b]. This is of course what you get
when you compute
_
d
c
f
X
(x) dx =
_
d
c
1
b a
dx =
d c
b a
Exponential: (one real parameter > 0 )
f(x) =
_
e
x
, if x 0
0, if x < 0
Check that its total integral is 1. Note that the range is [0, ).
Normal: (two real parameters , > 0 )
f(x) =
1

2
exp
_

1
2
_
x

_
2
_
The range of a normal RV is the entire real line. It is anything but obvious
that the integral of this function is 1. Try to show it.
Cauchy:
f(x) =
1
(1 +x
2
)
3
Example: Suppose X is a random variable with an exponential distribution
with parameter = 2. Find P(X 2) and P(X 1|X 2).
Example: Suppose X has the Cauchy distribution. Find the number c with
the property that P(X c) = 1/4.
Example: Suppose X has the density
f(x) =
_
c x(2 x) if 0 x 2
0 otherwise
where c is a constant. Find the constant c and then compute P(1/2 X).
5.3 Expected value
A rigorous treatment of the expected value of a continuous random variable
requires the theory of abstract Lebesgue integration, so our discussion will
not be rigorous.
For a discrete RV X, the expected value is
E[X] =

x
xf
X
(x)
We will use this denition to derive the expected value for a continuous RV.
The idea is to write our continuous RV as the limit of a sequence of discrete
RVs.
Let X be a continuous RV. We will assume that it is bounded. So there is
a constant M such that the range of X lies in [M, M], i.e., M X M.
Fix a positive integer n and divide the range into subintervals of width 1/n.
In each of these subintervals we round the value of X to the left endpoint
of the interval and call the resulting RV X
n
. So X
n
is dened by
X
n
() =
k
n
, where k is the integer with
k
n
X() <
k + 1
n
Note that for all outcomes , |X() X
n
()| 1/n. So X
n
converges to X
pointwise on the sample space . In fact it converges uniformly on . The
expected value of X should be the limit of E[X
n
] as n .
The random variable X
n
is discrete. Its values are k/n with k running
from Mn to Mn 1 (or possibly a smaller set). So
E[X
n
] =
Mn1

k=Mn
k
n
f
Xn
(
k
n
)
4
Now
f
Xn
(
k
n
) = P(X
n
=
k
n
) = P(
k
n
X() <
k + 1
n
) =
_ k+1
n
k
n
f
X
(x) dx
So
E[X
n
] =
Mn1

k=Mn
k
n
_ k+1
n
k
n
f
X
(x) dx
=
Mn1

k=Mn
_ k+1
n
k
n
k
n
f
X
(x) dx
When n is large, the integrals in the sum are over a very small interval. In
this interval, x is very close to k/n. In fact, they dier by at most 1/n. So
the limit as n of the above should be
Mn1

k=Mn
_ k+1
n
k
n
xf
X
(x) dx =
_
M
M
xf
X
(x) dx =
_

xf
X
(x) dx
The last equality comes from the fact that f
X
(x) is zero outside [M, M].
So we make the following denition
Denition 2. Let X be a continuous RV with density f
X
(x). The expected
value of X is
E[X] =
_

xf
X
(x) dx
provided
_

|x| f
X
(x) dx <
(If this last integral is innite we say the expected value of X is not dened.)
The variance of X is

2
= E[(X )
2
], = E[X]
provided the expected value is dened.
5
End of September 30 lecture
Just as with discrete RVs, if X is a continuous RV and g is a function
fromR to R, then we can dene a new RV by Y = g(X). How do we compute
the mean of Y ? One approach would be to work out the density of Y and
then use the denition of expected value. We have not yet seen how to nd
the density of Y , but for this question there is a shortcut just as there was
for discrete RV.
Theorem 1. Let X be a continuous RV, g a function from R to R. Let
Y = g(X). Then
E[Y ] = E[g(X)] =
_

g(x) f
X
(x) dx
Proof. Since we do not know how to nd the density of Y , we cannot prove
this yet. We just give a non-rigorous derivation. Let X
n
be the sequence of
discrete RVs that approximated X dened above. Then g(X
n
) are discrete
RVs. They approximate g(X). In fact, if the range of X is bounded and g is
continous, then g(X
n
) will converge uniformly to g(X). So E[g(X
n
)] should
converges to E[g(X)].
Now g(X
n
)] is a discrete RV, and by the law of the unconscious statistician
E[g(X
n
)] =

x
g(x) f
Xn
(x) (2)
Looking back at our previous derivation we see this is
E[g(X
n
)] =
Mn1

k=Mn
g(
k
n
)
_ k+1
n
k
n
f
X
(x) dx
=
Mn1

k=Mn
_ k+1
n
k
n
g(
k
n
) f
X
(x) dx
which converges to
_
g(x) f
X
(x) dx (3)
6
Example: Find the mean and variance of the uniform distribution on [a, b].
The mean is
=
_
b
a
xf(x) dx =
_
b
a
x
b a
dx =
1
2
b
2
a
2
b a
=
a + b
2
(4)
For the variance we have to rst compute
E[X
2
] =
_
b
a
x
2
f(x) dx (5)
We then subtract the square of the mean and nd
2
= (b a)
2
/12.
Example: Find the mean and variance of the normal distribution.
Example: Find the mean of the Cauchy distribution
The gamma function is dened by
(w) =
_

0
x
w1
e
x
dx (6)
The gamma distribution has range [0, ) and depends on two parameters
> 0, w > 0. The density is
f(x) =
_

w
(w)
x
w1
e
x
if x 0
0 otherwise
(7)
In one of the homework problems we compute its mean and variance. You
should nd that they are
=
w

,
2
=
w

2
(8)
Example: Let X be exponential with parameter . Let Y = X
2
. Find the
mean and variance of Y .
5.4 Cumulative distribution function
In this section X is a random variable that can be either discrete or contin-
uous.
Denition 3. The cumulative distribution function (cdf ) of the random vari-
able X is the function
F
X
(x) = P(X x)
7
Why introduce this function? It will be a powerful tool when we look at
functions of random variables and compute their density.
Example: Let X be uniform on [1, 1]. Compute the cdf.
GRAPH !!!!!!!!!!!!!!!!!!!!
Example: Let X be a discrete RV whose pmf is given in the table.
x 2 3 4 5 6
f
X
(x) 1/8 1/8 3/8 2/8 1/8
GRAPH !!!!!!!!!!!!!!!!!!!!
Example: Compute cdf of exponential distribution.
Theorem 2. For any random variable the cdf satises
1. F(x) is non-decreasing, 0 F(x) 1.
2. lim
x
F(x) = 0, lim
x
F(x) = 1.
3. F(x) is continuous from the right.
4. For a continuous random variable the cdf is continuous.
5. For a discrete random variable the cdf is piecewise constant. The points
where it jumps is the range of X. If x is a point where it has a jump,
then the height of the jump is P(X = x).
Proof. 1 is obvious ....
To prove 2, let x
n
. Assume that x
n
is increasing. Let E
n
= {X
x
n
}. Then E
n
is in increasing sequence of events. By the continuity of the
probability measure,
P(

n=1
E
n
) = lim
n
P(E
n
)
Since x
n
, every outcome is in E
n
for large enough n. So

n=1
E
n
= .
So
lim
n
F(x
n
) = lim
n
P(E
n
) = 1 (9)
8
The proof that the limit as x is 0 is similar.
GAP
Now consider a continuous random variable X with density f. Then
F(x) = P(X x) =
_
x

f(t) dt
So given the density we can compute the cdf by doing the above integral.
Dierentiating the above we get
F

(x) = f(x)
So given the cdf we can compute the density by dierentiating.
Theorem 3. Let F(x) be a function from R to [0, 1] such that
1. F(x) is non-decreasing.
2. lim
x
F(x) = 0, lim
x
F(x) = 1.
3. F(x) is continuous from the right.
Then F(x) is the cdf of some random variable, i.e., there is a probability
space (, F, P) and a random variable X on it such that F(x) = P(X x).
f
The proof of this theorem is way beyond the scope of this course.
5.5 Function of a random variable
Let X be a continuous random variable and g : R R. Then Y = g(X) is
a new random variable. We want to nd its density. This is not as easy as
in the discrete case. In particular f
Y
(y) is not

x:g(x)=y
f
X
(x).
Key idea: Compute the cdf of Y and then dierentiate it to get the pdf of
Y .
Example: Let X be uniform on [0, 1]. Let Y = X
2
. Find the pdf of Y .
!!!!!!! GAP
Example: Let X be uniform on [1, 1]. Let Y = X
2
. Find the pdf of Y .
9
!!!!!!! GAP Example: Let X be uniform on [0, 1]. Let > 0. Y =

ln(X). Show Y has an exponential distribution.

!!!!!!! GAP Example: The standard normal distribution is the
normal distribution with = 0 and = 1. Let X have a normal distribution
with parameters and . Show that Z = (X)/ has the standard normal
distribution.
!!!!!!! GAP
Proposition 2. (How to write a general random number generator) Let X be
a continuous random variable with values in [a, b]. Suppose that the cdf F(x)
is strictly increasing on [a, b]. Let U be uniform on [0, 1]. Let Y = F
1
(U).
Then X and Y are identically distributed.
Proof.
P(Y y) = P(F
1
(U) y) = P(U F(y)) = F(y) (10)
Application:My computer has a routine to generate random numbers that
are uniformly distributed on [0, 1]. We want to write a routine to generate
numbers that have an exponential distribution with parameter .
How do you simulate normal RVs? Not so easy since the cdf cannot be
explicitly computed. More on this later.
5.6 More on expected value
Recall that for a discrete random variable that only takes on values in
0, 1, 2, , we showed in a homework problem that
E[X] =

k=0
P(X > k) (11)
There is a similar result for non-negative continuous random variables.
Theorem 4. Let X be a non-negative continuous random variable with cdf
F(x). Then
E[X] =
_

0
[1 F(x)] dx (12)
provided the integral converges.
10
Proof. We use integration by parts on the integral. Let u(x) = 1 F(x) and
dv = dx. So du = fdx and v = x. So
_

0
[1 F(x)] dx = x(1 F(x))|

x=0
+
_

0
xf(x) dx = E[X] (13)
Note that the boundary term at is zero since F(x) 1 as x .
We can use the above to prove the law of the unconscious statistician for
a special case. We assume that X 0 and that the function g is from [0, )
into [0, ) and it strictly increasing. Note that this implies that g has an
inverse. Then
E[Y ] =
_

0
[1 F
Y
(x)] dx =
_

0
[1 P(Y x)] dx (14)
=
_

0
[1 P(g(X) x)] dx =
_

0
[1 P(X g
1
(x))] dx (15)
=
_

0
[1 F
X
(g
1
(x))] dx (16)
Now we do a change of variables. Let s = g
1
(x). So x = g(s) and dx =
g

(s)ds. So above becomes

_

0
[1 F
X
(s)] g

(s) ds (17)
Now integrate this by parts to get
[1 F
X
(s)] g(s)|

s=0
+
_

0
g(s) f(s) ds (18)
which proves the theorem in this special case.
11