Class Notes 3

DEFINITION:
The set of possible values that a random

variable X can take is called the range of X.
EQUIVALENCES
Unstructured Random
Experiment Variable
E X
Sample space range of X
Outcome of E One possible value x for X
Event Subset of range of X
Event A x ∈ subset of range of X
e.g., x = 3 or 2 ≤ x ≤ 4
Pr(A) Pr(X = 3), Pr(2 ≤ X ≤ 4)
Probability Distributions
or ‘How to describe the behaviour of a rv’
Suppose that the only values a random

variable X can take are x1, x2, . . . , xn. That
is, the range of X is the set of n values
x1, x2, . . . xn.
Since we can list all possible values, this
random variable X must be discrete.
Then the behaviour of X is completely

described by giving the probabilities of all
relevant events:
Event Probability
X = x1 Pr(X = x1)
X = x2 Pr(X = x2)
... ...
X = xn Pr(X = xn)
In other words, we specify the function

Pr(X = x) for all values x in the range of X.
EXAMPLE:
Let the probability of a head on any toss of a
particular coin be p. From independent
successive tosses of the coin, we record the
number X of tails before the first head
appears.
Range of X : {0, 1, 2, . . .}
Pr(X = 0) = p
Pr(X = 1) = (1 − p)p
...
Pr(X = x) = (1 − p)xp
...
The probability function for the random

variable X gives a convenient summary of its
behaviour; the pf pX (x) is given by:
pX (x) = (1 − p)xp, x = 0, 1, 2, . . . .
X is said to have a Geometric Distribution.

DEFINITION:
The cumulative distribution function of a
rv X is the function FX (x) of x given by
FX (x) = Pr(X ≤ x),
for all values x in the range of X.
Abbreviation: cdf
Terminology: The cdf is sometimes given the
alternative name of distribution function.
Notation: F (x) or FX (x). We use the FX (x)
form when we need to make the identity of
the rv clear.
Relationship with pf: For a discrete rv X,
X
FX (x) = pX (y)
y≤x
Example: If a rv has range {0, 1, 2, . . .},
FX (3) = pX (0) + pX (1) + pX (2) + pX (3)
and
px(2) = FX (2) − FX (1).
Properties of cdfs:
All cdfs
• are monotonic non-decreasing,

• satisfy FX (−∞) = 0 ,
• satisfy FX (∞) = 1 .
Any function satisfying these conditions can

be a cdf.
A function not satisfying these conditions

cannot be a cdf.
For a discrete rv the cdf is always a step

function.
Reminder: Properties of cdfs: Any function

satisfying the following conditions can be a
cdf:
• It is monotonic non-decreasing,
Types of random variable
Most rvs are either discrete or continuous,

but
• one can devise some complicated
counter-examples, and
• there are practical examples of rvs
which are partly discrete and partly
continuous.
EXAMPLE: Cars pass a roadside point, the

gaps (in time) between successive cars being
exponentially distributed.
Someone arrives at the roadside and crosses
as soon as the gap to the next car exceeds 10
seconds. The rv T is the delay before the
person starts to cross the road.
The delay T may be zero or positive. The

chance that T = 0 is positive; the cdf has a
step at t = 0. But for t > 0 the cdf will be
continuous.
Measures of variability
Two rvs can have equal means but very

different patterns of variability. Here is a
sketch of the probability functions p1(x) and
p2(x) of two rvs X1 and X2.
p1(x)
x
6
mean
p2(x)
x
6
mean
To distinguish between these, we need a

measure of spread or dispersion.
Summary and formula
The most important features of a distribution

are its location and dispersion, measured by
expectation and variance respectively.
X
Expectation: E(X) = x Pr(X = x) = µ .
x
Variance:
(x − µ)2 Pr(X = x)
X
Var(X) =
x
(x2 − 2µx + µ2) Pr(X = x)

X
=
x
x2 Pr(X = x) − 2µ · µ + µ2 · 1
X
=
x
= E(X 2) − {E(X)}2
X
Reminder: The notation means the sum
x
over all values x in the range of X.
Notation: We often denote E(X) by µ, and

Var(X) by σ 2.
Notes
1. The concepts of expectation and variance

apply equally to discrete and continuous
random variables. The formulae given
here relate to discrete rvs; formulae need
(slight) adaptation for the continuous
case.
2. Units: the mean is in the same units as

X, the variance Var(X), defined as
Var(X) = E{X − E(X)}2

is in squared units.
A measure of dispersion in the same units

as X is the standard deviation (s.d.)
q
s.d.(X) = Var(X).
Consider the events {X ≤ a} and {a < X ≤ b}.
These events are mutually exclusive, and
{X ≤ a} ∪ {a < X ≤ b} = {X ≤ b} .
So the addition law of probability (axiom A3)
gives:
Pr(X ≤ b) = Pr(X ≤ a) + Pr(a < X ≤ b) ,

or Pr(a < X ≤ b) = Pr(X ≤ b) − Pr(X ≤ a)
= FX (b) − FX (a) .
So, given the cdf for any continuous random

variable X, we can calculate the probability
that X lies in any interval.
Note: The probability Pr(X = a) that a

continuous rv X is exactly a is 0. Because of
this, we often do not distinguish between
open, half-open and closed intervals for
continous rvs.
Probability density function
If X is continuous, then Pr(X = x) = 0.

But what is the probability that ‘X is close to
some particular value x?’.
Consider Pr(x < X ≤ x + h), for small h.
d FX (x) F (x + h) − FX (x)
Recall: ' X .
dx h
So Pr(x < X ≤ x + h) = FX (x + h) − FX (x)
d FX (x)
' h .
dx
DEFINITION: The derivative (w.r.t. x) of

the cdf of a continous rv X is called the
probability density function of X.
The probability density function is the limit of

Pr(x < X ≤ x + h)
as h → 0 .
h
Properties of probability density functions
Because the pdf of a rv X is the derivative of

the cdf of X, it follows that
• fX (x) ≥ 0, for all x,

Z ∞
• fX (x) dx = 1,
−∞
Z x
• FX (x) = fX (y)dy,
−∞
Z b
• Pr(a < X ≤ b) = fX (x)dx.
a
For the variance, recall the definition.
Var(X) = E[{X − E(X)}2]

Z ∞
Hence Var(X) = (x − µ)2 fX (x) dx
−∞
As in the discrete case, the best way to

caclulate a variance is by using the result:
Var(X) = E(X 2) − {E(X)}2 .

In practice, we therefore usually calculate
Z ∞
E(X 2) = x2 fX (x) dx
−∞
as a stepping stone on the way to obtaining
Var(X).
Uniform Distribution: cdf
For this distribution the cumulative

distribution function (cdf) is
Z x
FX (x) = fX (y) dy
−∞



 0, x < a,


= x−a , a ≤ x ≤ b ,

 b−a



1, x > b.
FX (x)
1

-
0
a b
The exponential distribution
A continuous random variable X is said to

have an exponential distribution if its range is
(0, ∞) and its pdf is proportional to e−λx, for
some positive λ.
That is,

 0, x < 0,
fX (x) =
 ke−λx , x ≥ 0 ,
for some constant k. To evaluate k, we use

the fact that all pdfs must integrate to 1.
Hence
Z ∞ Z ∞
fX (x) dx = ke−λx dx
−∞ 0
kh −λx
i∞
= −e
λ 0
k
=
λ
Since this must equal 1, k = λ.

The Normal Distribution
DEFINITION: A random variable X with

probability density function
2
1 − (x−µ)
fX (x) = √ e 2σ 2 ,
σ 2π
for all x, is said to have the Normal
distribution with parameters µ and σ 2.
It can be shown that E(X) = µ, Var(X) = σ 2.
We write: X ∼ N(µ, σ 2) .
Shape of the density function (pdf):

The pdf is symmetrical about x = µ.
It has a single mode at x = µ.
It has points of inflection at x = µ ± σ.
‘A bell-shaped curve,’ tails off rapidly.
Brief extract from a table of the SND
Z Φ(z)
0.0 0.5000
0.5 0.6915
1.0 0.8413
1.5 0.9332
2.0 0.9772
Tables in textbooks and elsewhere contain

values of Φ(z) for z = 0, 0.01, 0.02, and so
on, up to z = 4.0 or further.
But the range of Z is (−∞, ∞), so we need
values of Φ(z) for z < 0. To obtain these
values we use the fact that the pdf of N(0, 1)
is symmetrical about z = 0.
This means that
Φ(z) = 1 − Φ(−z).
This equation can be used to obtain Φ(z) for
negative values of z.
For example, Φ(−1.5) = 1 − 0.9332 = 0.0668.

Class Notes 3

Uploaded by

Copyright:

Available Formats

Class Notes 3

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Class Notes 3

Uploaded by

Copyright:

Available Formats

DEFINITION:

The set of possible values that a random

or ‘How to describe the behaviour of a rv’

Suppose that the only values a random

Then the behaviour of X is completely

In other words, we specify the function

The probability function for the random

X is said to have a Geometric Distribution.

• are monotonic non-decreasing,

Any function satisfying these conditions can

A function not satisfying these conditions

For a discrete rv the cdf is always a step

Reminder: Properties of cdfs: Any function

Most rvs are either discrete or continuous,

EXAMPLE: Cars pass a roadside point, the

The delay T may be zero or positive. The

Two rvs can have equal means but very

To distinguish between these, we need a

The most important features of a distribution

(x2 − 2µx + µ2) Pr(X = x)

Notation: We often denote E(X) by µ, and

1. The concepts of expectation and variance

2. Units: the mean is in the same units as

Var(X) = E{X − E(X)}2

A measure of dispersion in the same units

Pr(X ≤ b) = Pr(X ≤ a) + Pr(a < X ≤ b) ,

So, given the cdf for any continuous random

Note: The probability Pr(X = a) that a

If X is continuous, then Pr(X = x) = 0.

DEFINITION: The derivative (w.r.t. x) of

The probability density function is the limit of

Because the pdf of a rv X is the derivative of

• fX (x) ≥ 0, for all x,

Var(X) = E[{X − E(X)}2]

As in the discrete case, the best way to

Var(X) = E(X 2) − {E(X)}2 .

For this distribution the cumulative

A continuous random variable X is said to

for some constant k. To evaluate k, we use

Since this must equal 1, k = λ.

DEFINITION: A random variable X with

Shape of the density function (pdf):

Tables in textbooks and elsewhere contain

You might also like