Proba Lectures

Uniform distribution. Probability density functions. Random variables.
Independence
Uniform distribution.
Probability density functions.
Random variables. Independence
Dr.Stephen Edward Mwaijande
FTC (MUST), B.Sc.Ed (Hons)(UDSM),M.Sc

(NM-AIST),Ph.D(UDSM)
Department of Mathematics and Statistics,

The University of Dodoma
1/24
Uniform distribution. Probability density functions. Random variables. Independence
After developing a background in probabilistic models with

discrete outcomes we can now progress further and do
exercises where uncountably many outcomes are explicitly
involved.
Here, the events are associated with subsets of a continuous
space (a real line R, an interval (a, b), a plane R2 , a square,
etc.). The simplest case is where the outcome space Ω is
represented by a ’nice’ bounded set and the probability
distribution corresponds to a unit mass uniformly spread over
it. Then the event (i.e a subset) A ⊂ Ω acquires the
probability
ν(A)
P (A) = . (1)
ν(Ω)
where ν(A) is the standard Euclidean volume (or area or 2/24
Z
P (A) = f (x)dx. (2)
A
Here f is a given PDF f ≥ 0 with
Z
f (x)dx = 1. (3)
Ω
3/24
Uniform Distribution
A continuous random variable X is said to have a Uniform

distribution over the interval [a, b] , shown as
X ∼ U nif orm(a, b), if its PDF is given by

1

 b−a , if a < x < b,
fX (x) = (4)
0, if x < a, or x ≥ b.

4/24
It can be noted that the corresponding CDF is given below:




 0, if y ≤ a ≤ 1,



FX (y) = y−a
b−a
, if a < y < b, (5)




y, if y ≥ b.

its mean and variance are given respectively by

a+b
E(X) = , (6)
2
(b − a)2
V ar(X) = . (7)
12
5/24
Normal (Gaussian) Distribution

The normal distribution is by far the most important
probability distribution. One of the main reasons for that is
the Central Limit Theorem (CLT) that we will discuss later in
the book. To give you an idea, the CLT states that if you add
a large number of random variables, the distribution of the
sum will be approximately normal under certain conditions.
The importance of this result comes from the fact that many
random variables in real life can be expressed as the sum of a
large number of randomvariables and, by the CLT, we can
argue that distribution of the sum should be normal. The CLT
is one of the most important results in probability and we will
6/24
We first define the standard normal random variable. The

PDF of an N (µ, σ 2 ) RV X is

1 1 2
p exp − 2 (x − µ) , x ∈ R; (8)
(2πσ) 2σ
with the mean and variance E(X) = µ, V ar(X) = σ 2
and the MGF and CHF
1 2 2 1 2 2
EeθX = eθµ+ 2 θ σ , EeitX = eitµ− 2 t σ , θ, t ∈ R If
X ∼ N (µ, σ 2 ), then x−µ
σ
∼ N (0, 1) and
∀b, c ∈ :cX + b ∼ N (cµ + b, c2 δ 2 ).
7/24
also Gausian distribution has a CDF:

Z ∞
1 1 2 y−µ
FX (y) = p exp − 2 (x − µ) dx = Φ( ), y ∈ R;
σ (2π) ∞ 2σ σ
(9)
8/24
We will then see that we can obtain other normal random

variables by scaling and shifting a standard normal random
variable.
A continuous random variable Z is said to be a standard
normal (standard Gaussian) random variable, shown as
Z ∼ N (0, 1) , if its PDF is given by

1 1 2
FZ (z) = p exp − z , z ∈ R; (10)
(2π) 2
The √ 1 is there to make sure that the area under the PDF
(2π)
is equal to one. 9/24
Note that, in all calculations involving PDFs, the sets C with

Z
dx = 0. (11)
C
(sets of measure 0) can be disregarded. Therefore,

probabilities P (a ≤ x ≤ b) and P (a < x < b) coincide. (This
is, of course, not true for discrete RVs.)
The median m(X) of RV X gives the value that ’divides’ the
range of X into two pieces of equal mass. In terms of the CDF
and PDF:
m(X) = max {y : FX (y) ≥ 0} . (12)
9/24
10/24
Exponential Distribution
The exponential distribution is one of the widely used

continuous distributions. It is often used to model the time
elapsed between events. We will now mathematically define
the exponential distribution, and derive its mean and expected
value. Then we will develop the intuition for the distribution
and discuss several interesting properties that it has.
11/24
A continuous random variable X is said to have an

exponential distribution with parameter λ > 0 , shown as
X ∼ Exponential(λ), if its PDF is given by

−λx
λe , x > 0;


fX (x) = (13)


 0, otherwise
and its CDF is given by


0, if x ≤ 0,


FX (x) = (14)
−λx
1 − e , x > 0;


12/24
We can find its expected value as follows, using integration by

parts:
Z ∞ Z ∞
1
E(X) = xf (x)dx = xλe−λx dx =
0 0 λ
and its variance is
1
V ar(X) =
λ2
13/24
Gamma Distribution
The gamma distribution is another widely used distribution.

Its importance is largely due to its relation to exponential and
normal distributions. Here, we will provide an introduction to
the gamma distribution. Interested person may discuss more
properties of the gamma random variables. Before introducing
the gamma random variable, we need to introduce the gamma
function.
14/24
Gamma function: The gamma function shown by Γ(x), is an

extension of the factorial function to real (and complex)
numbers. Specifically, if n = {1, 2, 3, ..., }, then
Γ(n) = (n − 1)! (15)
More generally, for any positive real number α , Γ(α) is

defined as
Z ∞
Γ(α) = xα−1 e−λx dx, for α > 0. (16) 15/24
for a Gamma RV, has a PDF

λα α−1 −λx
fX (x) = x e ; (17)
Γ(α)
and a CDF y
λα
Z
FX (x) = xα−1 e−λx ; (18)
Γ(α) 0
15/24
For integrals over (0, ∞) with an integrand consisting of a

power term and an exponential term, one should try
transforming into a Gamma function, defined as
Z ∞
Γ(p) = xα−1 e−λx dx. (19)
0
Given p, Γ(p) can be found from tables or by means of a
computer program. Some useful properties of the Gamma
function are:
Γ(p + 1) = pΓ(p),
Γ(n + 1) = n!, (20)
1 √
Γ( ) = π.
2
for non-negative integer n. 16/24
CHECK FOR GAMMA DISTRIBUTION:

α
E(X) =
λ
and its variance is
α
V ar(X) =
λ2
17/24
Cauchy Distribution
For a Cauchy RV has a PDF

τ
fX (y) = , y ∈ R. (21)
τ2 + (y − α)2
and a CDF

1 −1 y − α π
FX (y) = tan ( )+ , y ∈ R. (22)
π τ 2
To be brief we write X ∼ Ca(α, τ ).
18/24
In general, we say that X has a PDF f (and write X ∼ f ) if

∀y ∈ R
Z y
Gam(p) = tp−1 e−t dt, p > 0. (23)
−∞
Then, ∀a, b ∈ R with a < b:

Z b
P (a < X < b) = f (x)dx. (24)
a
and in general, ∀ measurable set A ⊂ R:
Z
P (X ∈ A) = f (x)dx. (25)
A
19/24
Expectation
The expected value or expectation of a continuous r.v. X with
p.d.f. f(x) is denoted by E(X) and is defined as
Z ∞
E(X) = xf (x)dx. (26)
−∞
provided that the integral is absolutely convergent (i.e.

R∞
−∞
|x|f (x)dx is finite). As in the discrete case, E(X) is
often termed the expected value or mean of X.
Z ∞
E(g(X)) = g(x)f (x)dx. (27)
−∞
20/24
Variance
An immediate application is to the variance of X, denoted by

V ar(X) and defined as
V ar(X) = E([X − E(X)]2 ). (28)
21/24
Writing µ = E(X), we have

Z ∞
V ar(X) = (x − µ)2 f (x)dx,
Z−∞
∞
= (x2 − 2µx + µ2 )f (x)dx,
Z−∞
∞ Z ∞ Z ∞
2 2
= x f (x)dx − 2µ xf (x)dx + µ f (x)dx,
−∞ −∞ −∞
Z ∞ Z ∞
2 2
= x f (x)dx − 2µ.µ + µ .1 = x2 f (x)dx − µ2 ,
−∞ −∞
2 2
= E(X ) − [E(X)] ,
(29)
just as in the discrete case.

22/24
Normal distributions. Convergence of random

variables and distributions.The Central Limit
Theorem
I We have already learned a number of properties of a
normal distribution. Its importance was realised at an
early stage by, among others, Laplace, Poisson and of
course Gauss.
I Understanding the special nature of normal distributions
required facts and methods from other fields of
mathematics, including analysis and mathematical physics
(notably, complex analysis and partial differential
23/24
Recall the properties of Gaussian distributions which we have

established so far:
I The PDF of an N (µ, σ 2 ) RV X is

1 1 2
p exp − 2 (x − µ) , x ∈ R; (30)
(2πσ) 2δ
with the mean and variance E(X) = µ, V ar(X) = σ 2
and the MGF and CHF
1 2 2 1 2 2
EeθX = eθµ+ 2 θ σ , EeitX = eitµ− 2 t σ , θ, t ∈ R If
X ∼ N (µ, σ 2 ), then x−µ
σ
∼ N (0, 1) and
∀b, c ∈ :cX + b ∼ N (cµ + b, c2 δ 2 ).
I Two jointly normal RVs X and Y are independent iff
Cov(X, Y ) = Corr(X, Y ) = 0.
24/24
The sum X +Y of two jointly normal RVs X ∼ N (µ1 , σ12 ),

Y ∼ N (µ2 , σ22 ), with CorrX − Y = r is normal, with mean
µ1 + µ2 and the variance σ12 + 2σ1 σ2 + σ22 . See equation
(2.41). In particular, if X, Y are independent,
X + Y ∼ N (µ1 + µ2 , σ12 + σ22 ). In general, for independent
RVs X1 , X2 , ..., where Xi ∼ N (µi , σi2 ), the linear combination
Σi ci Xi ∼ N (Σi ci µi , Σi c2i σi2 ).
To sum up for any random variable X we have discussed so
far, we write X ∼ U (a, b), X ∼ N (µ, σ 2 ), X ∼ Exp(λ),
X ∼ Gam(α, λ),and X ∼ Ca(α, τ ).
25/24
Example-Exercise
Let X ∼ Ca(2, 4). Find the probability that
a. X is less than 3,
b. X is greater than 4,
c. X is between 1 and 3.5
Solution
A CDF for Cauchy is given by

1 −1 y − α π
FX (y) = tan ( )+ , y ∈ R. (31)
π τ 2
26/24
here α = 2, τ = 4, y = 3
we have
P (X < 3) = F (3),

1 −1 3 − 2 π
= tan ( )+ ,
π 4 2
1 h −1 πi
= tan (0.25) + = 0.578
π 2
Verify for part b) and c)
ANS
b)0.3524 c) 0.156
27/24

Proba Lectures

Uploaded by

Copyright:

Available Formats

Proba Lectures

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Proba Lectures

Uploaded by

Copyright:

Available Formats

Uniform distribution. Probability density functions. Random variables.

Dr.Stephen Edward Mwaijande

FTC (MUST), B.Sc.Ed (Hons)(UDSM),M.Sc

Department of Mathematics and Statistics,

After developing a background in probabilistic models with

A continuous random variable X is said to have a Uniform

It can be noted that the corresponding CDF is given below:

its mean and variance are given respectively by

Normal (Gaussian) Distribution

We first define the standard normal random variable. The

also Gausian distribution has a CDF:

We will then see that we can obtain other normal random

Note that, in all calculations involving PDFs, the sets C with

(sets of measure 0) can be disregarded. Therefore,

The exponential distribution is one of the widely used

A continuous random variable X is said to have an

and its CDF is given by

We can find its expected value as follows, using integration by

The gamma distribution is another widely used distribution.

Gamma function: The gamma function shown by Γ(x), is an

Γ(n) = (n − 1)! (15)

More generally, for any positive real number α , Γ(α) is

for a Gamma RV, has a PDF

For integrals over (0, ∞) with an integrand consisting of a

CHECK FOR GAMMA DISTRIBUTION:

For a Cauchy RV has a PDF

To be brief we write X ∼ Ca(α, τ ).

In general, we say that X has a PDF f (and write X ∼ f ) if

Then, ∀a, b ∈ R with a < b:

provided that the integral is absolutely convergent (i.e.

An immediate application is to the variance of X, denoted by

V ar(X) = E([X − E(X)]2 ). (28)

Writing µ = E(X), we have

just as in the discrete case.

Normal distributions. Convergence of random

Recall the properties of Gaussian distributions which we have

The sum X +Y of two jointly normal RVs X ∼ N (µ1 , σ12 ),

You might also like