1 Random variables and distributions

In this chapter we consider real valued functions, called random variables, de-

fined on the sample space.

X : S → RX

The set of possible values of X is denoted by the set RX . Random variables

constitute the quantities of interest in most experiments.

If RX is countable, we say X is discrete. If on the other hand, RX is an

interval or a union of intervals, we say X is continuous.

1
Example Toss a balanced coin 3 times. Our interest is the number of heads

obtained and not necessarily on the order of the outcomes. Each point in

the sample space is equally likely.

Sample space X =Number of heads Number of tails Probability

1
(HHH) 3 0 8

1
(HHT ) 2 1 8

1
(HT H) 2 1 8

1
(HT T ) 1 2 8

1
(T HH) 2 1 8

1
(T HT ) 1 2 8

1
(T T H) 1 2 8

1
(T T T ) 0 3 8
Here, RX = {0, 1, 2, 3}and hence X is discrete. The pmf is given by the table

x 0 1 2 3

1 3 3 1
f(x) 8 8 8 8

Example Let X be the number of e-mails received during the time interval

(0, t) and let Y be the time between e-mails. The random variable X

is discrete and takes values 0, 1, 2, .... On the other hand, the random

variable Y is continuous and takes values in an interval of length at most

t.

2
Defiinitions

• The probability mass function or probability distribution of the discrete

random variable X is the function f (x) = P (X = x) for all possible values

x. It satisfies the following properties

i) f (x) ≥ 0

P
ii) f (x) = 1

The cumulative distribution function F (x) of a discrete random variable X is

X
F (x) = f (t) , −∞ < x < ∞
t≤x

Associated with a random variable is the probability mass function which

attaches to each value. We see for example that the value 2 is taken 3 times

and the probability for each instance is 81 . Hence the probability of the random

1


variable taken the value 3 is 3 8

3
Example1 Coin tossing

i) Probability mass function plot for coin tossing

ii) Probability histogram

iii) Discrete cumulative distribution function




0 x<0









1
0≤x<1


8






F (x) = 4 1 ≤ x < 2
8





7 2 ≤ x < 3



 8





1 3 ≤ x



4
Example Find the constant c which makes the following a proper density

f (x) = cx, x = 1, 2, 3

P P3 1
We must have 1 = f (x) = c x=1 x = 6c. Hence c = 6

5
1.1 Continuous probability distributions

If a sample space contains an infinite number of possibilities equal to the number

of points on a line segment, it is called a continuous sample space. For such

a space, probabilities can no longer be assigned to individual values. Instead,

probabilities must be assigned to sub-intervals

P (a < X < b)

Definition The function f (x) is a probability density function for the contin-

uous random variable X defined over the set of real numbers if

i) f (x) ≥ 0, all real x

R∞
ii) −∞
f (x) dx = 1
Rb
iii) P (a < X < b) = a
f (t) dt

iv) the cumulative distribution function

Z x
F (x) = P (X ≤ x) = f (t) dt
−∞

6
Example1 f (x) = 1, 0 < x < 1

Rt
Example2 f (x) = e−x , 0 < x < ∞. F (t) = 0
e−x dx = 1 − e−t , t > 0

Example Find the constant c which makes f (x) = cx, 0 < x < 1 a proper

density.

R1 2
xdx = c x2 |10 = 2c . Hence, c = 2.
R
We must have 1 = f (x) dx = c 0

We may calculate

  Z 3
1 3 4 1
P <X< = 2xdx =
4 4 1
4
2

Example (Exercise 2.7) Let






x 0<x<1




f (x) = 2 − x 1 ≤ x < 2







0
 elsewhere

R 1.2 R1 R 1.2
a) P (X < 1.2) = 0
f (x) dx = 0
xdx + 1
(2 − x) dx = 0.68
R1 3
b) P (0.5 < X < 1) = 0.5
xdx = 8

7
1.2 Joint probability distributions

We have situations where more than characteristic is of interest. Suppose we

toss a pair of dice once. The discrete sample space consists of the pairs

{(x, y) : x = 1, ..., 6; y = 1, ..., 6}

where X, Y are the random variables representing the results of the first and

second die respectively. For two electric components in a series connection, we

may be interested in the lifetimes of each.

8
Definition The function f (x, y) is a joint probability distribution or probabil-

ity mass function oϑf the discrete random variables X, Y if

i) f (x, y) ≥ 0, all (x, y)

P P
ii) x y f (x, y) = 1

iii) P (X = x, Y = y) = f (x, y)

PP
iv) P ((X, Y ) A) = (x,y)A f (x, y)

9
 1
Example One toss of a pair of dice: P (X = x, Y = y) = 36

1 2 3 4 5 6

1 1 1 1 1 1
1 36 36 36 36 36 36

1 1 1 1 1 1
2 36 36 36 36 36 36

1 1 1 1 1 1
X 3 36 36 36 36 36 36

1 1 1 1 1 1
4 36 36 36 36 36 36

1 1 1 1 1 1
5 36 36 36 36 36 36

1 1 1 1 1 1
6 36 36 36 36 36 36

10
Definition The function f (x, y) is a joint probability distribution of the con-

tinuous random variables X, Y if

i) f (x, y) ≥ 0, all (x, y)

RR
ii) f (x, y) dxdy = 1
RR
iii) P ((X, Y ) A) = A
f (x, y) dxdy

11
Example1 Find the constant c which makes f (x, y) = cxy 2 , 0 < x < 1, 0 <

y < 1 a proper density

R1R1 2 3
xy 2 dxdy = c x2 |10 y3 |10 = c
RR
We must have 1 = f (x, y) dxdy = c 0 0 6

Hence, c = 6

Example1 (continued) We calculate

  Z 12 Z 21
1 1 1
P 0 < X < ,0 < Y < =6 xy 2 dxdy =
2 2 0 0 4

Example1 (continued) Calculate

P 0 < X < 12 , 0 < Y < 1

 

1 1 2
P 0 < X < |0 < Y < =
P 0 < Y < 12


2 2
1
4
= = 0.25
1

12
 It is possible to retrieve the individual distribution of a random variable from

the joint distribution. This is done through the notion of marginal distributions.

Definition The marginal distributions of X alone and of Y alone are

X X
g (x) = f (x, y) , h (y) = f (x, y)
y x

for the discrete case and

Z Z
g (x) = f (x, y) dy, h (y) = f (x, y) dx

for the continuous case.

13
Example1 Suppose that the joint density of X, Y is given as f (x, y) = 8xy, 0 <

x < y < 1. Find the marginal densities of X and Y. We calculate

Z 1
8xydy = 4x 1 − x2 , 0 < x < 1


gX (x) =
x

Z y
gY (y) = 8xydx = 4y 3 , 0 < y < 1
0

Example2 Suppose we draw two balls from an urn containing 5 white, 2 blue

and 4 red balls. Let X and Y be the number of white and red balls drawn

respectively. The joint density of X, Y is given by

2
5

4



x y 2−x−y
f (x, y) = 11

,x + y ≤ 2
2

y gX (x)

0 1 2

1 8 6 15
x 0 55 55 55 55

10 20 30
1 55 55 0 55

10 10
2 55 0 0 55

21 28 6
gY (y) 55 55 55 1
The marginal densities are expressed in the “margins”.

14
1.3 Conditional distributions

Definition The conditional distribution of the random variable Y given X = x

is
f (x, y)
f (y|x) = , g (x) > 0
g (x)

Similarly the conditional distribution of X given Y = y

f (x, y)
f (x|y) = , h (y) > 0
h (y)

Example Suppose that the joint density of X, Y is given as f (x, y) = 8xy, 0 <

x < y < 1.

8xy
f (y|x) = ,0 < x < y < 1
4x (1 − x2 )
2y
= ,0 < x < y < 1
1 − x2

R1
We may easily verify that f (y|x) is a proper density since x
f (y|x) dy = 1

Example2 (continued) We can calculate



 1
 y=0
 15





f (y|x) = 8
y=1

 15




6


15 y=2

15
 The variables are said to be independent if

f (x, y) = g (x) h (y) , all (x, y)

For random variables X1 , ..., Xn with joint density f (x1 , ..., xn ) and marginals

f1 (x1 ) , ..., fn (xn ) we say they are mutually independent if and only if

f (x1 , ..., xn ) = f1 (x1 ) ...fn (xn )

for all tuples (x1 , ..., xn ) within their range.

Examples In the urn example, X and Y are not independent. Similarly for

the continuous density example.

16
1.4 Properties of random variables

Definition The mean or expected value of a function L of a random variable

X , say L (X) is


 P
 L (x) f (x) X discrete


µ = E [L (X)] =

 R
 L (x) f (x) dx

X continuous

Definition The mean or expected value of a function L of random variables

X, Y , say L (X, Y ) is


 P
 L (x, y) f (x, y) X Y, discrete


µ = E [L (X, Y )] =

 R
 L (x, y) f (x, y) dxdy

X Y, continuous

17
 15 30 10 50




Example (Urn example) E (X) = 0 55 +1 55 +2 55 = 55 = 0.909.

21 28 6 40




E (Y ) = 0 55 +1 55 +2 55 = 55 = 0.727.

15 30 10 70





E X2 = 0 55 + 12 55 + 22 55 = 55 = 1.272.

15 28 6 52





E Y2 =0 55 + 12 55 + 22 55 = 55 = 0.945.

20 20


E (XY ) = 1 (1) 55 = 55

18
Definition The covariance of two random variables X, Y is σXY = E [(X − µX ) (Y − µY )] =

E (XY ) − µX µY .

2 2
Definition The variance of a random variable X is σX = E (X − µX ) =
2
E (X) − µ2X

σXY
Definition The correlation between X, Y is ρ = σX σY

The variance measures the amount of variation in a random variable taking

into account the weighting given to the values of the random variable. The

correlation measures the “interaction” between the variables. A positive corre-

lation indicates that an increase in X results in an increase in Y . A negative

correlation indicates that an increase inX results in a decrease in Y.

19
 70 50 2 54 20 50 40
2




Example (urn example) σX = 55 − 55 = 121 = 0.446,σXY = 55 − 55 55 =

36
− 121 = −0.2975 ρ = − − .690

Example Suppose that the joint density of X, Y is given as f (x, y) = 8xy, 0 <

x < y < 1. Then

8
µX =
15
4
µY =
5
2 11
σX =
225
2
σY2 =
75
4
σXY =
225

ρ = 0.49237

20
 In this section, we assemble several results on the calculation of expectation

for linear combinations of random variables

Theorem Let X, Y be two random variables and let a, b, c be three arbitrary

constants. Also, let h1 (x) , h2 (x) , h3 (x, y) , h4 (x, y) be real valued func-

tions. Then,

E (aX + b) = aE (X) + b

E [h1 (X) ± h2 (X)] = E [h1 (X)] ± E [h2 (X)]

E [h3 (X, Y ) ± h4 (X, Y )] = E [h3 (X, Y )] ± E [h4 (X, Y )]

2
σaX+bY +c = a2 σX
2
+ b2 σY2 + 2abσXY

21
Theorem Let X, Y be two independent random variables and let a, b be two

arbitrary constants. Then

E (XY ) = E (X) E (Y )

σXY = 0

2
σaX+bY = a2 σX
2
+ b2 σY2

22
 This theorem generalizes to several independent random variables X1 , ..., Xn

Theorem Let X1 , ..., Xn be random variables and let a1 , ..., an be arbitrary

constants. Then

P P
i) E [ ai Xi ] = ai E [Xi ]

ii)If in addition, X1 , ..., Xn are independent random variables

X
2
σP a
= a2i σX
2
i
i Xi

Example Flip a coin with probability of heads p, n times and let Xi = 1 if

we observe heads on the ith toss and Xi = 0, if we observe tails. Assume

the results of the tosses are independent and performed under indentical

conditions. Then, X1 , ..., Xn are independent and identically distributed

P
random variables. The sum, Xi represents the number of heads in n

tosses. It follows from the theorem

1 P  P1 1
i) E n Xi = n E [Xi ] = n (np) = p

1 2 σ2 p(1−p)


ii) σ 21 P 2
P
= n σX i
= n = n
Xi
n

The example demonstrates that the average of a set of independent random

variables preserves the mean and importantly reduces the variance by a factor

equal to the size of the sample. .

23