Chapter 3:

Random Variables

By Tessema K Chapter 3: Multiple Random Variables 1

 Multiple Random Variables
Outline
 Introduction
 The Joint Cumulative Distribution Function
 The Joint Probability Density and Mass Functions
 Marginal Statistics
 Independence
 Conditional Distributions
 Correlation and Covariance
 Functions of Two Random Variables

By Mohammed M Chapter 3: Multiple Random Variables 2

 Introduction
•In many situations, we are interested in 2 or more
characteristics observed in experiments
•Often used to study the relationship among
characteristics and the prediction of one based on the
other(s)
•Three types of distributions:
– Joint: Distribution of outcomes across all combinations of
variables levels
– Marginal: Distribution of outcomes for a single variable
– Conditional: Distribution of outcomes for a single variable,
given the level(s) of the other variable(s)

By Mohammed M Chapter 3: Multiple Random Variables 3

The Joint Cumulative Distribution Function
 The joint cdf of two random variables X and Y denoted by
FXY(x, y) is a function defined by:
FXY ( x, y ) P[ X ( )  x and Y ( )  y ]
 FXY ( x, y ) P ( X  x, Y  y )
where x and y are arbitrary real numbers.

Properties of the Joint cdf, FXY(x, y):

i. 0 FXY ( x, y ) 1
ii. lim FXY ( x, y ) FXY (, ) 1
x 
y 

iii. lim FXY ( x, y ) FXY ( , ) 0

x  
y  
By Mohammed M Chapter 3: Multiple Random Variables 4
The Joint Probability Density Function

 The joint probability function (pdf) of two continuous random

variables X and Y is defined as:

 2 FXY ( x, y )
f XY ( x, y ) 
xy

 Thus, the joint cumulative distribution function (cdf) is given

by:
y x
FXY ( x, y )   f XY (u , v)dudv
- -

By Mohammed M Chapter 3: Multiple Random Variables 5

The Joint Probability Density Function Cont’d…..

Properties of the Joint pdf, fXY(x, y):

1. f XY ( x, y ) 0
 
2.  f
- -
XY ( x, y )dxdy 1
y2 x2
3. P ( x1  X  x2 , y1 Y  y2 )   f XY ( x, y )dxdy
y1 x1

By Mohammed M Chapter 3: Multiple Random Variables 6

The Joint Probability Mass Function
 The joint probability mass function (pmf) of two discrete
random variables X and Y is defined as:
PXY ( xi , y j ) P ( X  xi , Y  y j )
 The joint cdf can be written as:

FXY ( x, y )    PXY ( xi , y j )
xi x y j y

Properties of the Joint pmf, PXY (xi , yj ):

1. 0 PXY ( xi , y j ) 1
2. P
xi yj
XY ( xi , y j ) 1

By Mohammed M Chapter 3: Multiple Random Variables 7

Marginal Statistics
 In the case of two or more random variables, the statistics of
each individual variable are called marginal statistics.
i. Marginal cdf of X and Y
FX ( x) lim FXY ( x, y ) FXY ( x, )
y 

FY ( y ) lim FXY ( x, y ) FXY (, y )

x 

ii. Marginal pdf of X and Y


f X ( x)  f XY ( x, y )dy
-

f Y ( y )  f XY ( x, y )dx
-

By Mohammed M Chapter 3: Multiple Random Variables 8

Marginal Statistics Cont’d…..

iii. Marginal pmf of X and Y

P ( X  xi ) PX ( xi )  PXY ( xi , yi )
yj

P (Y  y j ) PY ( y j )  PXY ( xi , yi )
xi

By Mohammed M Chapter 3: Multiple Random Variables 9

Independence
 If two random variables X and Y are independent, then
i. from the joint cdf

FXY ( x, y ) FX ( x) FY ( y )

ii. from the joint pdf

f XY ( x, y )  f X ( x) f Y ( y )

iii.from the joint pmf

PXY ( xi , y j ) PX ( xi ) PY ( y j )

By Mohammed M Chapter 3: Multiple Random Variables 10

Conditional Distributions
i. Conditional Probability Density Functions
f XY ( x, y )
f X /Y ( x / y)  , fY ( y )  0
fY ( y )
f XY ( x, y )
fY / X ( y / x)  , f X ( x)  0
f X ( x)
i. Conditional Probability Mass Functions
PXY ( xi , y j )
PX / Y ( xi / y j )  , PY ( y j )  0
PY ( y j )
PXY ( xi , y j )
PY / X ( y j / xi )  , PX ( xi )  0
PX ( xi )
By Mohammed M Chapter 3: Multiple Random Variables 11
Correlation and Covariance
i. Correlation

R XY Cor ( X , Y ) E ( XY )

ii. Covariance

 XY Cov( X , Y ) E[( X   X )(Y  Y )]

  XY Cov( X , Y ) E ( XY )  E ( X ) E (Y )

iii. Correlation Coefficient

Cov ( X , Y )  XY
 XY  
 XY  XY
By Mohammed M Chapter 3: Multiple Random Variables 12
 Examples on Two Random Variables

Example-1:
The joint pdf of two continuous random variables X and Y is
given by:
kxy , 0  x  1, 0  y  1
f XY ( x, y ) 
0 , otherwise
whe re k is a constant.
a. Find the value of k .
b. Find the marginal pdf of X and Y .
c. Are X and Y independent?
d . Find P ( X  Y  1)
e. Find the conditional pdf of X and Y .
By Mohammed M Chapter 3: Multiple Random Variables 13
 Examples on Two Random Variables Cont’d……

Solution:
  1 1
a. 
-  
f XY ( x, y )dxdy 1  kxydxdy 1
0 0

 x2  1
1
 k y  1
0
 2 0
k 1  y2 1 k
 ydy k    1
2 0  4 0 4
 k 4

By Mohammed M Chapter 3: Multiple Random Variables 14

 Examples on Two Random Variables Cont’d……

Solution:
b. Marginal pdf of X and Y
i. Marginal pdf of X
 1
f X ( x)  f XY ( x, y )dy 4 xydy
 0

 y 2
1
 f X ( x) 4 x  2 x
 2 0
2 x , 0  x 1
 f X ( x) 
0, otherwise
By Mohammed M Chapter 3: Multiple Random Variables 15
 Examples on Two Random Variables Cont’d……

Solution:
b. Marginal pdf of X and Y
ii. Marginal pdf of Y
 1
fY ( y )  f XY ( x, y )dx 4 xydx
 0

 x 2
1
 fY ( y ) 4 y  2 y
 2 0
2 y , 0  y 1
 fY ( y ) 
0, otherwise
By Mohammed M Chapter 3: Multiple Random Variables 16
 Examples on Two Random Variables Cont’d……

Solution:
c. f XY ( x, y )  f X ( x) fY ( y )
 X and Y are independent
1 1 y  x2  1
1
d . P( X  Y  1)  4 xydxdy 4 y  dy
0 0 0
 2 0
1 1
4 y[1 / 2(1  y ) ]dy 2( y  2 y 2  y 3 )dy
2
0 0

2( y 2 / 2  2 y 3 / 3  y 4 / 4) 1 / 6
 P( X  Y  1) 1 / 6

By Mohammed M Chapter 3: Multiple Random Variables 17

 Examples on Two Random Variables Cont’d……

Solution:
e. Conditional pdf of X and Y
i. Conditional pdf of X
f XY ( x, y ) 4 xy
f X /Y ( x / y)   2 x
fY ( y ) 2y

2 x, 0  x  1, 0  y  1
 f X / Y ( x / y ) 
0, otherwise

By Mohammed M Chapter 3: Multiple Random Variables 18

 Examples on Two Random Variables Cont’d……

Solution:
e. Conditional pdf of X and Y

ii. Conditional pdf of Y

f XY ( x, y ) 4 xy
fY / X ( y / x)   2 y
f X ( x) 2x

2 y, 0  x  1, 0  y  1
 fY / X ( y / x) 
0, otherwise

By Mohammed M Chapter 3: Multiple Random Variables 19

 Examples on Two Random Variables Cont’d……

Example-2:
The joint pdf of two continuous random variables X and Y is
given by:
k , 0  y x  1
f XY ( x, y ) 
0, otherwise
where k is a constant.
a. Determine the value of k .
b. Find the marginal pdf of X and Y .
c. Are X and Y independent?
d . Find P (0  X  1 / 2)
e. Find the conditional pdf of X and Y .
By Mohammed M Chapter 3: Multiple Random Variables 20
 Examples on Two Random Variables Cont’d……

Solution:
  1 1
a. 
-  
f XY ( x, y )dxdy 1  kdxdy 1
0 y

1 1
 k x  1
0 y
1  y2 1 k
 k (1  y )dy k  y    1
0
 2 0 2
 k 2

By Mohammed M Chapter 3: Multiple Random Variables 21

 Examples on Two Random Variables Cont’d……

Solution:
b. Marginal pdf of X and Y
i. Marginal pdf of X
 x
f X ( x)  f XY ( x, y )dy  2dy
 0

x
 f X ( x) 2 y  2 x
0
2 x , 0  x 1
 f X ( x) 
0, otherwise
By Mohammed M Chapter 3: Multiple Random Variables 22
 Examples on Two Random Variables Cont’d……

Solution:
b. Marginal pdf of X and Y
ii. Marginal pdf of Y
 1
fY ( y )  f XY ( x, y )dx 2dx
 y

1
 fY ( y ) 2 x  2(1  y )
y
2(1  y ), 0  y 1
 fY ( y ) 
0, otherwise
By Mohammed M Chapter 3: Multiple Random Variables 23
 Examples on Two Random Variables Cont’d……

Solution:
c. f XY ( x, y )  f X ( x) fY ( y )
 X and Y are not independent
1/ 2 x
d . P(0  X  1 / 2)  f XY ( x, y )dydx
0 0

1/ 2 x 1/ 2 x
 2dydx  (2 y ) dx
0 0 0 0
1/ 2 1/ 2
 2 xdx  x 2
1 / 4
0 0
 P(0  X  1 / 2) 1 / 4
By Mohammed M Chapter 3: Multiple Random Variables 24
 Examples on Two Random Variables Cont’d……

Solution:

e. Conditional pdf of X and Y

i. Conditional pdf of X
f XY ( x, y ) 2 1
f X /Y ( x / y)   
fY ( y ) 2(1  y ) (1  y )
 1
 , 0  y x  1
 f X / Y ( x / y ) 1  y
0, otherwise


By Mohammed M Chapter 3: Multiple Random Variables 25

 Examples on Two Random Variables Cont’d……

Solution:

e. Conditional pdf of X and Y

ii. Conditional pdf of Y
f XY ( x, y ) 2 1
fY / X ( y / x)   
f X ( x) 2x x
1
 , 0  y x  1
 fY / X ( y / x)  x
0, otherwise

By Mohammed M Chapter 3: Multiple Random Variables 26

 Examples on Two Random Variables Cont’d……

Example-3:
The joint pmf of two discrete random variables X and Y is given
by:
k (2 xi  y j ) , xi 1, 2; y 1, 2
PXY ( xi , y j ) 
0 , otherwise
where k is a constant.
a. Find the value of k .
b. Find the marginal pmf of X and Y .
c. Are X and Y independent?

By Mohammed M Chapter 3: Multiple Random Variables 27

 Examples on Two Random Variables Cont’d……

Solution:

a. P
xi yj
XY ( xi , y j ) 1

2 2
   k (2 x
xi 1 y j 1
i  y j ) 1

 k[(2  1)  (2  2)  (4  1)  (4  2)] 1

 18k 1

 k 1 / 18
By Mohammed M Chapter 3: Multiple Random Variables 28
 Examples on Two Random Variables Cont’d……

Solution:
b. Marginal pmf of X and Y
i. Marginal pmf of X
2
1
PX ( xi )  PXY ( xi , y j )   (2 xi  y j )
yj y j 118

1 1
 PX ( xi )  (2 xi  1)  (2 xi  2)
18 18
1
 (4 xi  3), xi 1, 2
 PX ( xi ) 18

0, otherwise
By Mohammed M Chapter 3: Multiple Random Variables 29
 Examples on Two Random Variables Cont’d……

Solution:
b. Marginal pmf of X and Y
ii. Marginal pmf of Y
2
1
PY ( y j )  PXY ( xi , y j )  (2 xi  y j )
xi xi 1 18

1 1
 PY ( y j )  (2  y j )  (4  y j )
18 18
1
 (2 y j  6), y j 1, 2
 PY ( y j ) 18

0, otherwise
By Mohammed M Chapter 3: Multiple Random Variables 30
 Examples on Two Random Variables Cont’d……

Solution:

c. PXY ( xi , y j )  PX ( xi ) PY ( y j )
 X and Y are not independent.

By Mohammed M Chapter 3: Multiple Random Variables 31

• Part2

By Mohammed M Chapter 3: Multiple Random Variables 32

 Multiple Random Variables
Outline
 Introduction
 The Joint Cumulative Distribution Function
 The Joint Probability Density and Mass Functions
 Marginal Statistics
 Independence
 Conditional Distributions
 Correlation and Covariance
 Functions of Two Random Variables

By Mohammed M Chapter 3: Multiple Random Variables 33

 One Function of Two Random Variables

Given two random variables X and Y and a function g(x,y),

we form a new random variable Z as
Z  g ( X , Y ).

Given the joint pdf f XY ( x, y ), how does one obtain f Z (z ),

the pdf of Z ? Problems of this type are of interest from a
practical standpoint. For example, a receiver output signal
usually consists of the desired signal buried in noise, and
the above formulation in that case reduces to Z = X + Y.
By Mohammed M Chapter 3: Multiple Random Variables 34
 It is important to know the statistics of the incoming signal
for proper receiver design. In this context, we shall analyze
problems of the following type:
X Y
max( X , Y ) XY
min( X , Y ) Z g ( X ,Y ) XY

X 2 Y 2 X /Y
tan  1 ( X / Y )

The cdf of Z is given by:

FZ ( z ) P Z ( )  z  P g ( X , Y )  z 
 f XY ( x, y )dxdy,
By Mohammed M Chapter
x , y 3: Multiple Random Variables 35
 Example -1: Let Z = X + Y. Find f Z (z ).
Solution:
 z y
FZ ( z )  P X  Y  z    f XY ( x, y )dxdy ,
y  x 

Since the required region in the xy plane where x  y  z is

the shaded area shown below to the left of the line x  y  z.
Integrating over the horizontal strip along the x-axis first
(inner integral) followed by sliding that strip along the y-axis
from   to   (outer integral) we cover the entire shaded
area. y

x z  y

x
By Mohammed M Chapter 3: Multiple Random Variables 36
 We can find f Z (z ) by differentiating FZ (z ) directly. In this
context, it is useful to recall the differentiation rule by
Leibnitz. Suppose
b( z )
H ( z )  h ( x, z )dx.
a( z)
Then,
dH ( z ) db( z ) da ( z ) b ( z ) h ( x , z )
 h b( z ), z  h a ( z ), z   dx.
dz dz dz a ( z ) z
Using the above two equations, we get
  z y z  y f
XY ( x , y ) 
   
f Z ( z )     f XY ( x, y )dx  dy    f XY ( z  y , y )  0    dy
  z   z
  
 


  f XY ( z  y , y )dy. (i)


Alternatively, the above integration can be carried out first

along
By Mohammed M the y-axis followed by the
Chapter 3: Multiple x-axis
Random as below.
Variables 37
 In the second case
 z x y
FZ ( z )   f XY ( x, y )dxdy ,
x  y 

and differentiating the above

y z  x
equation gives
dFZ ( z )    z x 
f Z ( z)     f XY ( x, y )dy  dx x
dz x 
 z y  

 f XY ( x, z  x )dx. (ii)
x 

If X and Y are independent, then

f XY ( x, y )  f X ( x ) fY ( y ) (iii)

and inserting equation (iii) into equations (i) and (ii), we get
 
f Z ( z )  f X ( z  y ) fY ( y )dy  f X ( x ) fY ( z  x )dx. (iv)
y  x 
By Mohammed M Chapter 3: Multiple Random Variables 38
 The above integral is the standard convolution of the
functions f X (z ) and fY (z ) expressed two different ways. We
thus reach the following conclusion: If two random variables
are independent, then the density of their sum equals the
convolution of their density functions.
As a special case, suppose that f X ( x ) 0 for x  0 and fY ( y ) 0
for y  0, then we can use the following figure to determine
the pdf of Z.
y

(z,0)

x z  y
x
(0, z )
By Mohammed M Chapter 3: Multiple Random Variables 39
 In the above case,
z z y
FZ ( z )   f XY ( x, y )dxdy
y 0 x 0

or
  z

f Z ( z )    f XY ( x, y )dx  dy  0 XY
z  z  y   f ( z  y , y )dy, z  0,
y 0 z x 0
   0, z 0.
On the other hand, by considering vertical strips first in
above figure, we get
z z x
FZ ( z )   f XY ( x, y )dydx
x 0 y 0

or
 z f ( x ) f ( z  x )dx, z  0,
f XY ( x, z  x )dx  y 0 X
z
f Z ( z )  Y
x 0
 0, z 0,
if X and Y are independent random variables.
By Mohammed M Chapter 3: Multiple Random Variables 40
 Example-2: Suppose X and Y are independent exponential
r.vs with common parameter , and let Z = X + Y.
Determine f (z ).
Z
Solution: We have f ( x ) e  xU ( x ), f ( y ) e  yU ( y ),
X Y
and we can make use of (13) to obtain the pdf
of Z = X + Y.
z z
f Z ( z )  2 e  x e   ( z  x ) dx 2 e  z  dx  z2 e  zU ( z ).
0 0

As the next example shows, care should be taken in using

the convolution formula for r.vs with finite range.
Example-3: X and Y are independent uniform r.vs in the
f Z (z ),
common interval (0,1). Determine where Z = X + Y.
Z X Y  0  z  2
Solution: Clearly, here, and as following
figure shows there are two cases of z for which the shaded
areas
By Mohammed M are quite different
Chapterin shape
3: Multiple andVariables
Random they should be 41
 y y

x z  y
x z  y
x x
(a ) 0  z  1 ( b) 1  z  2

For 0 z  1,
z z y z z2
FZ ( z )   1 dxdy  ( z  y )dy  , 0  z  1.
y 0 x 0 y 0 2
For 1 z  2, notice that it is easy to deal with the unshaded
region. In that case
1 1
FZ ( z ) 1  P Z  z  1    1 dxdy
y z  1 x z  y

1 (2  z )2
By Mohammed M
1   Chapter (1  z  y )dy 1  , 1  z  2.
3: Multiple Random Variables 42
y z  1 2
 Finally, differentiating the cdf, we obtain
dFZ ( z )  z 0  z  1,
fZ ( z)  
dz 2  z, 1  z  2.
By direct convolution of f X ( x ) and fY ( y ), we obtain the
same result as above. In fact, for 0 z  1 [Figure (a)]
z
f Z ( z ) f X ( z  x ) fY ( x )dx  1 dx  z.
0

and for 1 z  2 [Figure (b)]

1
f Z ( z )  1 dx 2  z.
z 1

Figure (c) shows f Z (z ) which agrees with the convolution

of two rectangular waveforms as well.

By Mohammed M Chapter 3: Multiple Random Variables 43

 fY (x ) f X ( z  x) f X ( z  x ) fY ( x )

x x x
1 z 1 z z

( a ) 0 z  1

fY (x ) f X ( z  x) f X ( z  x ) fY ( x )

x x x
1 z 1 z
z 1 1
(b) 1 z  2

f Z (z )

z
0 1 2

By Mohammed M
(c)
Chapter 3: Multiple Random Variables 44
 Example-4: Let Z  X  Y . Determine its pdf f Z (z ).
Solution:
 z y
FZ ( z )  P X  Y  z    f XY ( x, y )dxdy
y  x 

and hence
dFZ ( z )    z y  
fZ ( z)    f XY ( x, y )dx  dy   f XY ( y  z , y )dy.
dz y    z
 x  
 

If X and Y are independent, then the above formula reduces

to 
f Z ( z )  f X ( z  y ) fY ( y )dy  f X (  z )  f Y ( y ),


which represents the convolution of f X (  z ) with fY (z ).

y

y x  y z
x y  z
x
By Mohammed M Chapter 3: Multiple Random Variables 45
 As a special case, suppose
f X ( x ) 0, x  0, and fY ( y ) 0, y  0.

In this case, Z can be negative as well as positive, and that

gives rise to two situations that should be analyzed
separately, since the region of integration for z 0 and z  0
are quite different. For z 0, from Figure (a) y
 z y
FZ ( z )   f XY ( x, y )dxdy x z  y
y 0 x 0

z x
and for z  0, from Figure (b)  z
(a)
 z y
FZ ( z )   f XY ( x, y )dxdy y
y  z x 0

x z  y
After differentiation, this gives
 z
 f ( z  y , y )dy , z 0,
  XY x
f Z ( z )  0 
(b)

 f XY ( z  y , y )dy , z  0.
z
By Mohammed M Chapter 3: Multiple Random Variables 46
 Example-5: Given Z = X / Y, obtain its density function.
Solution: We have
FZ ( z )  P X / Y  z .
The
X Yz Y  0,
X / Y z
inequality
X Yz
can be rewritten as if and
Y  0.
if
P X / Y  z   P X / Y  z, Y  0   P X / Y  z, Y  0 
 P X Yz , Y  0   P X Yz , Y  0 .

Figure (a) shows the area corresponding to the first term,

and Figure (b) shows that corresponding to the second term
in the above equation.
y
y
x  yz
x  yz x

x
(b)
(a)

By Mohammed M Chapter 3: Multiple Random Variables 47

 Integrating over these two regions, we get
 yz 0 
FZ ( z )   f XY ( x, y )dxdy    f XY ( x, y )dxdy.
y 0 x  y  x yz

Differentiation with respect to z gives

 0
f Z ( z )  yf XY ( yz, y )dy   (  y ) f XY ( yz, y )dy
0 

 | y | f XY ( yz, y )dy,    z  .


Note that if X and Y are nonnegative random variables, then

the area of integration reduces to that shown below
y
x  yz

By Mohammed M Chapter 3: Multiple Random Variables 48

 Exercise

The joint pdf of two random variables X and Y is given by:

x  y , 0  x 1, 0  y 1
f XY ( x, y ) 
0 , otherwise
Find the pdf of Z if :

a. Z  X  Y c. Z  XY
X
b. Z  d. Z  X  Y
Y

By Mohammed M Chapter 3: Multiple Random Variables 49

 Assignment-III

1. Suppose that two continuous random variables X and Y

have joint pdf given by:
k ( 2 x  y ) , 2  x  6, 0  y  5
f XY ( x, y ) 
0 , otherwise
where k is a constant.
a. Find the constant k .
b. Find the joint cdf of X and Y .
c. Find the marginal cdf and pdf of X and Y .
d . Are X and Y independen t?
e. Find the conditional pdf of X and Y .
e. Evaluate the following probabilities.
i. P (3  X  4, Y  2) iii. P ( X  Y  4)
By Mohammed M  3) 3: Multiple Random Variables
ii. P ( X Chapter iv. P (0  Y  2) 50
 Assignment-III Cont’d……

2. Let the joint pmf of two discrete random variables X and

Y is given by:
k ( xi  y j ) , xi 1, 2, 3; y 1, 2
PXY ( xi , y j ) 
0 , otherwise
where k is a constant.
a. Find the value of k .
b. Find the marginal pmf of X and Y .
c. Are X and Y independent?

By Mohammed M Chapter 3: Multiple Random Variables 51

 Assignment-III Cont’d…..
3. Suppose that two continuous random variables X and Y
are independent and uniform in the interval (0, 4).
Find the pdf of Z if :
a. Z  X  Y
b. Z Y  X
c. Z  X  Y
d . Z  XY
Y
e. Z 
X

By Mohammed M Chapter 3: Multiple Random Variables 52