Probability and Statistics

Prof. Zheng Zheng

z. zheng 1
 Two Random Variables
In many experiments, the observations are expressible not
as a single quantity, but as a family of quantities. For
example to record the height and weight of each person in
a community or the number of people and the total income
in a family, we need two numbers.

Let X and Y denote two random variables (r.v) based on a

probability model (, F, P). Then
P  x 1  X ( )  x 2   F X ( x 2 )  F X ( x 1 )   f X ( x ) dx ,
x 2

x 1

P  y 1  Y ( )  y 2   FY ( y 2 )  FY ( y 1 ) 
y2
and  y1
f Y ( y ) dy .
z. zheng 2
What about the probability that the pair of r.vs (X,Y) belongs
to an arbitrary region D? In other words, how does one
estimate, for example, P ( x1  X ( )  x 2 )  ( y1  Y ( )  y 2 )   ?
Towards this, we define the joint probability distribution
function of X and Y to be
FXY ( x , y )  P ( X ( )  x )  (Y ( )  y ) 
 P ( X  x , Y  y )  0,
where x and y are arbitrary real numbers.
Properties
(i) FXY (  , y )  FXY ( x ,  )  0, FXY (  ,  )  1 .
since  X (  )   , Y (  )  y    X ( )    , we get
z. zheng 3
FXY ( , y )  P X ( )    0. Similarly  X ( )   , Y ( )      ,
we get FXY (, )  P()  1.
(ii) P  x 1  X ( )  x 2 , Y ( )  y   F XY ( x 2 , y )  F XY ( x 1 , y ).

P  X ( )  x , y 1  Y ( )  y 2   F XY ( x , y 2 )  F XY ( x , y 1 ).

To prove it, we note that for x2  x1,

 X ( )  x2 , Y ( )  y    X ( )  x1 , Y ( )  y   x1  X ( )  x2 , Y ( )  y 

and the mutually exclusive property of the events on the

right side gives
P  X ( )  x2 , Y ( )  y   P  X ( )  x1 , Y ( )  y   P  x1  X ( )  x2 , Y ( )  y 

z. zheng 4
(iii) P  x1  X ( )  x2 , y1  Y ( )  y2   FXY ( x2 , y2 )  FXY ( x2 , y1 )
 FXY ( x1 , y2 )  FXY ( x1 , y1 ).
This is the probability that (X,Y) belongs to the rectangle R0
in Fig. 1. To prove it, we can make use of the following
identity involving mutually exclusive events on the right.
x1  X ( )  x2 ,Y ( )  y2   x1  X ( )  x2 ,Y ( )  y1   x1  X ( )  x2 , y1  Y ( )  y2 .
This gives
P x1  X ( )  x2 ,Y ( )  y2   Px1  X ( )  x2 ,Y ( )  y1   P x1  X ( )  x2 , y1  Y ( )  y2 
Y

y2
R0
y1

x1 x2
z. zheng 5
Fig. 1
 Joint probability density function (Joint p.d.f)
By definition, the joint p.d.f of X and Y is given by
 2 FXY ( x , y )
f XY ( x , y )  .
x y
and hence we obtain the useful formula
x y
F XY ( x , y )   
 
f XY ( u , v ) dudv .

Using (7-2), we also get

 
 
 
f XY ( x , y ) dxdy  1 .

z. zheng 6
To find the probability that (X,Y) belongs to an arbitrary
region D,
P  x  X ( )  x  x, y  Y ( )  y  y   FXY ( x  x, y  y )
 FXY ( x, y  y )  FXY ( x  x, y )  FXY ( x, y )
x  x y  y
  f XY (u, v )dudv  f XY ( x, y )xy.
x y

Thus the probability that (X,Y) belongs to a differential

rectangle x y equals f XY ( x , y )   x  y , and repeating this
procedure over the union of no overlapping differential
rectangles in D, we get the useful result Y

P ( X , Y )  D     f XY ( x , y ) dxdy . D y
( x , y ) D x

Fig. 2
z. zheng 7
Example:
Two refills for a ballpoint pen are selected at random from
a box that contains 3 blue refills, 2 red refills, and 3 green
refills. If X is the number of blue refills and Y is the number
of red refills selected, find
(a) the joint probability function f(x,y),
(b) P[(X,Y)  A], where A is the region {(x,y) | x + y < 1}.

Solution:
(x,y) ={ (0, 0), (0,1), (1, 0), (1, 1), (0, 2), (2, 0)}

Where, f(0,1), for example, represents the probability that a red

and a green refill are selected
z. zheng 8
Continued

8 
# of equally likely ways of selecting any 2 refills from the 8 =  
=28  2

number of ways of selecting 1 red from 2 red refills and 1 green

from 3 green refills =  2  3  = 6
1 1 
Thus f (0,1)  6 / 28  3 / 14
Similarly

z. zheng 9
Continued

(b) P( X , Y )  A  P( X  Y  1)  f (0,0)  f (0,1)  f (1,0)

3 3 9 9
   
28 14 28 14

z. zheng 10
Example:
A fast food business operates both a drive-in facility and a
walk-in facility. On a randomly selected day, let X and Y ,
respectively, be the proportions of the time that the drive-in
and the walk-in facilities are in use, and suppose that the
joint density function of these random variables is
2
 (2 x  3 y ), 0  x  1,0  y  1
f ( x, y )   5
0, elsewhere .

Question:  
(a) Verify   f ( x, y )  1
  

(b) Find P[(X, Y  A], where A = { (x,y) | 0< x < ½, ¼ < y < ½ }.
z. zheng 11
(a)

(b)

z. zheng 12
 Marginal Statistics
In the context of several r.vs, the statistics of each individual
ones are called marginal statistics. Thus FX (x) is the
marginal probability distribution function of X, and f X (x) is
the marginal p.d.f of X. It is interesting to note that all
marginals can be obtained from the joint p.d.f. In fact
FX ( x )  FXY ( x ,  ), FY ( y )  FXY (  , y ).

Also f ( x )   f ( x , y ) dy , f ( y )   f ( x , y ) dx .
X   XY Y   XY
To it, we can make use of the identity
( X  x )  ( X  x )  (Y   )
z. zheng 13
so that F X ( x )  P  X  x   P  X  x , Y     F XY ( x ,  ).
To prove it, we can make use of the earlier equations,
which gives F X ( x )  F XY ( x ,  )   x    f XY ( u , y ) dudy
 

and taking derivative with respect to x in it, we get


f X ( x)   
f XY ( x , y ) dy .

At this point, it is useful to know the formula for

differentiation under integrals. Let
b( x )
H ( x)   a(x)
h ( x , y ) dy .

Then its derivative with respect to x is given by

dH ( x ) db( x ) da ( x ) b ( x ) h ( x , y )
 h ( x , b)  h( x, a )   dy.
dx dx dx a ( x ) x
Thus we get it proven. z. zheng 14
 
If X and Y are discrete r.vs, then pij  P ( X  xi , Y  y j ) represents
their joint p.d.f, and their respective marginal p.d.fs are
given by
P( X  xi )   P( X  xi , Y  y j )   pij
j j
and
P (Y  y j )   P ( X  xi , Y  y j )   pij
i i

Assuming that P( X  xi , Y  y j ) is written out in the form of a

rectangular array, to obtain P( X  xi ), from (7-17), one need
to add up all entries in the i-th row.
i
p ij
p11 p12  p1 j  p1n
p21 p22  p2 j  p2 n
     
j
p ij pi1 pi 2  pij  pin
     
pm1 pm 2  pmj  pmn z. zheng 15
Fig. 3
 The joint P.D.F and/or the joint p.d.f represent complete
information about the r.vs, and their marginal p.d.fs can be
evaluated from the joint p.d.f. However, given marginals,
(most often) it will not be possible to compute the joint p.d.f.
Consider the following example:
Y
Example 1: Given
1
 constant, 0  x  y  1,
f XY ( x , y )   y
 0, otherwise . X
0 1
Obtain the marginal p.d.fs and f X (x) fY ( y ).
Fig. 4
Solution: Since it is given that the joint p.d.f f XY ( x, y ) is a
constant in the shaded region, we can determine the constant c.
2 1
  1
y
 dy 1 cy c
 
 
f XY ( x , y ) dxdy  y  0   x  0 
c  dx  y 0 cydy 
2

2
 1.
0

z. zheng 16
Thus c = 2. Moreover
 1
f X ( x)   
f XY ( x , y ) dy   yx
2 dy  2 (1  x ), 0  x  1,

and similarly
 y
fY ( y )   
f XY ( x , y ) dx   x 0
2 dx  2 y , 0  y  1.

Clearly, in this case given f X (x) and fY ( y ) as in the above, it

will not be possible to obtain the original joint p.d.f.
Example 2: X and Y are said to be jointly normal (Gaussian)
distributed, if their joint p.d.f has the following form:
1  ( x   X ) 2 2  ( x   X )( y  Y ) ( y  Y ) 2 
   
1 2 (1  2 )   X2  XY  Y2 
f XY ( x , y )  e 
,
2 X  Y 1   2

   x   ,    y   , |  | 1.
z. zheng 17
By direct integration, it can be shown that
 1
 f
fX ( x)  XY ( x , y ) dy  e  ( x   X )2 / 2 2
X ~ N (  X ,  X2 ),

2  2
X

and similarly
 1
fY ( y )  f XY ( x , y ) dx  e  ( y   Y ) 2 / 2  Y2
~ N (  Y ,  Y2 ),

2  2
Y

Following the above notation, we will denote the original

as N (  X ,  Y ,  X2 ,  Y2 ,  ). Once again, knowing the marginals
alone doesn’t tell us everything about the joint p.d.f .

z. zheng 18
Example:
Find the marginal pdf of x and y for the joint density
function of the earlier example.

Solution:

z. zheng 19
Example:

Solution:

z. zheng 20
 Expectation of Jointly Distributed RV’s

• Recall: • Now:
Discrete Discrete

E[X]   x  f (x) E[u]    u  f (x, y)

x y

E[u]   u  f (x) E[h(x, y)]    h(x, y)  f (x, y)

x y

E[h(x)]   h(x)  f (x) E[X 2  5Y ]    (x 2  5y)  f (x, y)

x y

Continuous
Continuous
E[h(x)]   h(x)  f (x)  
E[h(x, y)]    h(x, y)  f (x, y)
 

z. zheng 21
As we show below, the only situation where the marginal
p.d.fs can be used to recover the joint p.d.f is when the
random variables are statistically independent.

z. zheng 22
 Independence of r.vs
Definition: The random variables X and Y are said to be
statistically independent if the events X ( )  A and {Y ( )  B}
are independent events for any two sets A and B in x and y
axes respectively. Applying the above definition to the
events X ( )  x and Y ( )  y , we conclude that, if the
r.vs X and Y are independent, then
P ( X ( )  x )  (Y ( )  y )   P ( X ( )  x ) P (Y ( )  y )
i.e.,
FXY ( x , y )  FX ( x ) FY ( y )

or equivalently, if X and Y are independent, then we must

have f XY ( x , y )  f X ( x ) fY ( y ).
z. zheng 23
If X and Y are discrete-type r.vs then their independence
implies
P( X  xi ,Y  y j )  P( X  xi ) P(Y  y j ) for all i, j.

These Equations give us the procedure to test for

independence. Given f XY ( x, y ), obtain the marginal p.d.fs f X (x)
and fY ( y ) and examine whether the above is valid. If so, the
r.vs are independent, otherwise they are dependent.
Returning back to Example 1, from those equations, we
observe by direct verification that f XY ( x, y )  f X ( x) fY ( y ). Hence
X and Y are dependent r.vs in that case. It is easy to see that
such is the case in the case of Example 2 also, unless   0.
In other words, two jointly Gaussian r.vs are independent if
and only if the fifth parameter   0.
z. zheng 24
Example 3: Given
 xy 2 e  y , 0  y  , 0  x  1,
f XY ( x, y )  
 0, otherwise.
Determine whether X and Y are independent.
Solution:
 
f X ( x)   0
f XY ( x , y ) dy  x 
0
y 2 e  y dy

 x   2 ye  y ye  y dy   2 x ,
 
 2 0  x  1.
 0 0 
Similarly
1 y2 y
fY ( y )   0
f XY ( x , y ) dx 
2
e , 0  y  .

In this case
f XY ( x , y )  f X ( x ) fY ( y ),

and hence X and Y are independent random variables.

z. zheng 25