MATH TRIPOS: PART II L2011 YMS

APPLIED PROBABILITY
Example Sheet 1 (with comments)
1. Suppose X, Y and Z are exponential random variables of parameters
, and respectively. What is the distribution of W = min{X, Y, Z}?
What is the probability that Z Y ? Z Y X? Show that random
variable W and the event {Z Y X} are independent. State and prove
a similar result for n random variables. Hint: Work with tail probabilities.
2. Let T
1
, T
2
, . . . be independent exponential random variables of parameter
and let N be an independent geometric random variable with
P(N = n) = (1 )
n1
, n = 1, 2, . . . .
Show that T =
N

i=1
T
i
has exponential distribution of parameter . Show
that, n 1, the sum S =
n

i=1
T
i
has Gamma-distribution Gamma (n, ),
with the PDF f
S
(x) =

n
x
n1
(n 1)!
e
x
, x > 0. Hint: For S you may use
induction in n. For T, calculating the moment-generating function can be
useful, along with the fact that it determines the distribution uniquely.
[For S: use induction in n. For n = 1: S = T
1
Exp() Gam(1, ),
trivially. Make the induction hypothesis for n and pass to n + 1:
S
n+1
= S
n
+ T
n+1
, independently.
Thus, the PDF f
S
n+1
is given by
f
S
n+1
(x) =
_
x
0
f
Sn
(s)f
T
1
(x s)ds (the convolution formula).
Now use the induction hypothesis:
f
S
n+1
(x) =
_
x
0

n
s
n1
(n 1)!
e
s
e
(xs)
ds
= e
x

n+1
(n 1)!
_
x
0
s
n1
ds
=

n+1
x
n
n!
.
1
For T: use the above fact and sum a geometric progression or proceed
with the MGFs (also arriving at a geometric progression):

T
() = Ee
T
= E
_
E
_
e
T

N
_
_
=

n1
E
_
e
T

N = n
_
P(N = n)
=

n
(1 )
n1
_
Ee
T
1
_
n
=

1

n
_
(1 )


_
n
=

1
(1 )

__
1
(1 )

_
=

(1 )
=


.
As the MGF determines the PDF uniquely, conclude that T Exp().]
3. Prove that the determinant det e
tQ
is of the form e
tq
and hence is > 0 for
any nite Q-matrix. Can you determine q?
Hint: Use the semi-group property to calculate det e
(t+s)Q
; by analysing
the expansion at t 0 conclude that q =tr Q.
[ det e
(t+s)Q
=
_
det e
sQ
__
det e
tQ
_
, and det e
tQ
is continuous (and even
dierentiable) in t, with det e
0Q
=det I = 1. Hence, det e
tQ
= e
tq
for some
(real) q. To nd q, one can observe that
q =
d
dt
det e
tQ

t=0
=
d
dt
det (1 +tQ)

t=0
.
That is, q is the rst order coecient of the polynomial det (I + tQ) which
is precisely tr Q. Note that unless Q = 0, det e
tQ
0 as t .]
(b) Prove that the following transition probability matrices cannot be
written in the form e
Q
where Q is a 3 3 Q-matrix: (i) P =
_
_
1 0 0
1 0 0
0 1 0
_
_
,
(ii) P =
_
_
0 1 0
0 0 1
1 0 0
_
_
. Hint: it is instructive to consider the matrix P
3
.
4. State the denition of a Poisson process in terms of independent incre-
ments. Show directly from this denition that the rst jump time of a Poisson
process of rate is exponential of parameter .
2
Furthermore, show that the holding times of the process are IID expo-
nential variables of rate . Hint: Deduce the characterisation of a Poisson
process through its holding times from its characterisation through indepen-
dent increments.
[For Poisson processes, then for birth and birth and death processes and
nally for continuous-time Markov chains, there were three denitions given,
(a) in terms of the transition probabilities (in the case of a Poisson process,
in terms of increments), (b) in terms of the innitesimal probabilities, (c) in
terms of the jump chain and holding times. Many of the standard problems
revolve around the equivalence of these various denitions, so it is necessary
to make explicit which is to be regarded as the starting point. If none is
mentioned the student is free to choose the most convenient denition.
5. Assume the innitesimal denition of the Poisson process. Let p(t) de-
note the probability that the rst jump time exceeds t. By considering the
dierence p(t + h) p(t), deduce a linear diferential equation for p(t) and
solve it.
[Set p(t) = P(X
t
= 0), then
p(t + h) = P(X
t
= 0 and X
t+h
X
t
= 0)
= P(X
t
= 0)P(X
t+h
X
t
= 0) = p(t)(1 h + o(h))
so
p(t + h) p(t) = hp(t) + o(h)
uniformly in t 0. Hence p(t) is dierentiable and
p

(t) = p(t), p(0) = 1 ,

which shows p(t) = e
t
. This is a simple version of an argument used in
lectures to get the distribution of the whole process.]
6. Arrivals of the Number 1 bus form a Poisson process of rate 1 bus per
hour, and arrivals of the Number 7 bus form an independent Poisson process
of rate 7 buses per hour.
(a) What is the probability that exactly 5 buses pass by in 1 hour?
(b) What is the probability that exactly 3 Number 7 buses pass by while
I am waiting for a Number 1?
(c) When the maintenance depot goes on strike half the buses break down
before they reach my stop. What then is the probability that I wait for 30
minutes without seeing a single bus?
3
Answers: (a) e
8
8
5
/4!, (b) 7
3
/8
4
, (c) e
2
.
[Done by using y standard properties of the Poisson process.]
7. Customers arrive in a supermarket as a Poisson-process of rate N. There
are N aisles in the supermarket and each customer selects one aisle at ran-
dom, independently of the other customers. Let X
N
t
denote the proportion
of aisles which remain empty at time t and let T
N
denote the time until half
the aisles are busy. Show that
X
N
t
e
t
, T
N
log 2
in probability as N .
Hint: Identify the processses of customers arrival in each aisle and relate
them to the event {T
N
> t}.
[Customers arrive in each aisle as independent Poisson processes of rate
1. Let Y
i
t
be the indicator function of the event that the ith aisle is empty at
time t. Then Y
1
t
, . . . , Y
N
t
are independent Bernoulli with P(Y
i
t
= 1) = e
t
.
By the law of large numbers
X
N
t
= N
1
(Y
1
t
+ + Y
N
t
) e
t
almost surely as N . Since {T
N
> t} = {X
N
t
1/2} it follows that
P(T
N
> t)
_
1 if t < ln 2,
0 if t > ln 2,
so T
N
ln 2 in probability.]
8. A pedestrian wishes to cross a single lane of fast-moving trac. Suppose
the number of vehicles that have passed by time t is a Poisson process of
rate , and suppose it takes time a to walk across the lane. Assuming the
pedestrian can foresee correctly the times at which vehicles will pass by, how
long on average does it take to cross over safely?
Hint: Let T be the time to cross and J
1
the ttime at which the rst
car passes. Identify the contributions to E T from the events {J
1
> a} and
{J
1
< a}.
How long on average does it take to cross two similar lanes (a) when one
must walk straight across, (b) when an island in the middle of the road makes
it safe to stop half way?
4
[Let T be the time to cross and J
1
the time at which the rst car passes.
Then
E(T) = aP(J
1
> a) +
_
a
0
(s +E(T))e
s
ds
= (1 e
a
) (E(T) + 1/) .
Hence E(T). Check correct as 0. Walking straight across replaces a by
2a and by 2, because the sum of two independent Poisson processes of rate
is a Poisson process of rate 2. Using an island, it takes twice as long on
average as a single lane. How much quicker is it to use an island?]
9. Customers enter a supermarket as a Poisson process of rate 2. There are
two salesmen near the door who oer passing customers samples of a new
product. Each customer takes an exponential time of parameter 1 to think
about the new product, and during this time occupies the full attention of
one salesman. Having tried the product, customers proceed into the store
and leave by another door. When both salesmen are occupied, customers
walk straight in. Assuming that both salesmen are free at time 0, nd the
probability that both are busy at a later time t.
Hint: Find out the 3 3 Q-matrix and work with its eigen-values. Viz.,
q
21
= 2.
[There are three states: 0 (both salesmen are free), 1 (one busy, one free)
and 2 (both busy). The non-zero jump rates are
q
01
= , q
10
= , q
12
= , q
21
= 2,
with = 2, = 1. This leads to the Q-matrix
_
_
2 2 0
1 3 2
0 2 2
_
_
,
with q
(2)
00
= 6 and the eigen-values 0, 2, 5. Then
p
00
(t) = A + Be
2t
+ Ce
5t
, t 0,
where
A+ B + C = 1, 2B 5C = 2, 4B + 25C = 6.
Hence, A = 1/5, B = 2/3, C = 2/15, and
P
0
(both free at time t) =
1
5
+
2
3
e
2t
+
2
15
e
5t
,
5
P
0
(both busy at time t) =
2
5

2
3
e
2t
+
4
15
e
5t
.]
10. Let (X
t
)
t0
be a Markov chain on the integers with transition rates
q
ii+1
= q
i
, q
ii1
= q
i
, and q
i
= 0 if |j i| 2, where + = 1 and q
i
> 0
for all i. Find for all i
(a) The probability, starting from 0, that X
t
hits i. Hint: The answer
depends on the value of the /, viz., for /mu 1, h
i
0
=
_
(/)
i
, i 0,
1, i 0.
(b) The expected total time spent in state i, starting from 0. Answer:
h
i
0
__
q
i
| |
_
.
In the case = 0 (jumps to the right), write down a necessary and sucient
condition for (X
t
)
t0
to be explosive. Why is this condition necessary for
(X
t
)
t0
to be explosive for all [0,
1
2
)? Show that, in general, (X
t
)
t0
is
non-explosive if and only if one of the following conditions holds:
(i) = , (Hint: In this case the jump chain is recurrent.)
(ii) > and

i=1
1/q
i
= ,
(iii) < and
1

i=
1/q
i
= .
[If h
k
= P
k
(hit i) then, as usually, h
i
= 1 and (Qh)
k
= 0 for k = i. That
is,
q
k
h
k+1
+ q
k
h
k1
= q
k
h
k
, or (h
k1
h
k
) = (h
k
h
k1
), k = i.
Then, with h
+
= h
i+1
,
h
k
= h
i
+
ki

l=1
(h
i+l
h
i+l1
) = 1 + (h
+
1)
ki

l=1
_

_
l
, k i.
Similarly, with h

= h
i1
,
h
k
= h
i
+
ik

l=1
(h
il
h
il1
) = 1 + (h

1)
ik

l=1
_

_
l
, k i.
6
Parameters h

have to be xed so that the solution is minimal non-negative.

This yields
h
k
_

_
= (/)
ki
, k i, if ,
= (/)
ik
, k i, if ,
= 1, otherwise.
. . . . . . . . . . .
k i
.
.
. . . h
k
=
1
h
k
=

>

(
/
)
k
_
i
. . . . . . . . . . .
k i
. . .
.
.
h
k
=
1
>

h
)
i
_
k
/

(
=
k
. . . . . . . . . . .
k i
h
k
=
1

=
. . . . . . .
Substituting k = 0:
h
0
= P
0
(hit i) =
_

_
(/)
i
, i 0, if ,
(/)
i
, i 0, if ,
1, otherwise.
It useful to observe that k the return probability to a state k equals
P
k1
(hit k) + P
k+1
(hit k) = 2 min [, ].
Next, E
0
(time at i) = q
1
i
E
0
( # of visits to i), so introduce
E
k
= E
k
( # of visits to i) = P
k
(hit i)(1 +E
i
) = h
k
(1 +E
i
),
7
where, i,
E
i
= E
i
(# of visits to i)
=

n1
nP
i
(# of visits to i is n)
=

n0
P
i
(# of visits to i is n)
=

n0
(2 min [, ])
n
=
1
1 2 min [, ]
=
1
| |
.
So,
E
0
(time at i) =
h
0
q
i
| |
.
Now assume that , i.e., 0 1/2. If = 0, then (X
t
)
t0
is
explosive if and only if

i=1
1/q
i
< . If 0 < 1/2 then certainly the
jump chain Y
n
as n . Hence, (X
t
)
t0
cannot explode to , but
comparison with the case = 0 shows that if

i=1
1/q
i
= then (X
t
)
t0
cannot explode to either. To see that this condition is also necessary for
non-explosion when 0 < 1/2, observe that (X
t
)
t0
makes only nitely
many visits to states i 0 and
E
0
(total time at states i > 0) =

i=1
1/( )q
i
.
Finally, in the case = , (Y
n
)
n0
is recurrent so (X
t
)
t0
does not explode.
This covers cases (i) and (ii). Case (iii) is considered similarly.]
11. Let (X
t
)
t0
be a birth-and-death process with rates
n
= n and
n
=
n, and suppose X
0
= 1. Show that h(t) = P(X
t
= 0) satises
h(t) =
_
t
0
e
(+)s
{ + h(t s)
2
}ds
and deduce that if = then
h(t) = (e
t
e
t
)/(e
t
e
t
) .
Hint: If the population reaches level n then it will become extinct i
each of the n independent descendants trees terminates.
8
[A crucial point is that, with
n
= n and
n
= n, every member of the
population produces an ospring or dies independently. Therefore, the pop-
ulation with n members becomes extinct i the descendants trees generated
by each member terminates, which occurs independently. Thus we can write:
h(t) = P(X
t
= 0|X
0
= 1)
=
_
t
0
e
(+)s
( + )
_

+
+

+
P(X
t
= 0|X
s
= 2)
_
ds
=
_
t
0
e
(+)s
[ + P(X
t
= 0|X
s
= 2)]ds.
Now: P(X
t
= 0|X
s
= 2) = h(t s)
2
, and
h(t) =
_
t
0
e
(+)s
[ + h(t s)
2
)]ds
=

+
_
1 e
(+)t
_
+
_
t
0
e
(+)s
h(t s)
2
ds
=

+
_
1 e
(+)t
_
+
_
t
0
e
(+)(tu)
h(u)
2
du.
This leads to the equation
e
(+)t
h(t) =

+
_
1 e
(+)t
_
+
_
t
0
e
(+)u
h(u)
2
du,
or, after dierentiation:
( + )h + h

= + h
2
,
i.e.
dh
+ h
2
( + )h
= dt,
or
_

h

1
h 1
_
dh = dt( ),
whence
log
h
h 1
= ( )t + c,
or
h = A(h 1)e
()t
.
From h(0) = 0: A = , and
h( e
()t
) = e
()t
9
which immediately leads to the answer.]
12. Each bacterium in a colony splits into two identical bacteria after an
exponential time of parameter , which then split in the same way but inde-
pendently. Let X
t
denote the size of the colony at time t, and suppose X
0
= 1.
Show that the probability generating function (t) = E(z
Xt
) satises
(t) = ze
t
+
_
t
0
e
s
(t s)
2
ds
and deduce that, for q = 1 e
t
, and n = 1, 2, . . .,
P(X
t
= n) = q
n1
(1 q) .
Hint:Take 0 s t. Conditional on the rst split time J
1
= s, X
t
is
decomposed as X
ts
+

X
ts
.
[For 0 s t, the conditional distribution of X
t
, given the rst split J
1
occurs at s, is the same as the distribution of X
ts
+

X
ts
, where

X is an
independent copy of X. So
E(z
Xt
|J
1
= s) =
_
E(z
X
ts
+

X
ts
) for 0 s t,
z for s t,
and
(t) =
_

0
e
s
E(z
Xt
|J
1
= s)ds
= ze
t
+
_
t
0
e
s
(t s)
2
ds .
Put u = t s, multiply by e
t
and dierentiate to see that d/dt = (1 )
and so e
t
= z( 1)/(z 1). Then
(t) =

n=1
(1 q(t))q(t)
n1
z
n
where q = 1 e
t
.]
13. Compute p
11
(t) for P(t) = e
tQ
, where
Q =
_
_
2 1 1
4 4 0
2 1 3
_
_
.
10
Find an invariant distribution for Q and verify that p
11
(t)
1
as t .
[The eigenvalues of Q are 0, 4, 5 so p
11
(t) = A+Be
4t
+Ce
5t
for some
constants A, B, C which may be determined from p
11
(0) = 1, p

11
(0) = 2,
p

11
(0) = q
(2)
11
= 10. In fact, A =
3
5
=
1
, B = 0, C =
2
5
.]
14. Two eas are bound together to take part in a nine-legged race on the
vertices A, B, C of a triangle. Flea 1 hops at random times in the clockwise
direction; each hop takes the pair from one vertex to the next and the times
between successive hops of Flea 1 are independent random variables, each
with with exponential distribution, mean 1/. Flea 2 behaves similarly, but
hops in the anti-clockwise direction, the times between his hops having mean
1/. Show that the probability that they are at A at a given time t > 0 is
1
3
+
2
3
exp
_

3( + )t
2
_
cos
_

3( )t
2
_
.
[The position evolves as a Markov chain with Q-matrix
Q =
_
_



_
_
and the problem is to compute p
11
(t). The method of Example 1 applies.
Convergence to equilibrium gives p
11
() =
1
3
, which is useful in xing the
constants.]
11