3

Stochastic Control of Jump Diusions

3.1 Dynamic Programming
Fix a domain o R
k
(our solvency region) and let Y (t) = Y
(u)
(t) be a
stochastic process of the form
dY (t) = b(Y (t), u(t))dt +(Y (t), u(t))dB(t)
+
_
R
k
(Y (t

), u(t

), z)

N(dt, dz), Y (0) = y R
k
, (3.1.1)
where
b : R
k
U R
k
, : R
k
U R
km
, and : R
k
U R
k
R
k
are given functions, U R
p
is a given set. The process u(t) = u(t, ) :
[0, ) U is our control process, assumed to be c`adl` ag and adapted. We
call Y (t) = Y
(u)
(t) a controlled jump diusion.
We consider a performance criterion J = J
(u)
(y) of the form
J
(u)
(y) = E
y
__

S
0
f(Y (t), u(t))dt +g(Y (
S
)) A
{
S
<}
_
,
where

S
= inft > 0; Y
(u)
(t) , o (the bankruptcy time)
and f : o R and g : R
k
R are given continuous functions.
We say that the control process u is admissible and write u / if (3.1.1)
has a unique, strong solution Y (t) for all y o and
E
y
__

S
0
f

(Y (t), u(t))dt +g

(Y (
S
)) A
{
S
<}
_
< .
46 3 Stochastic Control of Jump Diusions
The stochastic control problem is to nd the value function (y) and an optimal
control u

/ dened by
(y) = sup
uA
J
(u)
(y) = J
(u

)
(y). (3.1.2)
It turns out that under mild conditions (see, e.g., [1, Theorem 11.2.3])
it suces to consider Markov controls, i.e., controls u(t) of the form
u(t) = u
0
(Y (t

))
for some function u
0
: R
k
U. Therefore, from now on we will only consider
Markov controls and we will, with a slight abuse of notation, write u(t) =
u(Y (t

)).
Note that if u = u(y) is a Markov control then Y (t) = Y
(u)
(t) is a Levy
diusion with generator
A(y) = A
u
(y) =
k

i=1
b
i
(y, u(y))

y
i
(y) +
1
2
k

i,j=1
(
T
)
ij
(y, u(y))

2

y
i
y
j
(y)
+

j=1
_
R
(y +
(j)
(y, u(y), z
j
)) (y)
(y)
(j)
(y, u(y), z
j
)
j
(dz
j
).
We now formulate a verication theorem for the optimal control prob-
lem (3.1.2), analogous to the classical HamiltonJacobiBellman (HJB) for
(continuous) It o diusions.
Theorem 3.1 (HJB for Optimal Control of Jump Diusions).
(a) Suppose C
2
(o) C(

o) satises the following:
(i) A
v
(y) +f(y, v) 0 for all y o, v U.
(ii) Y (
S
) o a.s. on
S
< and
lim
t

S
(Y (t)) = g(Y (
S
)) A
{
S
<}
a.s., for all u /.
(iii) E
y
_
[(Y ())[ +
_

S
0
[A(Y (t))[dt
_
< , for all u / and all T .
(iv)

(Y ())

S
is uniformly integrable for all u / and y o.
Then
(y) (y) for all y o. (3.1.3)
(b) Moreover, suppose that for each y o there exists v = u(y) U such that
(v) A
u(y)
(y) +f(y, u(y)) = 0
and
(vi) (Y
( u)
())

S
is uniformly integrable.
Suppose u

(t) := u(Y (t

)) /. Then u

is an optimal control and

(y) = (y) = J
(u

)
(y) for all y o. (3.1.4)
3.1 Dynamic Programming 47
Proof. (a) Let u /. For n = 1, 2, . . . put
n
= min(n,
S
). Then by the
Dynkin formula (Theorem 1.24) we have
E
y
[(Y (
n
))] = (y) + E
y
_
_

n
0
A
u
(Y (t))dt
_
(y) E
_
_

n
0
f(Y (t), u(t))dt
_
.
Hence
(y) liminf
n
E
y
_
_

n
0
f(Y (t), u(t))dt +(Y (
n
))
_
E
y
_
_

S
0
f(Y (t), u(t))dt +g(Y (
S
)) A
{
S
<}
_
= J
(u)
(y). (3.1.5)
Since u / was arbitrary we conclude that
(y) (y) for all y o. (3.1.6)
(b) Now apply the above argument to u(t) = u(Y (t)), where u is as in (v).
Then we get equality in (3.1.5) and hence
(y) = J
( u)
(y) (y) for all y o. (3.1.7)
Combining (3.1.6) and (3.1.7) we get (3.1.4). .
Example 3.2 (Optimal Consumption and Portfolio in a Levy Type
BlackScholes Market [Aa, FS1]).
Suppose we have a market with two possible investments:
(i) A safe investment (bond, bank account) with price dynamics
dP
1
(t) = rP
1
(t)dt, P
1
(0) = p
1
> 0.
(ii) A risky investment (stock) with price dynamics
dP
2
(t) = P
2
(t

)
_
dt + dB(t) +
_

1
z

N(dt, dz)
_
, P
2
(0) = p
2
> 0,
where r > 0, > 0, and R are constants. We assume that
_

1
[z[d(z) < and > r.
Assume that at any time t the investor can choose a consumption rate c(t) 0
(adapted and c` adl` ag) and is also free to transfer money from one investment
to the other without transaction cost. Let X
1
(t) and X
2
(t) be the amounts of
money invested in the bonds and the stocks, respectively. Let
48 3 Stochastic Control of Jump Diusions
(t) =
X
2
(t)
X
1
(t) +X
2
(t)
be the fraction of the total wealth invested in stocks at time t. Dene the
performance criterion by
J
(c,)
(s, x
1
, x
2
) = E
x
1
,x
2
__

0
e
(s+t)
c

(t)

dt
_
,
where > 0, (0, 1) are constants and E
x
1
,x
2
is the expectation w.r.t. the
probability law P
x
1
,x
2
of (X
1
(t), X
2
(t)) when X
1
(0) = x
1
, X
2
(0) = x
2
. Call
the control u(t) = (c(t), (t)) [0, ) [0, 1] admissible and write u / if
the corresponding total wealth
W(t) = W
(u)
(t) = X
(u)
1
(t) +X
(u)
2
(t)
is nonnegative for all t 0.
The problem is to nd (s, x
1
, x
2
) and u

(c

) / such that
(s, x
1
, x
2
) = sup
uA
J
(u)
(s, x
1
, x
2
) = J
(u

)
(s, x
1
, x
2
).
Case 1: = 0.
In this case the problem was solved by Merton [M]. He proved that if
>
_
r +
( r)
2
2
2
(1 )
_
, (3.1.8)
then the value function is

0
(s, x
1
, x
2
) = K
0
e
s
(x
1
+x
2
)

, (3.1.9)
where
K
0
=
1

_
1
1
_
r
( r)
2
2
2
(1 )
__
1
. (3.1.10)
Moreover, the optimal consumption rate c

0
(t) is given by
c

0
(t) = (K
0
)
1/(1)
(X
1
(t) +X
2
(t)) (3.1.11)
and the optimal portfolio

0
(t) is (the constant)

0
(t) =
r

2
(1 )
for all t [0, ). (3.1.12)
In other words, it is optimal to keep the state (X
1
(t), X
2
(t)) on the line
x
2
=

0
1

0
x
1
(3.1.13)
in the (x
1
, x
2
)-plane at all times (the Merton line). See Fig. 3.1.
3.1 Dynamic Programming 49
x
1
the Merton line ( = 0)
(x
1
, x
2
)
(X
1
(t), X
2
(t))
(x
1
, x
2
)
x
2
x
1
+ x
2
= 0
S
S
Fig. 3.1. The Merton line
Case 2: = 0
We now ask: How does the presence of jumps inuence the optimal strategy?
As in [M] we reduce the dimension by introducing
W(t) = X
1
(t) +X
2
(t).
Then we see that
dW(t) =([r(1 (t)) +(t)]W(t) c(t)) dt +(t)W(t)dB(t)
+(t)W(t

)
_

1
z

N(dt, dz), W(0) = x
1
+x
2
= w 0.
The generator A
(u)
of the controlled process
Y (t) =
_
s +t
W(t)
_
; t 0, Y (0) = y =
_
s
w
_
is
A
(u)
(y) =

s
+
_
[r(1 ) +]w c
_

w
+
1
2

2
w
2

2

w
2
+
_

1
_
(s, w +wz) (s, w)

w
(s, w)wz
_
(dz).
50 3 Stochastic Control of Jump Diusions
If we try
(y) = (s, w) = e
s
(w)
we get
A
(u)
(y) = e
s
A
(u)
0
(w), where
A
(u)
0
(w) = (w) + ([r(1 ) +] w c)

(w) +
1
2

2
w
2

(w)
+
_

1
((1 +z)w) (w)

(w)wz(dz).
In particular, if we try
(w) = Kw

we get
A
(u)
0
(w) +f(w, u) = Kw

+
_
[r(1 ) +]w c
_
Kw
1
+K
1
2

2
w
2
( 1)w
2
+Kw

_

1
(1 +z)

1 z(dz) +
c

.
Let h(c, ) be the expression on the right-hand side. Then h is concave in (c, )
and the maximum of h is attained at the critical points, i.e., when
h
c
= Kw
1
+c
1
= 0 (3.1.14)
and
h

=(r)Kw

+K
2
(1)w

+Kw

_

1
(1+z)
1
zz(dz)=0.
(3.1.15)
From (3.1.14) we get
c = c =
_
K
_
1/(1)
w (3.1.16)
and from (3.1.15) we get that =

should solve the equation
() := r
2
(1 )
_

1
_
1 (1 +z)
1
_
z(dz) = 0. (3.1.17)
Since (0) = r > 0 we see that if

2
(1 ) +
_

1
_
1 (1 +z)
1
_
z(dz) r (3.1.18)
then there exists an optimal =

(0, 1].
3.1 Dynamic Programming 51
With this choice of c = c =
_
K
_
1/(1)
w and =

(constant) we require
that
A
( u)
0
(w) +f(w, u) = 0, i.e.,
K +
_
[r(1

) +

]
_
K
_
1/(1)
_
K
+K
1
2

2
( 1) +K
_

1
(1 +

z)

z(dz)
+
_
K
_
/(1) 1

= 0
or
+[r(1

) +

] (K)
1/(1)

1
2

2
(1 ) +
_

1
(1 +

z)

z(dz) +K
1/(1)

/(1)

or
_
K
_
1/(1)
[1 ] = [r(1

) +

] +
1
2

2
(1 )

_

1
(1 +

z)

z(dz)
or
K =
1

_
1
1
_
r(1

) +

+
1
2

2
(1 )

_
R
(1 +

z)

z(dz)
__
1
. (3.1.19)
We now study condition (iii):
Here
T
(y) = e
s
wKw
1
= e
s
Kw

and
(Y (t) +(Y (t), u(t))) (Y (t)) = KW(t)

e
s
[(1 +z)

1].
So (iii) holds if
E
_
_
T
0
e
2t
W
2
(t)dt
_
+
_
R
[(1 +z)

1](dz) < . (3.1.20)

We refer to [FS1] for sucient conditions on the parameters for (3.1.20) to
hold.
We conclude that the value function is
(s, w) = (s, x
1
, x
2
) = e
s
K(x
1
+x
2
)

(3.1.21)
with optimal control u

(t) = (c

(t),

(t)) where c

= c = (K)
1/1
(x
1
+x
2
)
is given by (3.1.16) and

=

is given by (3.1.17), with K given by (3.1.19).
52 3 Stochastic Control of Jump Diusions
x
1
x
2
=

0
1

0
x
1
= 0 (classical Merton line)
> 0 (jump Merton line)
x
2
=

x
1
x
2
0
Fig. 3.2. The Merton line for = 0 and > 0
Finally we compare the solution in the jump case ( ,= 0) with Mertons
solution in the no jump case ( = 0).
As before let
0
, c

0
, and

0
be the solution when there are no jumps
( = 0). Then it can be seen that
K < K
0
and hence (s, w) = e
s
Kw

< e
s
K
0
w

=
0
(s, w)
c

(s, w) c

0
(s, w)

0
.
So with jumps it is optimal to place a smaller wealth fraction in the risky
investment, consume more relative to the current wealth and the resulting
value is smaller than in the no jump case. See Fig.3.2.
For more details we refer to [FS1].
Remark 3.3. For more information and other applications of stochastic con-
trol of jump diusions, see [GS, BKR1, BKR2, BKR3, BKR4, BKR5, BKR6,
Ma] and the references therein.
3.2 The Maximum Principle
Suppose the state X(t) = X
(u)
(t) of a controlled jump diusion in R
n
is given
by
3.2 The Maximum Principle 53
dX(t) = b(t, X(t), u(t))dt +(t, X(t), u(t))dB(t)
+
_
R

(t, X(t

), u(t

), z)

N(dt, dz). (3.2.1)

As before

N(dt, dz) = (

N
1
(dt, dz
1
), . . . ,

N

(dt, dz

))
T
, where

N
j
(dt, dz
j
) = N
j
(dt, dz
j
)
j
(dz
j
)dt, 1 j
(see the notation of Theorem 1.16).
The process u(t) = u(t, ) U R
k
is our control. We assume that u is
adapted and c` adl` ag, and that the corresponding (3.2.1) has a unique strong
solution X
(u)
(t), t [0, T]. Such controls are called admissible. The set of
admissible controls is denoted by /.
Suppose the performance criterion has the form
J(u) = E
_
_
T
0
f(t, X(t), u(t))dt +g(X(T))
_
, u /,
where f : [0, T] R
n
U R is continuous, g : R
n
R is C
1
, T < is a
xed deterministic time and
E
_
_
T
0
f

(t, X(t), u(t))dt +g

(X(T))
_
< for all u /.
Consider the problem to nd u

/ such that
J(u

) = sup
uA
J(u). (3.2.2)
In Chap. 2 we saw how to solve such a problem using dynamic program-
ming and the associated HJB equation. Here we present an alternative
approach, based on what is called the maximum principle. In the deter-
ministic case this principle was rst introduced by Pontryagin and his
group [PBGM]. A corresponding maximum principle for It o diusions was
formulated by Kushner [Ku], Bismut [Bi], and subsequently further devel-
oped by Bensoussan [Ben1, Ben2, Ben3], Haussmann [H1], and others. For
jump diusions a sucient maximum principle has recently been formulated
in [FS3] and it is this approach that is presented here, in a somewhat
simplied version.
Dene the Hamiltonian H : [0, T] R
n
U R
n
R
nm
R by
H(t, x, u, p, q, r) = f(t, x, u) +b
T
(t, x, u)p + tr(
T
(t, x, u)q)
+

j=1
n

i=1
_
R

ij
(t, x, u, z
j
)r
ij
(t, z)
j
(dz
j
), (3.2.3)
54 3 Stochastic Control of Jump Diusions
where is the set of functions r : R
+1
R
n
such that the integrals
in (3.2.3) converge. From now on we assume that H is dierentiable with
respect to x.
The adjoint equation (corresponding to u and X
(u)
) in the unknown
processes p(t) R
n
, q(t) R
nm
, and r(t, z) R
n
is the backward
stochastic dierential equation

dp(t) =
x
H(t, X(t), u(t), p(t), q(t), r(t, ))dt
+q(t)dB(t) +
_
R

r(t

, z)

N(dt, dz), t < T

p(T) = g(X(T)).
(3.2.4)
Theorem 3.4 (A Sucient Maximum Principle [FS3]). Let u /
with corresponding solution

X = X
( u)
and suppose there exists a solution
( p(t), q(t), r(t, z)) of the corresponding adjoint equation (3.2.4) satisfying
E
_
_
T
0
(

X(t) X
(u)
(t))
T
_
q q
T
(t)+
_
R

r r
T
(t, z)(dz)
_
(

X(t) X
(u)
(t))dt
_
<
(3.2.5)
and
E
_
_
T
0
p(t)
T
_

T
(t, X
(u)
(t), u(t))+
_
R

T
(t, X
(u)
(t), u(t), z)(dz)
_
p(t)dt
_
< for all u /.
Moreover, suppose that
H(t,

X(t), u(t), p(t), q(t), r(t, )) = sup
vU
H(t,

X(t), v, p(t), q(t), r(t, ))
for all t, that g(x) is a concave function of x and that

H(x) := max
vU
H(t, x, v, p(t), q(t), r(t, )) exists and is
a concave function of x, for all t [0, T] (the Arrow condition).
(3.2.6)
Then u is an optimal control.
Remark 3.5. For (3.2.6) to hold it suces that the function
(x, v) H(t, x, v, p(t), q(t), r(t, )) is concave, for all t [0, T]. (3.2.7)
To prove Theorem 3.4 we rst establish the following.
3.2 The Maximum Principle 55
Lemma 3.6 (Integration by Parts). Suppose E[(Y
(j)
(T)
2
] < for j =
1, 2, where
dY
(j)
(t) = b
(j)
(t, )dt +
(j)
(t, )dB(t) +
_
R

(j)
(t, z, )

N(dt, dz)
Y
(j)
(0) = y
(j)
R
n
, j = 1, 2
where b
(j)
R
n
,
(j)
R
nm
, and
(j)
R
n
. Then
E[Y
(1)
(T) Y
(2)
(T)]
= y
1
y
2
+E
_
_
T
0
Y
(1)
(t

)dY
(2)
(t)
+
_
T
0
Y
(2)
(t

)dY
(1)
(t) +
_
T
0
tr[
(1)
T

(2)
](t)dt
+
_
T
0
_

j=1
_
n

i=1
_
R

(1)
ij
(t, z
j
)
(2)
ij
(t, x)
_

j
(dz
j
)
_
dt
_
.
Proof. This follows from the It o formula (Theorem 1.16). (See also
Exercise 1.7.) .
Proof of Theorem 3.4. Let u / be an admissible control with corresponding
state process X(t) = X
(u)
(t). Then
J( u)J(u)=E
_
_
T
0
f(t,

X(t), u(t))f(t, X(t), u(t))dt+g(

X(T))g(X(T))
_
.
Since g is concave we get by Lemma 3.6
E[g(

X(T)) g(X(T))] E[(

X(T) X(T))
T
g(

X(T))]
= E[(

X(T) X(T))
T
p(T)]
= E
_
_
T
0
(

X(t

) X(t

))
T
d p(t)
+
_
T
0
p(t

)
T
(d

X(t) dX(t))
+
_
T
0
tr
_
{(t,

X(t), u(t)) (t, X(t), u(t))}
T
q(t)
_
dt
+
_
T
0
_

j=1
_
n

i=1
_
R
{
ij
(t,

X(t), u(t), z
j
)

ij
(t, X(t), u(t), z
j
)} r
ij
(t, z
j
)
_

j
(dz
j
)
_
dt
_
56 3 Stochastic Control of Jump Diusions
= E
_
_
T
0
(

X(t) X(t))
T

x
H(t,

X(t), u(t), p(t), q(t), r(t, ))
_
dt
+
_
T
0
p
T
(t

){b(t,

X(t), u(t)) b(t, X(t), u(t))}dt
+
_
T
0
tr[{(t,

X(t), u(t)) (t, X(t), u(t))}
T
q(t)]dt
+
_
T
0
_

j=1
_
n

i=1
_
R
{
ij
(t,

X(t), u(t), z
j
)

ij
(t, X(t), u(t), z
j
)} r
ij
(t, z
j
)
_

j
(dz
j
)
_
dt
_
. (3.2.8)
By the denition of H we nd
E
_
_
T
0
f(t,

X(t), u(t)) f(t, X(t), u(t))dt
_
= E
_
_
T
0
H(t,

X(t), u(t), p(t), q(t), r(t, ))
H(t, X(t), u(t), p(t), q(t), r(t, ))dt

_
T
0
b(t,

X(t), u(t)) b(t, X(t), u(t))
T
p(t)dt

_
T
0
tr[(t,

X(t), u(t)) (t, X(t), u(t))
T
q(t)]dt

_
T
0
_

j=1
_
n

i=1
_
R

ij
(t,

X(t), u(t), z
j
)

ij
(t, X(t), u(t), z
j
) r
ij
(t, z
j
)
_

j
(dz
j
)
_
dt
_
. (3.2.9)
Adding (3.2.8) and (3.2.9) we get
J( u) J(u) E
_
_
T
0
H(t,

X(t), u(t), p(t), q(t), r(t, ))
H(t, X(t), u(t), p(t), q(t), r(t, ))
(

X(t)X(t))
T

x
H(t,

X(t), u(t), p(t), q(t), r(t, ))dt
_
.
If (3.2.6) (or (3.2.7)) holds then J( u) J(u) 0. This follows from the
proof in [SeSy, p. 108]. For details we refer to [FS3]. .
3.3 Application to Finance 57
We mention briey the relation to dynamic programming. Dene
J
(u)
(s, x) = E
_
_
Ts
0
f(s +t, X
x
(t), u(t))dt +g(X
x
(T s))
_
, u /,
where X
x
(t) is the solution of (3.2.1) for t 0 with initial value X(0) = x.
Then put
V (s, x) = sup
uA
J
(u)
(s, x). (3.2.10)
Theorem 3.7 ([FS3]). Assume that V (s, x) C
1,3
(RR
n
) and that there
exists an optimal Markov control u

(t, x) for problem (3.2.2), with correspond-

ing solution X

(t) of (3.2.1). Dene

p
i
(t) =
V
x
i
(t, X

(t)), 1 i n,
q
jk
(t) =
n

i=1

ik
(t, X

(t), u

(t))

2
V
x
i
x
j
(t, X

(t)), 1 j n, 1 k m,
r
ik
(t, z) =
V
x
i
(t, X

(t) +
(k)
(t, X

(t), u

(t), z
k
))
V
x
i
(t, X

(t)),
1 i n, 1 k .
Then p(t), q(t), r(t, ) solve the adjoint equation (3.2.4).
For a proof see [FS3].
Remark 3.8. A general discussion of impulse control for jump diusions can
be found in [F]. A study with vanishing impulse costs is given in [UZ].
3.3 Application to Finance
The following example is from [FS3].
Consider a nancial market with two investment possibilities, a risk
free (e.g., a bond or bank account) and risky (e.g., a stock), whose prices
S
0
(t), S
1
(t) at time t [0, T] are given by
(bond) dS
0
(t) =
t
S
0
(t)dt, S
0
(0) = 1, (3.3.1)
(stock) dS
1
(t) = S
1
(t

)
_

t
dt +
t
dB(t) +
_
R
(t, z)

N(dt, dz)
_
, S
1
(0)>0,
(3.3.2)
where
t
> 0,
t
,
t
, and (t, z) 1 are given bounded deterministic func-
tions. We assume that the function
58 3 Stochastic Control of Jump Diusions
t
_
R

2
(t, z)(dz) is locally bounded. (3.3.3)
We may regard this market as a jump diusion extension of the classical
BlackScholes market (see Sect. 1.5).
A portfolio in this market is a two-dimensional c` adl` ag and adapted process
(t) = (
0
(t),
1
(t)) giving the number of units of bonds and stocks, respec-
tively, held at time t by an agent.
The corresponding wealth process X(t) = X
()
(t) is dened by
X(t) =
0
(t)S
0
(t) +
1
(t)S
1
(t), t [0, T]. (3.3.4)
The portfolio is called self-nancing if
X(t) = X(0) +
_
t
0

0
(s)dS
0
(s) +
_
t
0

1
(s)dS
1
(s) (3.3.5)
or, in shorthand notation,
dX(t) =
0
(t)dS
0
(t) +
1
(t)dS
1
(t). (3.3.6)
Alternatively, the portfolio can also be expressed in terms of the amounts
w
0
(t) and w
1
(t) invested in the bond and stock, respectively. They are given
by
w
i
(t) =
i
(t)S
i
(t), i = 0, 1. (3.3.7)
Now put
u(t) = w
1
(t). (3.3.8)
Then w
0
(t) = X(t) u(t) and (3.3.6) gets the form
dX(t) = [
t
X(t) + (
t

t
)u(t)]dt +
t
u(t)dB(t) +u(t

)
_
R
(t, z)

N(dt, dz).
(3.3.9)
We call u(t) admissible and write u(t) / if (3.3.9) has a unique solution
X(t) = X
(u)
(t) such that E[(X
(u)
(T))
2
] < .
The meanvariance portfolio selection problem is to nd u(t) which mini-
mizes
Var[X(T)] := E
_
(X(T) E[X(T)])
2

(3.3.10)
under the condition that
E[X(T)] = A, a given constant. (3.3.11)
By the Lagrange multiplier method the problem can be reduced to minimizing,
for a given constant a R,
E[(X(T) a)
2
]
3.3 Application to Finance 59
without constraints. To see this, consider
E[(X(T) A)
2
(E[X(T)] A)]
= E
_
X
2
(T) 2
_
A+

2
_
X(T) +A
2
+A
_
= E
_
_
X(T)
_
A+

2
__
2
_

2
4
, where R is constant.
We will consider the equivalent problem
sup
uA
E
_

1
2
(X
(u)
(T) a)
2
_
. (3.3.12)
In this case the Hamiltonian (3.2.3) gets the form
H(t, x, u, p, q, r) =
t
x + (
t

t
)up +
t
uq +u
_
R
(t, z)r(t, z)(dz).
Hence the adjoint equations (3.2.4) are

dp(t) =
t
p(t)dt +q(t)dB(t) +
_
R
r(t

, z)

N(dt, dz), t < T

p(T) = (X(T) a).
(3.3.13)
We try a solution of the form
p(t) =
t
X(t) +
t
, (3.3.14)
where
t
,
t
are deterministic C
1
functions. Substituting in (3.3.13) and using
(3.3.9) we get
dp(t) =
t
_

t
X(t) + (
t

t
)u(t)dt +
t
u(t)dB(t)
+u(t

)
_
R
(t, z)

N(dt, dz)
_
+X(t)

t
dt +

t
dt
= [
t

t
X(t) +
t
(
t

t
)u(t) +X(t)

t
+

t
]dt
+
t

t
u(t)dB(t) +
t
u(t

)
_
R
(t, z)

N(dt, dz). (3.3.15)

Comparing with (3.3.13) we get

t
X(t) +
t
(
t

t
)u(t) +X(t)

t
+

t
=
t
(
t
X(t) +
t
), (3.3.16)
q(t) =
t

t
u(t), (3.3.17)
r(t, z) =
t
u(t)(t, z). (3.3.18)
60 3 Stochastic Control of Jump Diusions
Let u / be a candidate for the optimal control with corresponding

X and
p, q, r. Then
H(t,

X(t), u, p(t), q(t), r(t, ))
=
t

X(t) p(t) +u
_
(
t

t
) p(t) +
t
q(t) +
_
R
(t, z) r(t, z)(dz)
_
.
Since this is a linear expression in u, it is natural to guess that the coecient
of u vanishes, i.e.,
(
t

t
) p(t) +
t
q(t) +
_
R
(t, z) r(t, z)(dz) = 0. (3.3.19)
Using that by (3.3.17) and (3.3.18) we have
q(t) =
t

t
u(t), r(t, z) =
t
u(t)(t, z)
we get from (3.3.19) that
u(t) =
(
t

t
) p(t)

t
=
(
t

t
)(
t

X(t) +
t
)

t
, (3.3.20)
where

t
=
2
t
+
_
R

2
(t, z)(dz). (3.3.21)
On the other hand, from (3.3.16) we have
u(t) =
(
t

t
+

t
)

X(t) +
t
(
t

X(t) +
t
) +

t
(
t

t
)
. (3.3.22)
Combining (3.3.20) and (3.3.22) we get the equations
(
t

t
)
2

t
[2
t

t
+

t
]
t
= 0,
T
= 1,
(
t

t
)
2

t
[
t

t
+

t
]
t
= 0,
T
= a,
which have the solutions

t
= exp
_
_
T
t
_
(
s

s
)
2

s
2
s
_
ds
_
, 0 t T, (3.3.23)

t
= a exp
_
_
T
t
_
(
s

s
)
2

s
_
ds
_
, 0 t T. (3.3.24)
With this choice of
t
and
t
the processes
p(t) :=
t

X(t) +
t
, q(t) :=
t

t
u(t), and r(t, z) :=
t
u(t)(t, z)
3.4 Exercises 61
solve the adjoint equation, and by (3.3.19) we see that all the conditions of the
sucient maximum principle (Theorem 3.4) are satised. We conclude that
u(t) given by (3.3.20) is an optimal control. In feedback form the control can
be written
u(t, x) =
(
t

t
)(
t
x +
t
)

t
. (3.3.25)
3.4 Exercises
Exercise* 3.1. Suppose the wealth X(t) = X
(u)
(t) of a person with con-
sumption rate u(t) 0 satises the following Levy type mean reverting
OrnsteinUhlenbeck SDE
dX(t) = ( X(t) u(t))dt + dB(t) +
_
R
z

N(dt, dz), t > 0,
X(0) = x > 0.
Fix T > 0 and dene
J
(u)
(s, x) = E
s,x
_
_
Ts
0
e
(s+t)
u

(t)

dt +X(T s)
_
.
Use dynamic programming to nd the value function (s, x) and the optimal
consumption rate (control) u

(t) such that

(s, x) = sup
u()
J
(u)
(s, x) = J
(u

)
(s, x).
In the above , , , , T, > 0, (0, 1), and > 0 are constants.
Exercise* 3.2. Solve the problem of Exercise 3.1 by using the stochastic
maximum principle.
Exercise* 3.3. Dene
dX
(u)
(t) = dX(t) =
_
dX
1
(t)
dX
2
(t)
_
=

u(t, )
_
R
z

N(dt, dz)
_
R
z
2

N(dt, dz)

R
2
and, for xed T > 0 (deterministic)
J(u) = E
_
(X
1
(T) X
2
(T))
2

.
Use the stochastic maximum principle to nd u

such that
J(u

) = sup
u
J(u).
62 3 Stochastic Control of Jump Diusions
Interpretation. Put F() =
_
R
z
2

N(T, dz). We may regard F as a given
T-claim in the normalized market with the two investment possibilities bond
and stock, whose prices are
(bond) dS
0
(t) = 0, S
0
(0) = 1,
(stock) dS
1
(t) =
_
R
z

N(dt, dz), a Levy martingale.
Then J(u) is the variance of the dierence between F = X
2
(T) and the
wealth X
1
(T) generated by a self-nancing portfolio u(t, ). See [BDLP]
for more information on minimal variance hedging in markets driven by Levy
martingales.
Exercise* 3.4. Solve the stochastic control problem

1
(s, x) = inf
u0
E
s,x
__

0
e
(s+t)
(X
2
(t) +u
2
(t))dt
_
,
where
dX(t) = u(t)dt +dB(t) +
_
R
z

N(dt, dz), X(0) = x,
where > 0, > 0, and > 0 are constants.
The interpretation of this problem is that we want to push the process
X(t) as close as possible to 0 by using a minimum of energy, its rate being
measured by u
2
(t).
[Hint: Try (s, x) = e
s
(ax
2
+b) for some constants a, b.]
Exercise* 3.5 (The Stochastic Linear Regulator Problem).
Solve the stochastic control problem

0
(x) = inf
u
E
x
_
_
T
0
(X
2
(t) +u(t)
2
)dt +X
2
(T)
_
,
where
dX(t) = u(t)dt +dB(t) +
_
R
z

N(dt, dz), X(0) = x
and
T > 0 is a constant.
(a) By using dynamic programming (Theorem 3.1).
(b) By using the stochastic maximum principle (Theorem 3.4).
Exercise* 3.6. Solve the stochastic control problem
(s, x) = sup
c(t)0
E
s,x
__

0
0
e
(s+t)
ln c(t)dt
_
,
3.4 Exercises 63
where the supremum is taken over all T
t
-adapted processes c(t) 0 and

0
= inft > 0; X(t) 0,
where
dX(t) = X(t

)
_
dt + dB(t) +
_
R
z

N(dt, dz)
_
c(t)dt, X(0) = x > 0,
where > 0, , , and are constants, and
z > 1 for a.a. z w.r.t. .
We may interpret c(t) as the consumption rate, X(t) as the correspond-
ing wealth, and
0
as the bankruptcy time. Thus represents the maximal
expected total discounted logarithmic utility of the consumption up to bank-
ruptcy time.
[Hint: Try (s, x) = e
s
(a ln x +b) as a candidate for (s, x), where a and b
are suitable constants.]
Exercise 3.7. Use the stochastic maximum principle (Theorem 3.4) to solve
the problem
sup
c(t)0
E
_
_
T
0
e
t
ln c(t)dt +e
T
ln X(T)
_
,
where
dX(t) = X(t

)
_
dt + dB(t) +
_
R
z

N(dt, dz)
_
c(t)dt, X(0) > 0.
Here > 0, > 0, , , and are constants, and
z > 1 for a.a. z().
(see Exercise 3.6 for an interpretation of this problem).
[Hint: Try p(t) = ae
t
X
1
(t) and c(t) = X(t)/a, for some constant a > 0.]