Master LN

STOCHASTIC CONTROL,
AND APPLICATION TO FINANCE
Nizar Touzi
nizar.touzi@polytechnique.edu
Ecole Polytechnique Paris

Département de Mathématiques Appliquées
This version: 10 November 2018

2
Contents
1 Conditional Expectation and Linear Parabolic PDEs 5

1.1 Stochastic differential equations with random coefficients . . . . . 5
1.2 Markov solutions of SDEs . . . . . . . . . . . . . . . . . . . . . . 10
1.3 Connection with linear partial differential equations . . . . . . . 10
1.3.1 Generator . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.3.2 Cauchy problem and the Feynman-Kac representation . . 11
1.3.3 Representation of the Dirichlet problem . . . . . . . . . . 13
1.4 The stochastic control approach to the Black-Scholes model . . . 14
1.4.1 The continuous-time financial market . . . . . . . . . . . 14
1.4.2 Portfolio and wealth process . . . . . . . . . . . . . . . . . 15
1.4.3 Admissible portfolios and no-arbitrage . . . . . . . . . . . 16
1.4.4 Super-hedging and no-arbitrage bounds . . . . . . . . . . 17
1.4.5 The no-arbitrage valuation formula . . . . . . . . . . . . . 18
1.4.6 PDE characterization of the Black-Scholes price . . . . . . 18
2 Stochastic Control and Dynamic Programming 21

2.1 Stochastic control problems in standard form . . . . . . . . . . . 21
2.2 The dynamic programming principle . . . . . . . . . . . . . . . . 24
2.2.1 A weak dynamic programming principle . . . . . . . . . . 24
2.2.2 Dynamic programming without measurable selection . . . 26
2.3 The dynamic programming equation . . . . . . . . . . . . . . . . 29
2.4 On the regularity of the value function . . . . . . . . . . . . . . . 32
2.4.1 Continuity of the value function for bounded controls . . 32
2.4.2 A deterministic control problem with non-smooth value
function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.4.3 A stochastic control problem with non-smooth value func-
tion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3 Optimal Stopping and Dynamic Programming 37

3.1 Optimal stopping problems . . . . . . . . . . . . . . . . . . . . . 37
3.2 The dynamic programming principle . . . . . . . . . . . . . . . . 39
3.3 The dynamic programming equation . . . . . . . . . . . . . . . . 40
3.4 Regularity of the value function . . . . . . . . . . . . . . . . . . . 42
3.4.1 Finite horizon optimal stopping . . . . . . . . . . . . . . . 42
3
4
3.4.2 Infinite horizon optimal stopping . . . . . . . . . . . . . . 43

3.4.3 An optimal stopping problem with nonsmooth value . . . 46
4 Solving Control Problems by Verification 49

4.1 The verification argument for stochastic control problems . . . . 49
4.2 Examples of control problems with explicit solutions . . . . . . . 52
4.2.1 Optimal portfolio allocation . . . . . . . . . . . . . . . . . 52
4.2.2 Law of iterated logarithm for double stochastic integrals . 54
4.3 The verification argument for optimal stopping problems . . . . . 57
4.4 Examples of optimal stopping problems with explicit solutions . 59
4.4.1 Perpetual American options . . . . . . . . . . . . . . . . . 59
4.4.2 Finite horizon American options . . . . . . . . . . . . . . 60
5 Introduction to Viscosity Solutions 63

5.1 Intuition behind viscosity solutions . . . . . . . . . . . . . . . . . 63
5.2 Definition of viscosity solutions . . . . . . . . . . . . . . . . . . . 64
5.3 First properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.3.1 Change of variable / function . . . . . . . . . . . . . . . . 65
5.3.2 Stability of viscosity solutions . . . . . . . . . . . . . . . . 66
5.3.3 Parameter variables . . . . . . . . . . . . . . . . . . . . . 68
5.4 Comparison result and uniqueness . . . . . . . . . . . . . . . . . 69
5.4.1 Comparison of classical solutions in a bounded domain . . 70
5.4.2 Comparison of viscosity solutions of first order equations . 70
5.4.3 Semijets definition of viscosity solutions . . . . . . . . . . 72
5.4.4 The Crandall-Ishii’s lemma . . . . . . . . . . . . . . . . . 73
5.4.5 Comparison of viscosity solutions in a bounded domain . 77
5.5 Comparison in unbounded domains . . . . . . . . . . . . . . . . . 79
5.6 Useful applications . . . . . . . . . . . . . . . . . . . . . . . . . . 82
5.7 Proof of the Crandall-Ishii’s lemma . . . . . . . . . . . . . . . . . 84
6 Dynamic Programming Equation in Viscosity Sense 89

6.1 DPE for stochastic control problems . . . . . . . . . . . . . . . . 89
6.2 DPE for optimal stopping problems . . . . . . . . . . . . . . . . 95
6.3 A comparison result for obstacle problems . . . . . . . . . . . . . 97
7 Stochastic Target Problems 99

7.1 Stochastic target problems . . . . . . . . . . . . . . . . . . . . . . 99
7.1.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 99
7.1.2 Geometric dynamic programming principle . . . . . . . . 100
7.1.3 The dynamic programming equation . . . . . . . . . . . . 102
7.1.4 Application: hedging under portfolio constraints . . . . . 106
7.2 Stochastic target problem with controlled probability of success . 109
7.2.1 Reduction to a stochastic target problem . . . . . . . . . 109
7.2.2 The dynamic programming equation . . . . . . . . . . . . 110
7.2.3 Application: quantile hedging in the Black-Scholes model 111
5
8 Backward SDEs and Stochastic Control 117

8.1 Motivation and examples . . . . . . . . . . . . . . . . . . . . . . 117
8.1.1 The stochastic Pontryagin maximum principle . . . . . . 118
8.1.2 BSDEs and stochastic target problems . . . . . . . . . . . 120
8.1.3 BSDEs and finance . . . . . . . . . . . . . . . . . . . . . . 120
8.2 Wellposedness of BSDEs . . . . . . . . . . . . . . . . . . . . . . . 121
8.2.1 Martingale representation for zero generator . . . . . . . . 122
8.2.2 BSDEs with affine generator . . . . . . . . . . . . . . . . 123
8.2.3 The main existence and uniqueness result . . . . . . . . . 124
8.3 Comparison and stability . . . . . . . . . . . . . . . . . . . . . . 126
8.4 BSDEs and stochastic control . . . . . . . . . . . . . . . . . . . . 128
8.5 BSDEs and semilinear PDEs . . . . . . . . . . . . . . . . . . . . 130
8.6 Appendix: essential supremum . . . . . . . . . . . . . . . . . . . 131
6
Chapter 1
Conditional Expectation
and Linear Parabolic
PDEs
Throughout this chapter, (Ω, F, F, P ) is a filtered probability space with filtra-

tion F = {Ft , t ≥ 0} satisfying the usual conditions. Let W = {Wt , t ≥ 0} be
a Brownian motion valued in Rd , defined on (Ω, F, F, P ).
Throughout this chapter, a maturity T > 0 will be fixed. By H2 , we denote
the collection of all progressively
hR measurble
i processes φ with appropriate (finite)
T 2
dimension such that E 0 |φt | dt < ∞.
1.1 Stochastic differential equations with ran-

dom coefficients
In this section, we recall the basic tools from stochastic differential equations
dXt = bt (Xt )dt + σt (Xt )dWt , t ∈ [0, T ], (1.1)
where T > 0 is a given maturity date. Here, b and σ are F⊗B(Rn )-progressively
measurable functions from [0, T ] × Ω × Rn to Rn and MR (n, d), respectively.
In particular, for every fixed x ∈ Rn , the processes {bt (x), σt (x), t ∈ [0, T ]} are
F−progressively measurable.
Definition 1.1. A strong solution of (1.1) is an F−progressively measurable

RT
process X such that 0 (|bt (Xt )| + |σt (Xt )|2 )dt < ∞, a.s. and
Z t Z t
Xt = X0 + bs (Xs )ds + σs (Xs )dWs , t ∈ [0, T ].
0 0
7
8 CHAPTER 1. CONDITIONAL EXPECTATION AND LINEAR PDEs
Let us mention that there is a notion of weak solutions which relaxes some
conditions from the above definition in order to allow for more general stochas-
tic differential equations. Weak solutions, as opposed to strong solutions, are
defined on some probabilistic structure (which becomes part of the solution),
and not necessarily on (Ω, F, F, P, W ). Thus, for a weak solution we search for a
probability structure (Ω̃, F̃, F̃, P̃, W̃ ) and a process X̃ such that the requirement
of the above definition holds true. Obviously, any strong solution is a weak
solution, but the opposite claim is false.
The main existence and uniqueness result is the following.
Theorem 1.2. Let X0 ∈ L2 be a r.v. independent of W . Assume that the
processes b. (0) and σ. (0) are in H2 , and that for some K > 0:
|bt (x) − bt (y)| + |σt (x) − σt (y)| ≤ K|x − y| for all t ∈ [0, T ], x, y ∈ Rn .
Then, for all T > 0, there exists a unique strong solution of (1.1) in H2 . More-
over,

2
≤ C 1 + E|X0 |2 eCT ,

E sup |Xt | (1.2)
t≤T
for some constant C = C(T, K) depending on T and K.

Proof. We first establish the existence and uniqueness result, then we prove the
estimate (1.2).
Step 1 For a constant c > 0, to be fixed later, we introduce the norm
"Z #1/2
T
−ct 2
kφkH2c := E e |φt | dt for every φ ∈ H2 .
0
Clearly , the norms k.kH2 and k.kH2c on the Hilbert space H2 are equivalent.
Consider the map U on H2 by:
Z t Z t
U (X)t := X0 + bs (Xs )ds + σs (Xs )dWs , 0 ≤ t ≤ T.
0 0
By the Lipschitz property of b and σ in the x−variable and the fact that
b. (0), σ. (0) ∈ H2 , it follows that this map is well defined on H2 . In order
to prove existence and uniqueness of a solution for (1.1), we shall prove that
U (X) ∈ H2 for all X ∈ H2 and that U is a contracting mapping with respect to
the norm k.kH2c for a convenient choice of the constant c > 0.
1- We first prove that U (X) ∈ H2 for all X ∈ H2 . To see this, we decompose:
" Z Z
T t 2 #
2 2

kU (X)kH2 ≤ 3T kX0 kL2 + 3T E
bs (Xs )ds dt
0 0
" Z Z
T t 2 #

+3E
σs (Xs )dWs dt
0 0
1.1. Stochastic differential equations 9
By the Lipschitz-continuity of b and σ in x, uniformly in t, we have |bt (x)|2 ≤

K(1 + |bt (0)|2 + |x|2 ) for some constant K. We then estimate the second term
by:
"Z
T Z t 2 # "Z
T
#
2 2

E
bs (Xs )ds dt ≤ KT E (1 + |bt (0)| + |Xs | )ds < ∞,
0 0 0
since X ∈ H2 , and b(., 0) ∈ L2 ([0, T ]).

As, for the third term, we use the Doob maximal inequality together with
the fact that |σt (x)|2 ≤ K(1 + |σt (0)|2 + |x|2 ), a consequence of the Lipschitz
property on σ:
"Z
T Z t 2 # " Z t 2 #

E σs (Xs )dWs dt ≤ T E max σs (Xs )dWs dt
0

0 t≤T 0
"Z #
T
≤ 4T E |σs (Xs )|2 ds
0
"Z #
T
2 2
≤ 4T KE (1 + |σs (0)| + |Xs | )ds < ∞.
0
2- To see that U is a contracting mapping for the norm k.kH2c , for some convenient
choice of c > 0, we consider two process X, Y ∈ H2 with X0 = Y0 , and we
estimate that:
2
E |U (X)t − U (Y )t |
Z t 2 Z t 2

≤ 2E
(bs (Xs ) − bs (Ys )) ds + 2E
(σs (Xs ) − σs (Ys )) dWs
0 0
Z t 2 Z t
2
= 2E (bs (Xs ) − bs (Ys )) ds + 2E |σs (Xs ) − σs (Ys )| ds
0 0
Z t Z t
2 2
= 2tE |bs (Xs ) − bs (Ys )| ds + 2E |σs (Xs ) − σs (Ys )| ds
0 0
Z t
2
≤ 2(T + 1)K E |Xs − Ys | ds.
0
2K(T + 1)
Hence, kU (X) − U (Y )kc ≤ kX − Y kc , and therefore U is a contract-
c
ing mapping for sufficiently large c.
Step 2 We next prove the estimate (1.2). We shall alleviate the notation writ-
ing bs := bs (Xs ) and σs := σs (Xs ). We directly estimate:

" Z u Z u 2 #
2

E sup |Xu | = E sup X0 +
bs ds + σs dWs
u≤t u≤t 0 0
Z t " Z u 2 #!
2 2

≤ 3 E|X0 | + tE |bs | ds + E sup σs dWs
0 u≤t 0
Z t Z t
≤ 3 E|X0 |2 + tE |bs |2 ds + 4E |σs |2 ds
0 0
where we used the Doob’s maximal inequality. Since b and σ are Lipschitz-
continuous in x, uniformly in t and ω, this provides:
Z t
2 2 2
E sup |Xu | ≤ C(K, T ) 1 + E|X0 | + E sup |Xu | ds
u≤t 0 u≤s
and we conclude by using the Gronwall lemma. ♦
The following exercise shows that the Lipschitz-continuity condition on the

coefficients b and σ can be relaxed. We observe that further relaxation of this
assumption is possible in the one-dimensional case, see e.g. Karatzas and Shreve
[8].
Exercise 1.3. In the context of this section, assume that the coefficients µ
and σ are locally Lipschitz and linearly growing in x, uniformly in (t, ω). By a
localization argument, prove that strong existence and uniqueness holds for the
stochastic differential equation (1.1).
In addition to the estimate (1.2) of Theorem 1.2, we have the following flow
continuity results of the solution of the SDE.
Theorem 1.4. Let the conditions of Theorem 1.2 hold true, and consider some
(t, x) ∈ [0, T ) × Rn with t ≤ t0 ≤ T .
(i) There is a constant C such that:

t,x0 2 0
≤ CeCt |x − x0 |2 .
t,x
E sup Xs − Xs |
(1.3)
t≤s≤t0
R t0
(ii) Assume further that B := supt<t0 ≤T (t0 − t)−1 E t |br (0)|2 + |σr (0)|2 dr <

∞. Then for all t0 ∈ [t, T ]:

0
E sup Xst,x − Xst ,x |2 ≤ CeCT (B + |x|2 )|t0 − t|.

(1.4)
t0 ≤s≤T
0
Proof. (i) To simplify the notations, we set Xs := Xst,x and Xs0 := Xst,x for all
s ∈ [t, T ]. We also denote δx := x − x0 , δX := X − X 0 , δb := b(X) − b(X 0 ) and
1.1. Stochastic differential equations 11
δσ := σ(X) − σ(X 0 ). We first decompose:

Z s 2 Z s 2
2 2
|δXs | ≤ 3 |δx| + δbu du + δσu dWu

t t
Z s Z s 2
2
2
≤ 3 |δx| + (s − t) δbu du + δσu dWu .

t t
Then, it follows from the Doob maximal inequality and the Lipschitz property
of the coefficients b and σ that:
Z s Z s
0 2 2
2 2
h(t ) := E sup |δXs | ≤ 3 |δx| + (s − t) E δbu du + 4
E δσu du

t≤s≤t0 t t
Z s
≤ 3 |δx|2 + K 2 (t0 + 4) E|δXu |2 du
t
Z s
≤ 3 |δx|2 + K 2 (t0 + 4) h(u)du .
t
Then the required estimate follows from the Gronwall inequality.

2. We next prove (1.4). We again simplify the notation by setting Xs := Xst,x ,
0
s ∈ [t, T ], and Xs0 := Xst ,x , s ∈ [t0 , T ]. We also denote δt := t0 −t, δX := X −X 0 ,
δb := b(X)−b(X 0 ) and δσ := σ(X)−σ(X 0 ). Then following the same arguments
as in the previous step, we obtain for all u ∈ [t0 , T ]:
Z u
2 2 2 2
h(u) := E sup |δXs | ≤ 3 E|Xt0 − x| + K (T + 4) E|δXr | dr
t0 ≤s≤u t0
Z u
≤ 3 E|Xt0 − x|2 + K 2 (T + 4) h(r)dr (1.5)
t0
Observe that
t0 t0
!
Z 2 Z 2
2
E|Xt0 − x| ≤ 2 E br (Xr )dr + E σr (Xr )dr

t t
0
t0
!
Z t Z
2 2
≤ 2 T E|br (Xr )| dr + E|σr (Xr )| dr
t t
Z t0
K 2 E|Xr − x|2 + |x|2 + E|br (0)|2 dr

≤ 6(T + 1)
t
Z t0
≤ 6(T + 1) (t0 − t)(|x|2 + B) + K 2 E|Xr − x|2 dr .
t
By the Gronwall inequality, this shows that
0
E|Xt0 − x|2 ≤ C(|x|2 + B)|t0 − t|eC(t −t) .
Plugging this estimate in (1.5), we see that:
Z u
2 0 C 0 2
h(u) ≤ 3 C(|x| + B)|t − t|e (t − t) + K (T + 4) h(r)dr , (1.6)
t0
and the required estimate follows from the Gronwall inequality. ♦

1.2 Markov solutions of SDEs

In this section, we restrict the coefficients b and σ to be deterministic functions
of (t, x). In this context, we write
bt (x) = b(t, x), σt (x) = σ(t, x) for t ∈ [0, T ], x ∈ Rn ,
where b and σ are continuous functions, Lipschitz in x uniformly in t. Let X.t,x
denote the solution of the stochastic differential equation
Z s Z s
Xst,x = x + b u, Xut,x du + σ u, Xut,x dWu s ≥ t

t t
The two following properties are obvious:

• Clearly, Xst,x = F (t, x, s, (W. − Wt )t≤u≤s ) for some deterministic function
F.
u,X t,x
• For t ≤ u ≤ s: Xst,x = Xs u . This follows from the pathwise uniqueness,
and holds also when u is a stopping time.
With these observations, we have the following Markov property for the solutions
of stochastic differential equations.
Proposition 1.5. (Markov property) For all 0 ≤ t ≤ s:
E [Φ (Xu , t ≤ u ≤ s) |Ft ] = E [Φ (Xu , t ≤ u ≤ s) |Xt ]
for all bounded function Φ : C([t, s]) −→ R.
1.3 Connection with linear partial differential

equations
1.3.1 Generator
Let {Xst,x , s
≥ t} be the unique strong solution of
Z s Z s
t,x t,x
Xs = x+ µ(u, Xu )du + σ(u, Xut,x )dWu , s ≥ t,
t t
where µ and σ satisfy the required condition for existence and uniqueness of a
strong solution.
For a function f : Rn −→ R, we define the function Af by
t,x
E[f (Xt+h )] − f (x)
Af (t, x) = lim if the limit exists.
h→0 h
Clearly, Af is well-defined for all bounded C 2 − function with bounded deriva-
tives and
∂2f

1
Af (t, x) = µ(t, x) · f (t, x) + Tr σσ T (t, x) , (1.7)
2 ∂x∂xT
1.3. Connection with PDE 13
(Exercise !). The linear differential operator A is called the generator of X. It

turns out that the process X can be completely characterized by its generator or,
more precisely, by the generator and the corresponding domain of definition...
As the following result shows, the generator provides an intimate connection
between conditional expectations and linear partial differential equations.
Proposition 1.6. Assume that the function (t, x) 7−→ v(t, x) := E g(XTt,x is

C 1,2 ([0, T ) × Rn ). Then v solves the partial differential equation:

∂v
+ Av = 0 and v(T, .) = g.
∂t
Proof. Given (t, x), let τ1 := T ∧ inf{s > t : |Xst,x − x| ≥ 1}. By the law of
iterated expectation together with the Markov property of the process X, it
follows that
t,x
v(t, x) = E v s ∧ τ1 , Xs∧τ 1
.
Since v ∈ C 1,2 ([0, T ), Rn ), we may apply Itô’s formula, and we obtain by taking
expectations:
Z s∧τ1
∂v t,x
0 = E + Av (u, Xu )du
t ∂t
Z s∧τ1
∂v t,x t,x
+E (u, Xs ) · σ(u, Xu )dWu
t ∂x
Z s∧τ1
∂v t,x
= E + Av (u, Xu )du ,
t ∂t
where the last equality follows from the boundedness of (u, Xut,x ) on [t, s∧τ1 ]. We
now send s & t, and the required result follows from the dominated convergence
theorem. ♦
1.3.2 Cauchy problem and the Feynman-Kac representa-

tion
In this section, we consider the following linear partial differential equation
∂v
∂t+ Av − k(t, x)v + f (t, x) = 0, (t, x) ∈ [0, T ) × Rd
(1.8)
v(T, .) = g
where A is the generator (1.7), g is a given function from Rd to R, k and f are
functions from [0, T ] × Rd to R, b and σ are functions from [0, T ] × Rd to Rd
and and MR (d, d), respectively. This is the so-called Cauchy problem.
For example, when k = f ≡ 0, b ≡ 0, and σ is the identity matrix, the above
partial differential equation reduces to the heat equation.
Our objective is to provide a representation of this purely deterministic prob-
lem by means of stochastic differential equations. We then assume that µ and
σ satisfy the conditions of Theorem 1.2, namely that
Z T
|µ(t, 0)|2 + |σ(t, 0)|2 dt < ∞.(1.9)

µ, σ Lipschitz in x uniformly in t,
0
Theorem 1.7. Let the coefficients µ, σ be continuous and satisfy (1.9). Assume
further that the function k is uniformly bounded from below, and f has quadratic
growth in x uniformly in t. Let v be a C 1,2 [0, T ), Rd ∩C 0 [0, T ) × Rd solution
of (1.8) with quadratic growth in x uniformly in t. Then
"Z #
T
t,x t,x t,x t,x
v(t, x) = E βs f (s, Xs )ds + βT g XT , t ≤ T, x ∈ Rd ,
t
Rs Rs Rs t,x
where Xst,x := x+ t
µ(u, Xut,x )du+ t
σ(u, Xut,x )dWu and βst,x := e− t
k(u,Xu )du
for t ≤ s ≤ T .
Proof. We first introduce the sequence of stopping times
1
∧ ∧ inf s > t : Xst,x − x ≥ n ,

τn := T−
n
and we oberve that τn −→ T P−a.s. Since v is smooth, it follows from Itô’s
formula that for t ≤ s < T :

t,x t,x t,x ∂v
+ Av s, Xst,x ds

d βs v s, Xs = βs −kv +
∂t
∂v
+βst,x s, Xst,x · σ s, Xst,x dWs

∂x
t,x t,x ∂v t,x
t,x

= βs −f (s, Xs )ds + s, Xs · σ s, Xs dWs ,
∂x
by the PDE satisfied by v in (1.8). Then:
E βτt,x v τn , Xτt,x

n n
− v(t, x)
Z τn
∂v
βst,x −f (s, Xs )ds + s, Xst,x · σ s, Xst,x dWs .

= E
t ∂x
Now observe that the integrands in the stochastic integral is bounded by def-
inition of the stopping time τn , the smoothness of v, and the continuity of σ.
Then the stochastic integral has zero mean, and we deduce that
Z τn
t,x t,x t,x t,x

v(t, x) = E βs f s, Xs ds + βτn v τn , Xτn . (1.10)
t
Since τn −→ T and the Brownian motion has continuous sample paths P−a.s.
it follows from the continuity of v that, P−a.s.
Z τn
βst,x f s, Xst,x ds + βτt,x v τn , Xτt,x

n n
t Z T
n→∞
βst,x f s, Xst,x ds + βTt,x v T, XTt,x

−→ (1.11)
t Z
T
βst,x f s, Xst,x ds + βTt,x g XTt,x

=
t
by the terminal condition satisfied by v in (1.8). Moreover, since k is bounded

from below and the functions f and v have quadratic growth in x uniformly in
t, we have
Z τn
t,x t,x t,x t,x 2

βs f s, Xs ds + βτn v τn , Xτn ≤ C 1 + max |Xt | .

t t≤T
By the estimate stated in the existence and uniqueness theorem 1.2, the latter
bound is integrable, and we deduce from the dominated convergence theorem
that the convergence in (1.11) holds in L1 (P), proving the required result by
taking limits in (1.10). ♦
The above Feynman-Kac representation formula has an important numerical

implication. Indeed it opens the door to the use of Monte Carlo methods in order
to obtain a numerical approximation of the solution of the partial differential
equation (1.8). For sake of simplicity,
we provide the main idea in the case
f = k = 0. Let X (1) , . . . , X (k) be an iid sample drawn in the distribution of
XTt,x , and compute the mean:
k
1 X (i)
v̂k (t, x) := g X .
k i=1
By the Law of Large Numbers, it follows that v̂k (t, x) −→ v(t, x) P−a.s. More-
over the error estimate is provided by the Central Limit Theorem:
√ k→∞
k (v̂k (t, x) − v(t, x)) −→ N 0, Var g XTt,x

in distribution,
and is remarkably independent of the dimension d of the variable X !
1.3.3 Representation of the Dirichlet problem

Let D be an open bounded subset of Rd . The Dirichlet problem is to find a
function u solving:
Au − ku + f = 0 on D and u = g on ∂D, (1.12)
where ∂D denotes the boundary of D, f and k are continuous functions from

Rd to R, and A is the generator of the process X 0,X0 defined as the unique
strong solution of the homogeneous (time independent coefficients) stochastic
differential equation
Z t Z t
0,X0 0,X0
Xt = X0 + µ(Xs )ds + σ(Xs0,X0 )dWs , t ≥ 0.
0 0
Similarly to the the representation result of the Cauchy problem obtained in

Theorem 1.7, we have the following representation result for the Dirichlet prob-
lem.
Theorem 1.8. Let u be a C 2 −solution of the Dirichlet problem (1.12). Assume

that k is nonnegative, and
n o
x
E[τD ] < ∞, x ∈ Rd , where τDx
:= inf t ≥ 0 : Xt0,x 6∈ D .
Then, we have the representation:

x
" #
R τDx Z τD Rt
0,x
u(x) = E g Xτ x e − 0 k(Xs )ds
+ f Xt0,x e− 0
k(Xs )ds
dt .
D
0
Exercise 1.9. Provide a proof of Theorem 1.8 by imitating the arguments in

the proof of Theorem 1.7.
1.4 The stochastic control approach to the Black-

Scholes model
1.4.1 The continuous-time financial market
Let T be a finite horizon, and (Ω, F, P) be a complete probability space sup-
porting a Brownian motion W = {(Wt1 , . . . , Wtd ), 0 ≤ t ≤ T } with values in Rd .
We denote by F = FW = {Ft , 0 ≤ t ≤ T } the canonical augmented filtration of
W , i.e. the canonical filtration augmented by zero measure sets of FT .
We consider a financial market consisting of d + 1 assets :
(i) The first asset S 0 is non-risky, and is defined by
Z t
St0 = exp ru du , 0 ≤ t ≤ T,
0
RT
where {rt , t ∈ [0, T ]} is a non-negative adapted processes with 0 rt dt < ∞ a.s.,
and represents the instantaneous interest rate.
(ii) The d remaining assets S i , i = 1, . . . , d, are risky assets with price
processes defined by the dynamics
d
dSti X
= µit dt + σti,j dWtj , t ∈ [0, T ],
Sti j=1
RT RT
for 1 ≤ i ≤ d, where µ, σ are F−adapted processes with 0 |µit |dt+ 0 |σti,j |2 dt <
∞ for all i, j = 1, . . . , d. It is convenient to use the matrix notations to represent
the dynamics of the price vector S = (S 1 , . . . , S d ):
dSt = St ? (µt dt + σt dWt ) , t ∈ [0, T ],
where, for two vectors x, y ∈ Rd , we denote x ? y the vector of Rd with compo-

nents (x ? y)i = xi yi , i = 1, . . . , d, and µ, σ are the Rd −vector with components
µi ’s, and the MR (d, d)−matrix with entries σ i,j .
We assume that the MR (d, d)−matrix σt is invertible for every t ∈ [0, T ]

a.s., and we introduce the process
λt := σt−1 (µt − rt 1) , 0 ≤ t ≤ T,
called the risk premium process. Here 1 is the vector of ones in Rd . We shall
frequently make use of the discounted processes
Z t
St
S̃t := 0 = St exp − ru du ,
St 0
Using the above matrix notations, the dynamics of the process S̃ are given by

dS̃t = S̃t ? (µt − rt 1)dt + σt dWt = S̃t ? σt (λt dt + dWt ) .
1.4.2 Portfolio and wealth process

A portfolio strategy is an F−adapted process π = {πt , 0 ≤ t ≤ T } with values
in Rd . For 1 ≤ i ≤ n and 0 ≤ t ≤ T , πti is the amount (in Euros) invested in
the risky asset S i .
We next recall the self-financing condition in the present framework. Let Xtπ
denote the portfolio value, or wealth, process at time t induced by the portfolio
Pn
strategy π. Then, the amount invested in the non-risky asset is Xtπ − i=1 πti
= Xtπ − πt · 1.
Under the self-financing condition, the dynamics of the wealth process is
given by
n
X πti Xtπ − πt · 1
dXtπ = i
dSt
i
+ 0 dSt0 .
i=1
St St
Let X̃ π be the discounted wealth process

Z t
X̃tπ := Xtπ exp − r(u)du , 0 ≤ t ≤ T.
0
Then, by an immediate application of Itô’s formula, we see that
dX̃t = π̃t · σt (λt dt + dWt ) , 0 ≤ t ≤ T, (1.13)
where π̃t := e−rt πt . We still need to place further technical conditions on π,

at least in order for the above wealth process to be well-defined as a stochastic
integral.
Before this, let us observe that, assuming that the risk premium process
satisfies the Novikov condition:
h 1 RT 2 i
E e 2 0 |λt | dt < ∞,
it follows from the Girsanov theorem that the process

Z t
Bt := Wt + λu du , 0≤t≤T, (1.14)
0
is a Brownian motion under the equivalent probability measure

!
Z T Z T
1
Q := ZT · P on FT where ZT := exp − λu · dWu − |λu |2 du .
0 2 0
In terms of the Q Brownian motion B, the discounted price process satisfies
dS̃t = S̃t ? σt dBt , t ∈ [0, T ],
and the discounted wealth process induced by an initial capital X0 and a port-
folio strategy π can be written in
Z t
X̃tπ = X̃0 + π̃u · σu dBu , for 0 ≤ t ≤ T. (1.15)
0
Definition 1.10. An admissible portfolio process π = {θt , t ∈ [0, T ]} is an

RT
F−progressively measurable process such that 0 |σtT πt |2 dt < ∞, a.s. and the
corresponding discounted wealth process is bounded from below by a Q−martingale
X̃tπ ≥ Mtπ , 0 ≤ t ≤ T, for some Q−martingale M π .
The collection of all admissible portfolio processes will be denoted by A.

The lower bound M π , which may depend on the portfolio π, has the interpre-
tation of a finite credit line imposed on the investor. This natural generalization
of the more usual constant credit line corresponds to the situation where the
total credit available to an investor is indexed by some financial holding, such as
the physical assets of the company or the personal home of the investor, used as
collateral. From the mathematical viewpoint, this condition is needed in order
to exclude any arbitrage opportunity, and will be justified in the subsequent
subsection.
1.4.3 Admissible portfolios and no-arbitrage

We first define precisely the notion of no-arbitrage.
Definition 1.11. We say that the financial market contains no arbitrage op-
portunities if for any admissible portfolio process θ ∈ A,
X0 = 0 and XTθ ≥ 0 P − a.s. implies XTθ = 0 P − a.s.
The purpose of this section is to show that the financial market described
above contains no arbitrage opportunities. Our first observation is that, by the
very definition of the probability measure Q, the discounted price process S̃

satisfies:
n o
the process S̃t , 0 ≤ t ≤ T is a Q − local martingale. (1.16)
For this reason, Q is called a risk neutral measure, or an equivalent local mar-
tingale measure, for the price process S.
We also observe that the discounted wealth process satisfies:
X̃ π is a Q−local martingale for every π ∈ A, (1.17)
as a stochastic integral with respect to the Q−Brownian motion B.
Theorem 1.12. The continuous-time financial market described above contains
no arbitrage opportunities, i.e. for every π ∈ A:
X0 = 0 and XTπ ≥ 0 P − a.s. =⇒ XTπ = 0 P − a.s.
Proof. For π ∈ A, the discounted wealth process X̃ π is a Q−local martingale
π
bounded from below h byi a Q−martingale. Then X̃ is a Q−super-martingale.
In particular, EQ X̃Tπ ≤ X̃0 = X0 . Recall that Q is equivalent to P and S 0
is strictly positive. Then, this inequality shows that, whenever X0π = 0 and
XTπ ≥ 0 P−a.s. (or equivalently Q−a.s.), we have X̃Tπ = 0 Q−a.s. and therefore
XTπ = 0 P−a.s. ♦
1.4.4 Super-hedging and no-arbitrage bounds

Let G be an FT −measurable random variable representing the payoff of a deriva-
tive security with given maturity T > 0. The super-hedging problem consists in
finding the minimal initial cost so as to be able to face the payment G without
risk at the maturity of the contract T :
V (G) := inf {X0 ∈ R : XTπ ≥ G P − a.s. for some π ∈ A} .
Remark 1.13. Notice that V (G) depends on the reference measure P only by
means of the corresponding null sets. Therefore, the super-hedging problem is
not changed if P is replaced by any equivalent probability measure.
We now show that, under the no-arbitrage condition, the super-hedging
problem provides no-arbitrage bounds on the market price of the derivative se-
curity.
Assume that the buyer of the contingent claim G has the same access to
the financial market than the seller. Then V (G) is the maximal amount that
the buyer of the contingent claim contract is willing to pay. Indeed, if the seller
requires a premium of V (G) + 2ε, for some ε > 0, then the buyer would not
accept to pay this amount as he can obtain at least G by trading on the financial
market with initial capital V (G) + ε.
Now, since selling of the contingent claim G is the same as buying the con-
tingent claim −G, we deduce from the previous argument that
−V (−G) ≤ market price of G ≤ V (G). (1.18)
1.4.5 The no-arbitrage valuation formula

We denote by p(G) the market price of a derivative security G.
Theorem 1.14. Let G be an FT −measurabel random variable representing the
payoff of a derivative security at the maturity T > 0, and recall the notation
RT
G̃ := G exp − 0 rt dt . Assume that EQ [|G̃|] < ∞. Then
p(G) = V (G) = EQ [G̃].

∗ ∗
Moreover, there exists a portfolio π ∗ ∈ A such that X0π = p(G) and XTπ = G,
a.s., that is π ∗ is a perfect replication strategy.
Proof. 1- We first prove that V (G) ≥ EQ [G̃]. Let X0 and π ∈ A be such that
XTπ ≥ G, a.s. or, equivalently, X̃Tπ ≥ G̃ a.s. Notice that X̃ π is a Q−super-
martingale, as a Q−local martingale bounded from below by a Q−martingale.
Then X0 = X̃0 ≥ EQ [X̃Tπ ] ≥ EQ [G̃].
2- We next prove that V (G) ≤ EQ [G̃]. Define the Q−martingale Yt := EQ [G̃|Ft ]
and observe that FW = FB . Then, it follows from the martingale representa-
RT
tion theorem that Yt = Y0 + 0 φt · dBt for some F−adapted process φ with
RT
0
|φt |2 dt < ∞ a.s. Setting π̃ ∗ := (σ T )−1 φ, we see that
Z T
∗
π ∈ A and Y0 + π̃ ∗ · σt dBt = G̃ P − a.s.
0
which implies that Y0 ≥ V (G) and π ∗ is a perfect hedging stratgey for G,

starting from the initial capital Y0 .
3- From the previous steps, we have V (G) = EQ [G̃]. Applying this result to −G,
we see that V (−G) = −V (G), so that the no-arbitrage bounds (1.18) imply that
the no-arbitrage market price of G is given by V (G). ♦
1.4.6 PDE characterization of the Black-Scholes price

In this subsection, we specialize further the model to the case where the risky
securities price processes are Markov diffusions defined by the stochastic differ-
ential equations:

dSt = St ? r(t, St )dt + σ(t, St )dBt .
Here (t, s) 7−→ s ? r(t, s) and (t, s) 7−→ s ? σ(t, s) are Lipschitz-continuous func-
tions from R+ × [0, ∞)d to Rd and Sd , successively. We also consider a Vanilla
derivative security defined by the payoff
G = g(ST ),
where g : [0, ∞)d → R is a measurable function bounded from below. From the
previous subsection, the no-arbitrage price at time t of this derivative security
is given by
h RT i h RT i
V (t, St ) = EQ e− t r(u,Su )du g(ST )|Ft = EQ e− t r(u,Su )du g(ST )|St ,
where the last equality follows from the Markov property of the process S.
Assuming further that g has linear growth, it follows that V has linear growth
in s uniformly in t. Since V is defined by a conditional expectation, it is expected
to satisfy the linear PDE:
1
−∂t V − rs ? DV − Tr (s ? σ)2 D2 V − rV

= 0. (1.19)
2
More precisely, if V ∈ C 1,2 (R+ , Rd ), the V is a classical solution of (1.19) and
satisfies the final condition V (T, .) = g. Coversely, if the PDE (1.19) combined
with the final condition v(T, .) = g has a classical solution v with linear growth,
then v coincides with the derivative security price V .
Chapter 2
Stochastic Control
and Dynamic Programming
In this chapter, we assume that the filtration F is the P−augmentation of the

canonical filtration of the Brownian motion W . This restriction is only needed
in order to simplify the presentation of the proof of the dynamic programming
principle. We will also denote by
S := [0, T ) × Rn where T ∈ [0, ∞].
The set S is called the parabolic interior of the state space. We will denote by
S̄ := cl(S) its closure, i.e. S̄ = [0, T ] × Rn for finite T , and S̄ = S for T = ∞.
2.1 Stochastic control problems in standard form

Control processes. Given a subset U of Rk , we denote by U the set of all pro-
gressively measurable processes ν = {νt , t < T } valued in U . The elements of
U are called control processes.
Controlled Process. Let
b : (t, x, u) ∈ S × U −→ b(t, x, u) ∈ Rn
and
σ : (t, x, u) ∈ S × U −→ σ(t, x, u) ∈ MR (n, d)
be two continuous functions satisfying the conditions
|b(t, x, u) − b(t, y, u)| + |σ(t, x, u) − σ(t, y, u)| ≤ K |x − y|, (2.1)

|b(t, x, u)| + |σ(t, x, u)| ≤ K (1 + |x| + |u|). (2.2)
23
24 CHAPTER 2. STOCHASTIC CONTROL, DYNAMIC PROGRAMMING
for some constant K independent of (t, x, y, u). For each control process ν ∈ U,
we consider the controlled stochastic differential equation :
dXt = b(t, Xt , νt )dt + σ(t, Xt , νt )dWt . (2.3)
If the above equation has a unique solution X, for a given initial data, then
the process X is called the controlled process, as its dynamics is driven by the
action of the control process ν.
We shall be working with the following subclass of control processes :
U0 := U ∩ H2 , (2.4)
where H2 is the collection of all progressively measurable processes with finite

L2 (Ω × [0, T ))−norm. Then, for every finite maturity T 0 ≤ T , it follows from
the above uniform Lipschitz condition on the coefficients b and σ that
"Z 0 #
T
|b| + |σ| (s, x, νs )ds < ∞ for all ν ∈ U0 , x ∈ Rn ,
2

E
0
which guarantees the existence of a controlled process on the time interval [0, T 0 ]
for each given initial condition and control. The following result is an immediate
consequence of Theorem 1.2.
Theorem 2.1. Let ν ∈ U0 be a control process, and ξ ∈ L2 (P) be an F0 −measurable
random variable. Then, there exists a unique F−adapted process X ν satisfying
(6.3) together with the initial condition X0ν = ξ. Moreover for every T > 0,
there is a constant C > 0 such that

E sup |Xsν |2 < C(1 + E[|ξ|2 ])eCt for all t ∈ cl([0, T )). (2.5)
0≤s≤t
Cost functional. Let
f, k : [0, T ) × Rn × U −→ R and g : Rn −→ R
be given functions. We assume that f, k are continuous and kk − k∞ < ∞ (i.e.

max(−k, 0) is uniformly bounded). Moreover, we assume that f and g satisfy
the quadratic growth condition :
|f (t, x, u)| + |g(x)| ≤ K(1 + |u| + |x|2 ),
for some constant K independent of (t, x, u). We define the cost function J on
[0, T ] × Rn × U by :
"Z #
T
J(t, x, ν) := E β ν (t, s)f (s, Xst,x,ν , νs )ds + β ν (t, T )g(XTt,x,ν )1T <∞ ,
t
when this expression is meaningful, where

Rs
k(r,Xrt,x,ν ,νr )dr
β ν (t, s) := e− t ,
2.1. Standard stochastic control 25
and {Xst,x,ν , s ≥ t} is the solution of (6.3) with control process ν and initial
condition Xtt,x,ν = x.
Admissible control processes. In the finite horizon case T < ∞, the quadratic
growth condition on f and g together with the bound on k − ensure that J(t, x, ν)
is well-defined for all control process ν ∈ U0 . We then define the set of admissible
controls in this case by U0 .
More attention is needed for the infinite horizon case. In particular, the
discount term k needs to play a role to ensure the finiteness of the integral. In
this setting the largest set of admissible control processes is given by
∞ ν
n Z o
β (t, s) 1+|Xst,x,ν |2 +|νs )| ds < ∞ for all x when T = ∞.

U0 := ν ∈ U : E
0
The stochastic control problem. The purpose of this section is to study the min-
imization problem
V (t, x) := sup J(t, x, ν) for (t, x) ∈ S.
ν∈U0
Our main concern is to describe the local behavior of the value function V
by means of the so-called dynamic programming equation, or Hamilton-Jacobi-
Bellman equation. We continue with some remarks.
Remark 2.2. (i) If V (t, x) = J(t, x, ν̂t,x ), we call ν̂t,x an optimal control for
the problem V (t, x).
(ii) The following are some interesting subsets of controls :
- a process ν ∈ U0 which is adapted to the natural filtration FX of the
associated state process is called feedback control,
- a process ν ∈ U0 which can be written in the form νs = ũ(s, Xs ) for some
measurable map ũ from [0, T ] × Rn into U , is called Markovian control;
notice that any Markovian control is a feedback control,
- the deterministic processes of U0 are called open loop controls.
(iii) Suppose that T < ∞, and let (Y, Z) be the controlled processes defined
by
dYs = Zs f (s, Xs , νs )ds and dZs = −Zs k(s, Xs , νs )ds ,
and define the augmented state process X̄ := (X, Y, Z). Then, the above
value function V can be written in the form :
V (t, x) = V̄ (t, x, 0, 1) ,
where x̄ = (x, y, z) is some initial data for the augmented state process X̄,

V̄ (t, x̄) := Et,x̄ ḡ(X̄T ) and ḡ(x, y, z) := y + g(x)z .
Hence the stochastic control problem V can be reduced without loss of
generality to the case where f = k ≡ 0. We shall appeal to this reduced
form whenever convenient for the exposition.
(iv) For notational simplicity we consider the case T < ∞ and f = k = 0. The
previous remark shows how to immediately adapt the following argument
so that the present remark holds true without the restriction f = k = 0.
The extension to the infinite horizon case is also immediate.
Consider the value function
sup E g(XTt,x,ν ) ,

Ṽ (t, x) := (2.6)
ν∈Ut
differing from V by the restriction of the control processes to
Ut := {ν ∈ U0 : ν independent of Ft } . (2.7)
Since Ut ⊂ U0 , it is obvious that Ṽ ≤ V . We claim that
Ṽ = V, (2.8)
so that both problems are indeed equivalent. To see this, fix (t, x) ∈ S and
ν ∈ U0 . Then, ν can be written as a measurable function of the canonical
process ν((ωs )0≤s≤t , (ωs − ωt )t≤s≤T ), where, for fixed (ωs )0≤s≤t , the map
ν(ωs )0≤s≤t : (ωs − ωt )t≤s≤T 7→ ν((ωs )0≤s≤t , (ωs − ωt )t≤s≤T ) can be viewed
as a control independent on Ft . Using the independence of the increments
of the Brownian motion, together with Fubini’s Lemma, it thus follows
that
Z h
t,x,ν(ωs )0≤s≤t i
J(t, x; ν) = E g(XT ) dP((ωs )0≤s≤t )
Z
≤ Ṽ (t, x)dP((ωs )0≤s≤t ) = Ṽ (t, x).
By arbitrariness of ν ∈ U0 , this implies that Ṽ (t, x) ≥ V (t, x).
2.2 The dynamic programming principle

2.2.1 A weak dynamic programming principle
The dynamic programming principle is the main tool in the theory of stochastic
control. In these notes, we shall prove rigorously a weak version of the dy-
namic programming which will be sufficient for the derivation of the dynamic
programming equation. We denote:
V∗ (t, x) := lim inf V (t0 , x0 ) and V ∗ (t, x) := lim sup V (t0 , x0 ),

(t0 ,x0 )→(t,x) (t0 ,x0 )→(t,x)
for all (t, x) ∈ S̄. We also recall the subset of controls Ut introduced in (2.7)
above.
2.2. Dynamic programming principle 27
Theorem 2.3. Assume that V is locally bounded and fix (t, x) ∈ S. Let {θν , ν ∈
Ut } be a family of finite stopping times independent of Ft with values in [t, T ].
Then:
"Z ν #
θ
ν t,x,ν ν ν ν t,x,ν
V (t, x) ≥ sup E β (t, s)f (s, Xs , νs )ds + β (t, θ )V∗ (θ , Xθν ) .
ν∈Ut t
Assume further that g is lower-semicontinuous and Xt,x ν

1[t,θν ] is L∞ −bounded
for all ν ∈ Ut . Then
"Z ν #
θ
t,x,ν
V (t, x) ≤ sup E β ν (t, s)f (s, Xst,x,ν , νs )ds + β ν (t, θν )V ∗ (θν , Xθν ) .
ν∈Ut t
We shall provide an intuitive justification of this result after the following

comments. A rigorous proof is reported in Section 2.2.2 below.
(i) If V is continuous, then V = V∗ = V ∗ , and the above weak dynamic pro-
gramming principle reduces to the classical dynamic programming princi-
ple:
"Z #
θ
V (t, x) = sup Et,x β(t, s)f (s, Xs , νs )ds + β(t, θ)V (θ, Xθ ) (2.9)
.
ν∈U t
(ii) In the discrete-time framework, the dynamic programming principle (2.9)

can be stated as follows :
h i
V (t, x) = sup Et,x f (t, Xt , u) + e−k(t+1,Xt+1 ,u) V (t + 1, Xt+1 ) .
u∈U
Observe that the supremum is now taken over the subset U of the finite
dimensional space Rk . Hence, the dynamic programming principle allows
to reduce the initial maximization problem, over the subset U of the in-
finite dimensional set of Rk −valued processes, into a finite dimensional
maximization problem. However, we are still facing an infinite dimen-
sional problem since the dynamic programming principle relates the value
function at time t to the value function at time t + 1.
(iii) In the context of the above discrete-time framework with finite horizon
T < ∞, notice that the dynamic programming principle suggests the fol-
lowing backward algorithm to compute V as well as the associated optimal
strategy (when it exists). Since V (T, ·) = g is known, the above dynamic
programming principle can be applied recursively in order to deduce the
value function V (t, x) for every t.
(iv) In the continuous time setting, there is no obvious counterpart to the
above backward algorithm. But, as the stopping time θ approaches t,
the above dynamic programming principle implies a special local behavior
for the value function V . When V is known to be smooth, this will be
obtained by means of Itô’s formula.
(v) It is usually very difficult to determine a priori the regularity of V . The

situation is even worse since there are many counter-examples showing
that the value function V can not be expected to be smooth in general;
see Section 2.4. This problem is solved by appealing to the notion of
viscosity solutions, which provides a weak local characterization of the
value function V .
(vi) Once the local behavior of the value function is characterized, we are
faced to the important uniqueness issue, which implies that V is com-
pletely characterized by its local behavior together with some convenient
boundary condition.
Intuitive justification of (2.9). Let us assume that V is continuous. In

particular, V is measurable and V = V∗ = V ∗ . Let Ṽ (t, x) denote the right
hand-side of (2.9).
By the tower Property of the conditional expectation operator, it is easily
checked that
"Z #
θ
J(t, x, ν) = Et,x β(t, s)f (s, Xs , νs )ds + β(t, θ)J(θ, Xθ , ν) .
t
Since J(θ, Xθ , ν) ≤ V (θ, Xθ ), this proves that V ≤ Ṽ . To prove the reverse

inequality, let µ ∈ U and ε > 0 be fixed, and consider an ε−optimal control ν ε
for the problem V (θ, Xθ ), i.e.
J(θ, Xθ , ν ε ) ≥ V (θ, Xθ ) − ε.
Clearly, one can choose ν ε = µ on the stochastic interval [t, θ]. Then
"Z #
θ
ε ε
V (t, x) ≥ J(t, x, ν ) = Et,x β(t, s)f (s, Xs , µs )ds + β(t, θ)J(θ, Xθ , ν )
t
"Z #
θ
≥ Et,x β(t, s)f (s, Xs , µs )ds + β(t, θ)V (θ, Xθ ) − ε Et,x [β(t, θ)] .
t
This provides the required inequality by the arbitrariness of µ ∈ U and ε > 0.

♦
Exercise. Where is the gap in the above sketch of the proof ?
2.2.2 Dynamic programming without measurable selec-

tion
In this section, we provide a rigorous proof of Theorem 2.3. Notice that, we
have no information on whether V is measurable or not. Because of this, the
2.2. Dynamic programming principle 29
right-hand side of the classical dynamic programming principle (2.9) is not even
known to be well-defined.
The formulation of Theorem 2.3 avoids this measurability problem since
V∗ and V ∗ are lower- and upper-semicontinuous, respectively, and therefore
measurable. In addition, it allows to avoid the typically heavy technicalities
related to measurable selection arguments needed for the proof of the classical
(2.9) after a convenient relaxation of the control problem, see e.g. El Karoui
and Jeanblanc [5].
Proof of Theorem 2.3 For simplicity, we consider the finite horizon case
T < ∞, so that, without loss of generality, we assume f = k = 0, See Remark
2.2 (iii). The extension to the infinite horizon framework is immediate.
1. Let ν ∈ Ut be arbitrary and set θ := θν . Then:
E g XTt,x,ν |Fθ (ω) = J(θ(ω), Xθt,x,ν (ω); ν̃ω ),

where ν̃ω is obtained from ν by freezing its trajectory up to the stopping time
θ. Since, by definition, J(θ(ω), Xθt,x,ν (ω); ν̃ω ) ≤ V ∗ (θ(ω), Xθt,x,ν (ω)), it follows
from the tower property of conditional expectations that
E g XTt,x,ν = E E g XTt,x,ν |Fθ ≤ E V ∗ θ, Xθt,x,ν ,

which provides the second inequality of Theorem 2.3 by the arbitrariness of

ν ∈ Ut .
2. Let ε > 0 be given, and consider an arbitrary function
ϕ : S −→ R such that ϕ upper-semicontinuous and V ≥ ϕ.
2.a. There is a family (ν (s,y),ε )(s,y)∈S ⊂ U0 such that:
ν (s,y),ε ∈ Us and J(s, y; ν (s,y),ε ) ≥ V (s, y) − ε, for every (s, y) ∈ S.(2.10)
Since g is lower-semicontinuous and has quadratic growth, it follows from Theo-

rem 2.1 that the function (t0 , x0 ) 7→ J(t0 , x0 ; ν (s,y),ε ) is lower-semicontinuous, for
fixed (s, y) ∈ S. Together with the upper-semicontinuity of ϕ, this implies that
we may find a family (r(s,y) )(s,y)∈S of positive scalars so that, for any (s, y) ∈ S,
ϕ(s, y) − ϕ(t0 , x0 ) ≥ −ε and J(s, y; ν (s,y),ε ) − J(t0 , x0 ; ν (s,y),ε ) ≤ ε

(2.11)
for (t0 , x0 ) ∈ B(s, y; r(s,y) ),
where, for r > 0 and (s, y) ∈ S,
B(s, y; r) := {(t0 , x0 ) ∈ S : t0 ∈ (s − r, s), |x0 − y| < r} .

Clearly, B(s, y; r) : (s, y) ∈ S, 0 < r ≤ r(s,y) forms an open covering of
[0, T ) × Rd . It then follows from the Lindelöf covering Theorem, see e.g. [4]
Theorem 6.3 Chap. VIII, that we can find a countable sequence (ti , xi , ri )i≥1
of elements of S × R, with 0 < ri ≤ r(ti ,xi ) for all i ≥ 1, such that S ⊂
{T } × Rd ∪ (∪i≥1 B(ti , xi ; ri )). Set A0 := {T } × Rd , C−1 := ∅, and define the

sequence
Ai+1 := B(ti+1 , xi+1 ; ri+1 ) \ Ci where Ci := Ci−1 ∪ Ai , i ≥ 0.
With this construction, it follows from (2.10), (2.11), together with the fact that
V ≥ ϕ, that the countable family (Ai )i≥0 satisfies
(θ, Xθt,x,ν ) ∈ ∪i≥0 Ai P − a.s., Ai ∩ Aj = ∅ for i 6= j ∈ N,

(2.12)
and J(·; ν i,ε ) ≥ ϕ − 3ε on Ai for i ≥ 1,
where ν i,ε := ν (ti ,xi ),ε for i ≥ 1.

2.b. We now prove the first inequality in Theorem 2.3. We fix ν ∈ Ut and
t n
θ ∈ T[t,T ] . Set A := ∪0≤i≤n Ai , n ≥ 1. Given ν ∈ Ut , we define for s ∈ [t, T ]:
n
X
t,x,ν
νsε,n := 1[t,θ] (s)νs + 1(θ,T ] (s) νs 1(An )c (θ, Xθ ) + 1Ai (θ, Xθt,x,ν )νsi,ε .
i=1
Notice that {(θ, Xθt,x,ν )

∈ Ai } ∈ Fθt . Then, it follows that ν ε,n ∈ Ut . Then, it
follows from (2.12) that:
h ε,n
i ε,n

E g XTt,x,ν |Fθ 1An θ, Xθt,x,ν = V T, XTt,x,ν 1A0 θ, Xθt,x,ν

n
X
J(θ, Xθt,x,ν , ν i,ε )1Ai θ, Xθt,x,ν

+
i=1
n
X
ϕ(θ, Xθt,x,ν − 3ε 1Ai θ, Xθt,x,ν

≥
i=0
ϕ(θ, Xθt,x,ν ) − 3ε 1An θ, Xθt,x,ν ,

=
which, by definition of V and the tower property of conditional expectations,

implies
V (t, x) ≥ J(t, x, ν ε,n )

h h ε,n
ii
= E E g XTt,x,ν |Fθ
≥ E ϕ θ, Xθt,x,ν − 3ε 1An θ, Xθt,x,ν

+E g XTt,x,ν 1(An )c θ, Xθt,x,ν .

Since g XTt,x,ν ∈ L1 , it follows from the dominated convergence theorem that:

V (t, x) ≥ −3ε + lim inf E ϕ(θ, Xθt,x,ν )1An θ, Xθt,x,ν

n→∞
= −3ε + lim E ϕ(θ, Xθt,x,ν )+ 1An θ, Xθt,x,ν

n→∞
− lim E ϕ(θ, Xθt,x,ν )− 1An θ, Xθt,x,ν

n→∞
= −3ε + E ϕ(θ, Xθt,x,ν ) ,

2.3. Dynamic programming equation 31
where the last equality follows from the left-hand side of (2.12)
and from the
monotone convergence theorem, due to the fact that either E ϕ(θ, Xθt,x,ν )+ <
∞ or E ϕ(θ, Xθt,x,ν )− < ∞. By the arbitrariness of ν ∈ Ut and ε > 0, this

shows that:
V (t, x) ≥ sup E ϕ(θ, Xθt,x,ν ) .

(2.13)
ν∈Ut
3. It remains to deduce the first inequality of Theorem 2.3 from (2.13). Fix
r > 0. It follows from standard arguments, see e.g. Lemma 3.5 in [12], that
we can find a sequence of continuous functions (ϕn )n such that ϕn ≤ V∗ ≤ V
for all n ≥ 1 and such that ϕn converges pointwise to V∗ on [0, T ] × Br (0).
Set φN := minn≥N ϕn for N ≥ 1 and observe that the sequence (φN )N is non-
decreasing and converges pointwise to V∗ on [0, T ] × Br (0). By (2.13) and the
monotone convergence Theorem, we then obtain:
V (t, x) ≥ lim E φN (θν , Xt,x
ν
(θν )) = E V∗ (θν , Xt,x
ν
(θν )) .

N →∞
2.3 The dynamic programming equation

The dynamic programming equation is the infinitesimal counterpart of the dy-
namic programming principle. It is also widely called the Hamilton-Jacobi-
Bellman equation. In this section, we shall derive it under strong smoothness
assumptions on the value function. Let S d be the set of all d × d symmetric
matrices with real coefficients, and define the map H : S × R × Rn × S d by :
H(t, x, r, p, γ)

1
:= sup −k(t, x, u)r + b(t, x, u) · p + Tr[σσ T (t, x, u)γ] + f (t, x, u) .
u∈U 2
We also need to introduce the linear second order operator Lu associated to the
controlled process {β(0, t)Xtu , t ≥ 0} controlled by the constant control process
u:
Lu ϕ(t, x) := −k(t, x, u)ϕ(t, x) + b(t, x, u) · Dϕ(t, x)
1
+ Tr σσ T (t, x, u)D2 ϕ(t, x) ,

2
where D and D2 denote the gradient and the Hessian operators with respect to
the x variable. With this notation, we have by Itô’s formula:
Z s
ν ν ν ν
β (0, s)ϕ(s, Xs ) − β (0, t)ϕ(t, Xt ) = β ν (0, r) (∂t + Lνr ) ϕ(r, Xrν )dr
t
Z s
+ β ν (0, r)Dϕ(r, Xrν ) · σ(r, Xrν , νr )dWr
t
for every s ≥ t and smooth function ϕ ∈ C 1,2 ([t, s], Rn ) and each admissible
control process ν ∈ U0 .
Proposition 2.4. Assume the value function V ∈ C 1,2 ([0, T ), Rn ), and let the
coefficients k(·, ·, u) and f (·, ·, u) be continuous in (t, x) for all fixed u ∈ U .
Then, for all (t, x) ∈ S:
−∂t V (t, x) − H t, x, V (t, x), DV (t, x), D2 V (t, x) ≥ 0.

(2.14)
Proof. Let (t, x) ∈ S and u ∈ U be fixed and consider the constant control
process ν = u, together with the associated state process X with initial data
Xt = x. For all h > 0, Define the stopping time :
θh := inf {s > t : (s − t, Xs − x) 6∈ [0, h) × αB} ,
where α > 0 is some given constant, and B denotes the unit ball of Rn . Notice
that θh −→ t, P−a.s. when h & 0, and θh = h for h ≤ h̄(ω) sufficiently small.
1. From the first inequality of the dynamic programming principle, it follows
that :
" #
Z θh
0 ≤ Et,x β(0, t)V (t, x) − β(0, θh )V (θh , Xθh ) − β(0, r)f (r, Xr , u)dr
t
"Z #
θh
·
= −Et,x β(0, r)(∂t V + L V + f )(r, Xr , u)dr
t
"Z #
θh
−Et,x β(0, r)DV (r, Xr ) · σ(r, Xr , u)dWr ,
t
the last equality follows from Itô’s formula and uses the crucial smoothness
assumption on V .
2. Observe that β(0, r)DV (r, Xr ) · σ(r, Xr , u) is bounded on the stochastic
interval [t, θh ]. Therefore, the second expectation on the right hand-side of the
last inequality vanishes, and we obtain :
" Z #
1 θh ·
−Et,x β(0, r)(∂t V + L V + f )(r, Xr , u)dr ≥ 0
h t
We now send h to zero. The a.s. convergence of the random value inside the
expectation is easily obtained by the mean value Theorem; recall that θh = h
Rθ
for sufficiently small h > 0. Since the random variable h−1 t h β(0, r)(L· V +
f )(r, Xr , u)dr is essentially bounded, uniformly in h, on the stochastic interval
[t, θh ], it follows from the dominated convergence theorem that :
−∂t V (t, x) − Lu V (t, x) − f (t, x, u) ≥ 0.
By the arbitrariness of u ∈ U , this provides the required claim. ♦
We next wish to show that V satisfies the nonlinear partial differential equa-
tion (2.15) with equality. This is a more technical result which can be proved by
different methods. We shall report a proof, based on a contradiction argument,
which provides more intuition on this result, although it might be slightly longer
than the usual proof reported in standard textbooks.
2.3. Dynamic programming equation 33
Proposition 2.5. Assume the value function V ∈ C 1,2 ([0, T ), Rn ), and let the
function H be upper semicontinuous, and kk + k∞ < ∞. Then, for all (t, x) ∈
S:
−∂t V (t, x) − H t, x, V (t, x), DV (t, x), D2 V (t, x) ≤ 0.

(2.15)
Proof. Let (t0 , x0 ) ∈ [0, T ) × Rn be fixed, assume to the contrary that
∂t V (t0 , x0 ) + H t0 , x0 , V (t0 , x0 ), DV (t0 , x0 ), D2 V (t0 , x0 ) < 0, (2.16)

and let us work towards a contradiction.

1. For a given parameter ε > 0, define the smooth function ϕ ≥ V by
ϕ(t, x) := V (t, x) + ε |t − t0 |2 + |x − x0 |4 .

Then
(V − ϕ)(t0 , x0 ) = 0, (DV − Dϕ)(t0 , x0 ) = 0, (∂t V − ∂t ϕ)(t0 , x0 ) = 0,

and (D2 V − D2 ϕ)(t0 , x0 ) = 0,
and (2.16) says that:
h(t0 , x0 ) := ∂t ϕ(t0 , x0 ) + H t0 , x0 , ϕ(t0 , x0 ), Dϕ(t0 , x0 ), D2 ϕ(t0 , x0 )

< 0.
2. By upper semicontinuity of H, we have:
h(t, x) < 0 on Nη := (−η, η) × ηB for η > 0 sufficiently small,
where B denotes the unit ball centered at x0 . We next observe that the param-
eter γ defined by the following is positive:
+
k∞
−γeηkk := max (V − ϕ) < 0. (2.17)
∂Nη
3. Let ν be an arbitrary control process, and denote by X and β the controlled

process and the discount factor defined by ν and the initial data Xt0 = x0 .
Consider the stopping time
θ := inf {s > t : (s, Xs ) 6∈ Nη } ,
and observe that, by continuity of the state process, (θ, Xθ ) ∈ ∂Nη , so that :
+
k∞
(V − ϕ)(θ, Xθ ) ≤ −γeηkk
by (2.17). Recalling that β(t0 , t0 ) = 1, we now compute that:

Z θ
+
β(t0 , θ)V (θ, Xθ ) − V (t0 , x0 ) ≤ d[β(t0 , r)ϕ(r, Xr )] − γeηkk k∞ β(t0 , θ)
t0
Z θ
≤ d[β(t0 , r)ϕ(r, Xr )] − γ.
t0
By Itô’s formula, this provides :

h Z θ i
V (t0 , x0 ) ≥ γ + Et0 ,x0 β(t0 , θ)V (θ, Xθ ) − β(t0 , r)(∂t ϕ + Lνr ϕ)(r, Xr )dr ,
t0
where the ”dW ” integral term has zero mean, as its integrand is bounded on the
stochastic interval [t0 , θ]. Observe also that (∂t ϕ + Lνr ϕ)(r, Xr ) + f (r, Xr , νr ) ≤
h(r, Xr ) ≤ 0 on the stochastic interval [t0 , θ]. We therefore deduce that :
hZ θ i
V (t0 , x0 ) ≥ γ + Et0 ,x0 β(t0 , r)f (r, Xr , νr )dr + β(t0 , θ)V (θ, Xθ ) .
t0
As γ is a positive constant independent of ν, this implies that:

hZ θ i
V (t0 , x0 ) ≥ γ + sup Et0 ,x0 β(t0 , r)f (r, Xr , νr )dr + β(t0 , θ)V (θ, Xθ ) ,
ν∈Ut t0
which is the required contradiction of the second part of the dynamic program-
ming principle, and thus completes the proof. ♦
As a consequence of Propositions 2.4 and 2.5, we have the main result of

this section :
Theorem 2.6. Let the conditions of Propositions 2.5 and 2.4 hold. Then, the
value function V solves the Hamilton-Jacobi-Bellman equation
−∂t V − H ., V, DV, D2 V = 0 on S.

(2.18)
2.4 On the regularity of the value function

The purpose of this paragraph is to show that the value function should not be
expected to be smooth in general. We start by proving the continuity of the
value function under strong conditions; in particular, we require the set U in
which the controls take values to be bounded. We then give a simple example
in the deterministic framework where the value function is not smooth. Since
it is well known that stochastic problems are “more regular” than deterministic
ones, we also give an example of stochastic control problem whose value function
is not smooth.
2.4.1 Continuity of the value function for bounded con-

trols
For notational simplicity, we reduce the stochastic control problem to the case
f = k ≡ 0, see Remark 2.2 (iii). Our main concern, in this section, is to show the
standard argument for proving the continuity of the value function. Therefore,
the following results assume strong conditions on the coefficients of the model
in order to simplify the proofs. We first start by examining the value function
V (t, ·) for fixed t ∈ [0, T ].
2.4. Regularity of the value function 35
Proposition 2.7. Let f = k ≡ 0, T < ∞, and assume that g is Lipschitz

continuous. Then:
(i) V is Lipschitz in x, uniformly in t.
(ii) Assume further that U is bounded. Then V is 21 −Hölder-continuous in t,
and there is a constant C > 0 such that:
p
V (t, x) − V (t0 , x) ≤ C(1 + |x|) |t − t0 |; t, t0 ∈ [0, T ], x ∈ Rn .

Proof. (i) For x, x0 ∈ Rn and t ∈ [0, T ), we first estimate that:

0

|V (t, x) − V (t, x0 )| ≤ sup E g XTt,x,ν − g XTt,x ,ν

ν∈U0
0

≤ Const sup E XTt,x,ν − XTt,x ,ν

ν∈U0
≤ Const |x − x0 |,
where we used the Lipschitz-continuity of g together with the flow estimates
of Theorem 1.4, and the fact that the coefficients b and σ are Lipschitz in x
uniformly in (t, u). This compltes the proof of the Lipschitz property of the
value function V .
(ii) To prove the Hölder continuity in t, we shall use the dynamic programming
principle.
(ii-1) We first make the following important observation. A careful review
of the proof of Theorem 2.3 reveals that, whenever the stopping times θν are
constant (i.e. deterministic), the dynamic programming principle holds true
with the semicontinuous envelopes taken only with respect to the x−variable.
Since V was shown to be continuous in the first part of this proof, we deduce
that:
V (t, x) = sup E V t0 , Xtt,x,ν

0 (2.19)
ν∈U0
for all x ∈ Rn , t < t0 ∈ [0, T ].

(ii-2) Fix x ∈ Rn , t < t0 ∈ [0, T ]. By the dynamic programming principle
(2.19), we have :

|V (t, x) − V (t0 , x)| = sup E V t0 , Xtt,x,ν 0

0 − V (t , x)

ν∈U0
sup E V t0 , Xtt,x,ν − V (t0 , x) .

≤ 0
ν∈U0
By the Lipschitz-continuity of V (s, ·) established in the first part of this proof,

we see that :
|V (t, x) − V (t0 , x)| ≤ Const sup E Xtt,x,ν

0 − x . (2.20)
ν∈U0
We shall now prove that

sup E Xtt,x,ν Const (1 + |x|)|t − t0 |1/2 ,

0 − x ≤ (2.21)
ν∈U
which provides the required (1/2)−Hölder continuity in view of (2.20). By

definition of the process X, and assuming t < t0 , we have
Z 0 Z t0 2
t,x,ν 2 t
E Xt0 − x = E b(r, Xr , νr )dr + σ(r, Xr , νr )dWr

t t
"Z 0 #
t
2
≤ Const E |h(r, Xr , νr )| dr
t
where h := [b2 + σ 2 ]1/2 . Since h is Lipschitz-continuous in (t, x, u) and has

quadratic growth in x and u, this provides:
Z t0 Z t0 !
t,x,ν 2 2 2
t,x,ν 2
E Xt0 − x ≤ Const (1 + |x| + |νr | )dr + E Xr − x dr .
t t
Since the control process ν is uniformly bounded, we obtain by the Gronwall

lemma the estimate:
2
E Xtt,x,ν − x ≤ Const (1 + |x|2 )|t0 − t|,

0 (2.22)
where the constant does not depend on the control ν. ♦
Remark 2.8. When f and/or k are non-zero, the conditions required on f and
k in order to obtain the (1/2)−Hölder continuity of the value function can be
deduced from the reduction of Remark 2.2 (iii).
Remark 2.9. Further regularity results can be proved for the value function
under convenient conditions. Typically, one can prove that Lu V exists in the
generalized sense, for all u ∈ U . This implies immediately that the result of
Proposition 2.5 holds in the generalized sense. More technicalities are needed in
order to derive the result of Proposition 2.4 in the generalized sense. We refer
to [6], §IV.10, for a discussion of this issue.
2.4.2 A deterministic control problem with non-smooth

value function
Let σ ≡ 0, b(x, u) = u, U = [−1, 1], and n = 1. The controlled state is then the
one-dimensional deterministic process defined by :
Z s
Xs = Xt + νt dt for 0 ≤ t ≤ s ≤ T .
t
Consider the deterministic control problem

V (t, x) := sup (XT )2 .
ν∈U
The value function of this problem is easily seen to be given by :

(x + T − t)2 for x ≥ 0 with optimal control û = 1 ,

V (t, x) =
(x − T + t)2 for x ≤ 0 with optimal control û = −1 .
2.4. Regularity of the value function 37
This function is continuous. However, a direct computation shows that it is not

differentiable at x = 0.
2.4.3 A stochastic control problem with non-smooth value

function
Let U = R, and the controlled process X be the scalar process defined by the
dynamics:
dXt = νt dWt ,
where W is a scalar Brownian motion. Let g be a lower semicontinuous mapping

on R, with −α0 −β 0 |x| ≤ g(x) ≤ α+βx, x ∈ R, for some constants α, α0 β, β 0 ∈ R.
We consider the stochastic control problem
V (t, x) := sup Et,x [g(XTν )] .

ν∈U0
Let us assume that V is smooth, and work towards a contradiction.
1. If V is C 1,2 ([0, T ), R), then it follows from Proposition 2.4 that V satisfies
1
−∂t V − u2 D2 V ≥ 0 for all u ∈ R,
2
and all (t, x) ∈ [0, T ) × R. By sending u to infinity, it follows that
V (t, ·) is concave for all t ∈ [0, T ). (2.23)

2. Notice that V (t, x) ≥ Et,x g(XT0 ) = g(x). Then, it follows from (2.23) that:
V (t, x) ≥ g conc (x) for all (t, x) ∈ [0, T ) × R, (2.24)
where g conc is the concave envelope of g, i.e. the smallest concave majorant of
g. Notice that g conc < ∞ as g is bounded from above by a line.
3. Since g ≤ g conc , we see that
V (t, x) := sup Et,x [g(XTν )] ≤ sup Et,x [g conc (XTν )] = g conc (x),
ν∈U0 ν∈U0
by the martingale property of X ν . In view of (2.24), we have then proved that
V ∈ C 1,2 ([0, T ), R)
=⇒ V (t, x) = g conc (x) for all (t, x) ∈ [0, T ) × R.
Now recall that this implication holds for any arbitrary non-negative lower semi-
continuous function g. We then obtain a contradiction whenever the function
g conc is not C 2 (R). Hence
g conc 6∈ C 2 (R) =⇒ V 6∈ C 1,2 ([0, T ), R2 ).

Chapter 3
Optimal Stopping and

Dynamic Programming
As in the previous chapter, we assume here that the filtration F is defined as the
P−augmentation of the canonical filtration of the Brownian motion W defined
on the probability space (Ω, F, P).
Our objective is to derive similar results, as those obtained in the previous
chapter for standard stochastic control problems, in the context of optimal stop-
ping problems. We will then first start by the formulation of optimal stopping
problems, then the corresponding dynamic programming principle, and dynamic
programming equation.
3.1 Optimal stopping problems

For 0 ≤ t ≤ T ≤ ∞, we denote by T[t,T ] the collection of all F−stopping
times with values in [t, T ]. We also recall the notation S := [0, T ) × Rn for the
parabolic state space of the underlying state process X defined by the stochastic
differential equation:
dXt = µ(t, Xt )dt + σ(t, Xt )dWt , (3.1)
n
where µ and σ are defined on S̄ and take values in R and Sn , respectively. We
assume that µ and σ satisfies the usual Lipschitz and linear growth conditions
so that the above SDE has a unique strong solution satisfying the integrability
proved in Theorem 1.2.
The infinitesimal generator of the Markov diffusion process X is denoted by
1
Aϕ := µ · Dϕ + Tr σσ T D2 ϕ .

2
Let g be a measurable function from Rn to R, and assume that:

E sup |g(Xt )| < ∞. (3.2)
0≤t<T
39
40 CHAPTER 3. OPTIMAL STOPPING, DYNAMIC PROGRAMMING
For instance, if g has polynomial growth, the latter integrability condition is

automatically satisfied. Under this condition, the following criterion:
J(t, x, τ ) := E g Xτt,x 1τ <∞

(3.3)
is well-defined for all (t, x) ∈ S and τ ∈ T[t,T ] . Here, X t,x denotes the unique
strong solution of (3.1) with initial condition Xtt,x = x.
The optimal stopping problem is now defined by:
V (t, x) := sup J(t, x, τ ) for all (t, x) ∈ S. (3.4)

τ ∈T[t,T ]
A stopping time τ̂ ∈ T[t,T ] is called an optimal stopping rule if V (t, x) =

J(t, x, τ̂ ).
The set
S := {(t, x) : V (t, x) = g(x)} (3.5)
is called the stopping region and is of particular interest: whenever the state is
in this region, it is optimal to stop immediately. Its complement S c is called
the continuation region.
Remark 3.1. As in the previous chapter, we could have considered an appear-
ently more general criterion
Z τ
t,x

V (t, x) := sup E β(t, s)f (s, Xs )ds + β(t, τ )g Xτ 1τ <∞ ,
τ ∈T[t,T ] t
with
Rs
β(t, s) := e− t
k(s,Xs )ds
for 0 ≤ t ≤ s < T.
However by introducing the additional state

Z t
Yt := Y0 + βs f (s, Xs )ds,
0
Z t
Zt := Z0 + Zs k(s, Xs )ds,
0
we see immediately that we may reduce this problem to the context of (3.4).
Remark 3.2. Consider the subset of stopping rules:
t

T[t,T ] := τ ∈ T[t,T ] : τ independent of Ft . (3.6)
By a similar argument as in Remark 2.2 (iv), we can see that the maximization
in the optimal stopping problem (3.4) can be restricted to this subset, i.e.

t
τ ∈T[t,T ]
41
3.2 The dynamic programming principle

In the context of optimal stopping problems, the proof of the dynamic pro-
gramming principle is easier than in the context of stochastic control problems
of the previous chapter. The reader may consult the excellent exposition in
the book of Karatzas and Shreve [9], Appendix D, where the following dynamic
programming principle is proved:
sup E 1{τ <θ} g(Xτt,x ) + 1{τ ≥θ} V (θ, Xθt,x ) ,

V (t, x) = (3.8)
t
τ ∈T[t,T ]
for all (t, x) ∈ S and τ ∈ T[t,T ] . In particular, the proof in the latter reference
does not require any heavy measurable selection, and is essentially based on the
supermartingale nature of the so-called Snell envelope process. Moreover, we
observe that it does not require any Markov property of the underlying state
process.
We report here a different proof in the sprit of the weak dynamic program-
ming principle for stochastic control problems proved in the previous chapter.
The subsequent argument is specific to our Markovian framework and, in this
sense, is weaker than the classical dynamic programming principle. However,
the combination of the arguments of this chapter with those of the previous
chapter allow to derive a dynamic programming principle for mixed stochastic
control and stopping problem.
t
The following claim will be making using of the subset T[t,T ] , introduced
in (3.6), of all stopping times in T[t,T ] which are independent of Ft , and the
notations:
V∗ (t, x) := lim inf V (t0 , x0 ) and V ∗ (t, x) := lim sup V (t0 , x0 )

(t0 ,x0 )→(t,x) (t0 ,x0 )→(t,x)
for all (t, x) ∈ S̄. We recall that V∗ and V ∗ are the lower and upper semicon-
tinuous envelopes of V , and that V∗ = V ∗ = V whenever V is continuous.
t
Theorem 3.3. Assume that V is locally bounded. For (t, x) ∈ S, let θ ∈ T̄[t,T ]
be a stopping time such that Xθt,x is bounded. Then:
sup E 1{τ <θ} g(Xτt,x ) + 1{τ ≥θ} V ∗ (θ, Xθt,x ) ,

V (t, x) ≤ (3.9)
t
τ ∈T[t,T ]
sup E 1{τ <θ} g(Xτt,x ) + 1{τ ≥θ} V∗ (θ, Xθt,x )) .

V (t, x) ≥ (3.10)
t
τ ∈T[t,T ]
Proof. Inequality (3.9) follows immediately from the tower property and the
fact that J ≤ V ∗ .
We next prove inequality (3.10) with V∗ replaced by an arbitrary function
ϕ : S −→ R such ϕ is upper-semicontinuous and V ≥ ϕ,
which implies (3.10) by the same argument as in Step 3 of the proof of Theorem
2.3.
Arguying as in Step 2 of the proof of Theorem 2.3, we first observe that, for
every ε > 0, we can find a countable family Āi ⊂ (ti − ri , ti ] × Ai ⊂ S, together
with a sequence of stopping times τ i,ε in T[ttii,T ] , i ≥ 1, satisfying Ā0 = {T } × Rd
and
¯ τ i,ε ) ≥ ϕ − 3ε on Āi for i ≥ 1.

∪i≥0 Āi = S, Āi ∩ Āj = ∅ for i 6= j ∈ N, J(·;
(3.11)
Set Ān := ∪i≤n Āi , n ≥ 1. Given two stopping times θ, τ ∈ T[t,T
t
] , it is clear that
n
!
X
τ n,ε := τ 1{τ <θ} + 1{τ ≥θ} θ, Xθt,x τ i,ε 1Āi θ, Xθt,x

T 1(Ān )c +
i=1
t
defines a stopping time in T[t,T ] . We then deduce from the tower property and
(3.11) that
V̄ (t, x) ≥ ¯ x; τ n,ε )
J(t,
E g Xτt,x 1{τ <θ} + 1{τ ≥θ} ϕ(θ, Xθt,x ) − 3ε 1Ān (θ, Xθt,x )

≥
+E 1{τ ≥θ} g(XTt,x )1(Ān )c (θ, Xθt,x ) .

By sending n → ∞ and arguing as in the end of Step 2 of the proof of Theorem

2.3, we deduce that
V̄ (t, x) ≥ E g Xτt,x 1{τ <θ} + 1{τ ≥θ} ϕ(θ, Xθt,x ) − 3ε,

t
and the result follows from the arbitrariness of ε > 0 and τ ∈ T[t,T ]. ♦
3.3 The dynamic programming equation

In this section, we explore the infinitesimal counterpart of the dynamic program-
ming principle of Theorem 3.3, when the value function V is a priori known to
be smooth. The smoothness that will be required in this chapter must be so
that we can apply Itô’s formula to V . In particular, V is continuous, and the
dynamic programming principle of Theorem 3.3 reduces to the classical dynamic
programming principle (3.8).
Loosely speaking, the following dynamic programming equation says the
following:
• In the stopping region S defined in (3.5), continuation is sub-optimal, and

therefore the linear PDE must hold with inequality in such a way that the
value function is a submartingale.
• In the continuation region S c , it is optimal to delay the stopping decision

after some small moment, and therefore the value function must solve a
linear PDE as in Chapter 1.
43
Theorem 3.4. Assume that V ∈ C 1,2 ([0, T ), Rn ), and let g : Rn −→ R be

continuous. Then V solves the obstacle problem:
min {−(∂t + A)V , V − g} = 0 on S. (3.12)
Proof. We organize the proof into two steps.

1. We first show that:
min {−(∂t + A)V , V − g} ≥ 0 on S. (3.13)
The inequality V − g ≥ 0 is obvious as the constant stopping rule τ = t ∈ T[t,T ]

is admissible. Next, for (t0 , x0 ) ∈ S, consider the stopping times
θh := inf t > t0 : (t, Xtt0 ,x0 ) 6∈ [t0 , t0 + h] × B , h > 0,

where B is the unit ball of Rn centered at x0 . Then θh ∈ T[t,T

t
] for sufficiently
small h, and it follows from (3.10)that:
V (t0 , x0 ) ≥ E [V (θh , Xθh )] .
We next apply Itô’s formula, and observe that the expected value of the diffusion
term vanishes because (t, Xt ) lies in the compact subset [t0 , t0 + h] × B for
t ∈ [t0 , θh ]. Then:
" #
−1 θh
Z
t0 ,x0
E (∂t + A)V (t, Xt )dt ≥ 0.
h t0
Clearly, there exists ĥω > 0, depending on ω, θh = h for h ≤ ĥω . Then, it

follows from the mean value theorem that the expression inside the expectation
converges P−a.s. to −(∂t + A)V (t0 , x0 ), and we conclude by dominated conver-
gence that −(∂t + A)V (t0 , x0 ) ≥ 0.
2. In order to complete the proof, we use a contradiction argument, assuming
that
V (t0 , x0 ) > g(x0 ) and − (∂t + A)V (t0 , x0 ) > 0 at some (t0 , x0 ) ∈ S, (3.14)
and we work towards a contradiction of (3.9). Introduce the function

ε
ϕ(t, x) := V (t, x) + |x − x0 |2 for (t, x) ∈ S.
2
Then, it follows from (3.14) that for a sufficiently small ε > 0, we may find
h > 0 and δ > 0 such that
V ≥ g + δ and − (∂t + A)ϕ ≥ 0 on Nh := [t0 , t0 + h] × hB. (3.15)
Moreover:
−γ := max(V − ϕ) < 0. (3.16)

∂Nh
Next, let
t, Xtt0 ,x0 ∈

θ := inf t > t0 : 6 Nh .
t
For an arbitrary stopping rule τ ∈ T[t,T ] , we compute by Itô’s formula that:
E [V (τ ∧ θ, Xτ ∧θ ) − V (t0 , x0 )] = E [(V − ϕ) (τ ∧ θ, Xτ ∧θ )]
+E [ϕ (τ ∧ θ, Xτ ∧θ ) − ϕ(t0 , x0 )]
= E [(V − ϕ) (τ ∧ θ, Xτ ∧θ )]
"Z #
τ ∧θ
+E (∂t + A)ϕ(t, Xtt0 ,x0 )dt ,
t0
where the diffusion term has zero expectation because the process (t, Xtt0 ,x0 ) is
confined to the compact subset Nh on the stochastic interval [t0 , τ ∧ θ]. Since
−Lϕ ≥ 0 on Nh by (3.15), this provides:
E [V (τ ∧ θ, Xτ ∧θ ) − V (t0 , x0 )] ≤ E [(V − ϕ) (τ ∧ θ, Xτ ∧θ )]
≤ −γP[τ ≥ θ],
by (3.16). Then, since V ≥ g + δ on Nh by (3.15):
V (t0 , x0 ) ≥ γP[τ ≥ θ] + E g(Xτt0 ,x0 ) + δ 1{τ <θ} + V θ, Xθt0 ,x0 1{τ ≥θ}

≥ (γ ∧ δ) + E g(Xτt0 ,x0 )1{τ <θ} + V θ, Xθt0 ,x0 1{τ ≥θ} .

t
By the arbitrariness of τ ∈ T[t,T ] , this provides the desired contradiction of (3.9).
♦
3.4 Regularity of the value function

3.4.1 Finite horizon optimal stopping
In this subsection, we consider the case T < ∞. Similar to the continuity result
of Proposition 2.7 for the stochastic control framework, the following continuity
result is obtained as a consequence of the flow continuiy of Theorem 1.4 together
with the dynamic programming principle.
Proposition 3.5. Assume g is Lipschitz-continuous, and let T < ∞. Then,
there is a constant C such that:
p
V (t, x) − V (t0 , x0 ) ≤ C |x − x0 | + |t − t0 | for all (t, x), (t0 , x0 ) ∈ S.

Proof. (i) For t ∈ [0, T ] and x, x0 ∈ Rn , it follows from the Lipschitz property
of g that:

0
|V (t, x) − V (t, x0 )| ≤ Const sup E Xτt,x − Xτt,x

τ ∈T[t,T ]

0
≤ Const E sup Xτt,x − Xτt,x

t≤s≤T
≤ Const |x − x0 |
45
by the flow continuity result of Theorem 1.4.

ii) To rpove the H”older continuity result in t, we argue as in the proof of
Proposition 2.7 using the dynamic programming principle of Theorem 3.3.
(ii-1) We first observe that, whenever the stopping time θ = t0 > t is
constant (i.e. deterministic), the dynamic programming principle (3.9)-(3.10)
holds true if the semicontinuous envelopes are taken with respect to the variable
x, with fixed time variable. Since V is continuous in x by the first part of this
proof, we deduce that
sup E 1{τ <t0 } g Xτt,x + 1{τ ≥t0 } V t0 , Xtt,x

V (t, x) = 0 (3.17)
t
τ ∈T[t,T ]
(ii) We then estimate that
0 ≤ V (t, x) − E V t0 , Xtt,x E 1{τ <t0 } g Xτt,x − V t0 , Xtt,x

0 ≤ sup 0
t
τ ∈T[t,T ]
E 1{τ <t0 } g Xτt,x − g Xtt,x

≤ sup 0 ,
t
τ ∈T[t,T ]
where the last inequality follows from the fact that V ≥ g. Using the Lipschitz
property of g, this provides:

0 t,x t,x
t,x
0 ≤ V (t, x) − E V t , Xt0 ≤ Const E sup Xs − Xt0

t≤s≤t0
√
≤ Const (1 + |x|) t0 − t
by the flow continuity result of Theorem 1.4. Using this estimate together with
the Lipschitz property proved in (i) above, this provides:
|V (t, x) − V (t0 , x)| ≤ V (t, x) − E V t0 , Xtt,x + E V t0 , X t,x − V (t0 , x)

0 t0
√
≤ Const (1 + |x|) t0 − t + E Xtt,x

0 − x
√
≤ Const (1 + |x|) t0 − t,
by using again Theorem 1.4. ♦
3.4.2 Infinite horizon optimal stopping

In this section, the state process X is defined by a homogeneous scalar diffusion:
dXt = µ(Xt )dt + σ(Xt )dWt . (3.18)
We introduce the hitting times:
Hbx := inf t > 0 : X 0,x = b ,

and we assume that the process X is regular, i.e.
P [Hbx < ∞] > 0 for all x, b ∈ R, (3.19)

which means that there is no subinterval of R from which the process X can
not exit.
We consider the infinite horizon optimal stopping problem:
V (x) := sup E e−βτ g Xτ0,x 1{τ <∞} ,

(3.20)
τ ∈T
where T := T[0,∞] , and β > 0 is the discount rate parameter.

According to Theorem 3.3, the dynamic programming equation correspond-
ing to this optimal stopping problem is the obstacle problem:
min {βv − Av, v − g} = 0,
where the differential operator in the present homogeneous context is given by

the generator of the diffusion:
1
Av := µv 0 + σ 2 v 00 . (3.21)
2
The ordinary differential equation
Av − βv = 0 (3.22)
has two positive linearly independent solutions
ψ, φ ≥ 0 such that ψ strictly increasing, φ strictly decreasing. (3.23)
Clearly ψ and φ are uniquely determined up to a positive constant, and all other
solution of (3.22) can be expressed as a linear combination of ψ and φ.
The following result follows from an immediate application of Itô’s formula.
Lemma 3.6. For any b1 < b2 , we have:

h x
i ψ(x)φ(b2 ) − ψ(b2 )φ(x)
E e−βHb1 1{Hbx ≤Hbx } = ,
1 2 ψ(b1 )φ(b2 ) − ψ(b2 )φ(b1 )
h x
i ψ(b1 )φ(x) − ψ(x)φ(b1 )
E e−βHb2 1{Hbx ≥Hbx } = .
1 2 ψ(b1 )φ(b2 ) − ψ(b2 )φ(b1 )
We now show that the value function V is concave up to some change of vari-
able, and provides conditions under which V is C 1 across the exercise boundary,
i.e. the boundary between the exercise and the continuation regions. For the
next result, we observe that the fnction (ψ/φ) is continuous and strictly increas-
ing by (3.23), and therefore invertible.
Theorem 3.7. (i) The function (V /φ) ◦ (ψ/φ)−1 is concave. In particular, V

is continuous on R.
(ii) Let x0 be such that V (x0 ) = g(x0 ), and assume that g, ψ and φ are differ-
entiable at x0 . Then V is differentiable at x0 , and V 0 (x0 ) = g 0 (x0 ).
47
Proof. For (i), it is sufficient to prove that:

V V V V
φ (x) − φ (b1 ) φ (b2 ) − φ (x)
ψ ψ
≥ ψ ψ
for all b1 < x < b 2 . (3.24)
φ (x) − φ (b1 ) φ (b2 ) − φ (x)
For ε > 0, consider the ε−optimal stopping rules τ1 , τ2 ∈ T for the problems
V (b1 ) and V (b2 ):
E e−βτi g Xτ0,b

i
i
≥ V (bi ) − ε for i = 1, 2.
We next define the stopping time

τ ε := Hbx1 + τ1 ◦ θHbx 1{Hbx <Hbx } + Hbx2 + τ2 ◦ θHbx 1{Hbx <Hbx } ,
1 1 2 2 2 1
where θ denotes the shift operator on the canonical space, i.e. θt (ω)(s) =
ω(t + s). In words, the stopping rule τ ε uses the ε−optimal stopping rule τ1 if
the level b1 is reached before the level b2 , and the ε−optimal stopping rule τ2
otherwise. Then, it follows from the strong Markov property that
h ε
i
V (x) ≥ E e−βτ g Xτ0,x ε
h x
i
= E e−βHb1 E e−βτ1 g Xτ0,b

1
1
1{Hbx <Hbx }
1 2
h i
−βHbx2 1
−βτ2 0,b2

+E e E e g Xτ2 1{Hbx <Hbx }
2 1
h i
−βHbx1
≥ (V (b1 ) − ε) E e 1{Hbx <Hbx }
1 2
h i
−βHbx2
+ (V (b2 ) − ε) E e 1{Hbx <Hbx } .
2 1
Sending ε & 0, this provides

h x
i h x
i
V (x) ≥ V (b1 )E e−βHb1 1{Hbx <Hbx } + V (b2 )E e−βHb2 1{Hbx <Hbx } .
1 2 2 1
By using the explicit expressions of Lemma 3.6 above, this provides:

ψ ψ ψ ψ
V (x) V (b1 ) φ (b2 ) − φ (x) V (b2 ) φ (x) − φ (b1 )
≥ ψ ψ
+ ψ ψ
,
φ(x) φ(b1 ) φ (b2 ) − φ (b1 )
φ(b2 ) φ (b2 ) − φ (b1 )
which implies (3.24).

(ii) We next prove the smoothfit result. Let x0 be such that V (x0 ) = g(x0 ).
Then, since V ≥ g, ψ is strictly increasing, φ ≥ 0 is strictly decreasing, it follows
from (3.24) that:
g
φ (x0 + ε) − φg (x0 ) V
φ (x0 + ε) − V
φ (x0 )
ψ ψ
≤ ψ ψ
(3.25)
φ (x0 + ε) − φ (x0 ) φ (x0 + ε) − φ (x0 )
g
V
φ (x0 − δ) − V
φ (x0 ) φ (x0 − δ) − φg (x0 )
≤ ψ ψ
≤ ψ ψ
φ (x0 − δ) − φ (x0 ) φ (x0 − δ) − φ (x0 )
for all ε > 0, δ > 0. Multiplying by ((ψ/φ)(x0 + ε) − (ψ/φ)(x0 ))/ε, this implies
that:
g
φ (x0 + ε) − φg (x0 ) V
φ (x0 + ε) − V
φ (x0 ) ∆+ (ε)
g
φ (x0 − δ) − φg (x0 )
≤ ≤ ,
ε ε ∆− (δ) δ
(3.26)
where
ψ ψ ψ ψ
φ (x0 + ε) − φ (x0 ) φ (x0 − δ) − φ (x0 )
∆+ (ε) := and ∆− (δ) := .
ε δ
We next consider two cases:
• If (ψ/φ)0 (x0 ) 6= 0, then we may take ε = δ and send ε & 0 in (3.26) to

obtain:
d+ ( Vφ )
0
g
(x0 ) = (x0 ). (3.27)
dx φ
• If (ψ/φ)0 (x0 ) = 0, then, we use the fact that for every sequence εn & 0,
there is a subsequence εnk & 0 and δk & 0 such that ∆+ (εnk ) = ∆− (δk ).
Then (3.26) reduces to:
g
φ (x0 + εnk ) − φg (x0 ) V
φ (x0 + ε nk ) − V
φ (x0 )
g
φ (x0 − δk ) − φg (x0 )
≤ ≤ ,
ε nk εnk δk
and therefore
V V 0
φ (x0 + ε nk ) − φ (x0 ) g
−→ (x0 ).
εnk φ
By the arbitrariness of the sequence (εn )n , this provides (3.27).
Similarly, multiplying (3.25) by ((ψ/φ)(x0 ) − (ψ/φ)(x0 − δ))/δ, and arguying as

above, we obtain:
d− ( Vφ )
0
g
(x0 ) = (x0 ),
dx φ
thus completing the proof. ♦
3.4.3 An optimal stopping problem with nonsmooth value

We consider the example
Xst,x := x + (Wt − Ws ) for s ≥ t.

49
Let g : R −→ R+ be a measurable nonnegative function with lim inf x→∞ g(x) =

0, and consider the infinite horizon optimal stopping problem:
sup E g Xτt,x 1{τ <∞}

V (t, x) :=
τ ∈T[t,∞]
E g Xτt,x .

= sup
τ ∈T[t,∞)
Let us assume that V ∈ C 1,2 (S), and work towards a contradiction. We first
observe by the homogeneity of the problem that V (t, x) = V (x) is independent
of t. Moreover, it follows from Theorem 3.4 that V is concave in x and V ≥ g.
Then
V ≥ g conc , (3.28)
where g conc is the concave envelope of g. If g conc = ∞, then V = ∞. We then

continue in the more inetersting case where g conc < ∞.
By the Jensen inequality and the non-negativity of g, the process {g (Xst,x ) , s ≥ t}
is a supermartingale, and:
sup E g conc Xτt,x ≤ g conc (x).

V (t, x) ≤
τ ∈T[t,T ]
Hence, V = g conc , and we obtain the required contradiction whenever g conc is

not differentiable at some point of R.
Chapter 4
Solving Control Problems

by Verification
In this chapter, we present a general argument, based on Itô’s formula, which

allows to show that some ”guess” of the value function is indeed equal to the
unknown value function. Namely, given a smooth solution v of the dynamic
programming equation, we give sufficient conditions which allow to conclude
that v coincides with the value function V . This is the so-called verification
argument. The statement of this result is heavy, but its proof is simple and relies
essentially on Itô’s formula. However, depending on the problem in hand, the
verification of the conditions which must be satisfied by the candidate solution
can be difficult.
The verification argument will be provided in the contexts of stochastic
control and optimal stopping problems. We conclude the chapter with some
examples.
4.1 The verification argument for stochastic con-

trol problems
We recall the stochastic control problem formulation of Section 2.1. The set of
admissible control processes U0 ⊂ U is the collection of all progressively measur-
able processes with values in the subset U ⊂ Rk . For every admissible control
process ν ∈ U0 , the controlled process is defined by the stochastic differential
equation:
dXtν = b(t, Xtν , νt )dt + σ(t, Xtν , νt )dWt .
The gain criterion is given by

"Z #
T
J(t, x, ν) := E β ν (t, s)f (s, Xst,x,ν , νs )ds + β ν (t, T )g(XTt,x,ν ) ,
t
51
52 CHAPTER 4. THE VERIFICATION ARGUMENT
with
Rs
k(r,Xrt,x,ν ,νr )dr
β ν (t, s) := e− t .
The stochastic control problem is defined by the value function:
V (t, x) := sup J(t, x, ν), for (t, x) ∈ S. (4.1)

ν∈U0
We follow the notations of Section 2.3. We recall the Hamiltonian H : S × R ×

Rd × Sd defined by :
H(t, x, r, p, γ)

1
:= sup −k(t, x, u)r + b(t, x, u) · p + Tr[σσ T (t, x, u)γ] + f (t, x, u) ,
u∈U 2
where b and σ satisfy the conditions (2.1)-(2.2), and the coefficients f and k are
measurable. From the results of the previous section, the dynamic programming
equation corresponding to the stochastic control problem (4.1) is:
−∂t v − H(., v, Dv, D2 v) = 0 and v(T, .) = g. (4.2)
A function v will be called a supersolution (resp. subsolution) of the equation

(4.2) if
−∂t v − H(., v, Dv, D2 v) ≥ (resp. ≤) 0 and v(T, .) ≥ (resp. ≤) g.
The proof of the subsequent result will make use of the following linear second
order operator
Lu ϕ(t, x) := −k(t, x, u)ϕ(t, x) + b(t, x, u) · Dϕ(t, x)

1
+ Tr σσ T (t, x, u)D2 ϕ(t, x) ,

2
which corresponds to the controlled process {β u (0, t)Xtu , t ≥ 0} controlled by
the constant control process u, in the sense that
Z s
β ν (0, s)ϕ(s, Xsν ) − β ν (0, t)ϕ(t, Xtν ) = β ν (0, r) (∂t + Lνr ) ϕ(r, Xrν )dr
t
Z s
+ β ν (0, r)Dϕ(r, Xrν ) · σ(r, Xrν , νr )dWr
t
for every t ≤ s and smooth function ϕ ∈ C 1,2 ([t, s], Rd ) and each admissible
control process ν ∈ U0 . The last expression is an immediate application of Itô’s
formula.
Theorem 4.1. Let T < ∞, and v ∈ C 1,2 ([0, T ), Rd ) ∩ C([0, T ] × Rd ). Assume
that kk − k∞ < ∞ and v and f have quadratic growth, i.e. there is a constant C
such that
|f (t, x, u)| + |v(t, x)| ≤ C(1 + |x|2 + |u|), (t, x, u) ∈ [0, T ) × Rd × U.

4.1. Verification in stochastic control 53
(i) Suppose that v is a supersolution of (4.2). Then v ≥ V on [0, T ] × Rd .

(ii) Let v be a solution of (4.2), and assume that there exists a minimizer û(t, x)
of u 7−→ Lu v(t, x) + f (t, x, u) such that

• 0 = ∂t v(t, x) + Lû(t,x) v(t, x) + f t, x, û(t, x) ,
• the stochastic differential equation
dXs = b (s, Xs , û(s, Xs )) ds + σ (s, Xs , û(s, Xs )) dWs
defines a unique solution X for each given initial data Xt = x,
• the process ν̂s := û(s, Xs ) is a well-defined control process in U0 .
Then v = V , and ν̂ is an optimal Markov control process.
Proof. Let ν ∈ U0 be an arbitrary control process, X the associated state process
with initial date Xt = x, and define the stopping time
θn := (T − n−1 ) ∧ inf {s > t : |Xs − x| ≥ n} .
By Itô’s formula, we have
Z θn
v(t, x) = β(t, θn )v (θn , Xθn ) − β(t, r)(∂t + Lνr )v(r, Xr )dr
t
Z θn
− β(t, r)Dv(r, Xr ) · σ(r, Xr , νr )dWr
t
Observe that (∂t + Lνr )v + f (·, ·, u) ≤ ∂t v + H(·, ·, v, Dv, D2 v) ≤ 0, and that

the integrand in the stochastic integral is bounded on [t, θn ], a consequence of
the continuity of Dv, σ and the condition kk − k∞ < ∞. Then :
" #
Z θn
v(t, x) ≥ E β(t, θn )v (θn , Xθn ) + β(t, r)f (r, Xr , νr )dr . (4.3)
t
We now take the limit as n increases to infinity. Since θn −→ T a.s. and

Z θn

β(t, θn )v (θn , Xθn ) + β(t, r)f (r, Xr , νr )dr

t
− T
≤ CeT kk k∞ (1 + |Xθn |2 + T + t |Xs |2 ds)
R
− RT
≤ CeT kk k∞ (1 + T )(1 + supt≤s≤T |Xs |2 + t |νs |2 ds) ∈ L1 ,
by the estimate (2.5) of Theorem 2.1, it follows from the dominated convergence
that
" Z T #
v(t, x) ≥ E β(t, T )v(T, XT ) + β(t, r)f (r, Xr , νr )dr
t
" #
Z T
≥ E β(t, T )g(XT ) + β(t, r)f (r, Xr , νr )dr ,
t
where the last inequality uses the condition v(T, ·) ≥ g. Since the control ν ∈ U0
is arbitrary, this completes the proof of (i).
Statement (ii) is proved by repeating the above argument and observing that
the control ν̂ achieves equality at the crucial step (4.3). ♦
Remark 4.2. When U is reduced to a singleton, the optimization problem V is
degenerate. In this case, the DPE is linear, and the verification theorem reduces
to the so-called Feynman-Kac formula.
Notice that the verification theorem assumes the existence of such a solution,
and is by no means an existence result. However, it provides uniqueness in the
class of functions with quadratic growth.
We now state without proof an existence result for the DPE together with
the terminal condition V (T, ·) = g (see [8] and the references therein). The main
assumption is the so-called uniform parabolicity condition :
there is a constant c > 0 such that
(4.4)
ξ 0 σσ 0 (t, x, u) ξ ≥ c|ξ|2 for all (t, x, u) ∈ [0, T ] × Rn × U .
In the following statement, we denote by Cbk (Rn ) the space of bounded functions
whose partial derivatives of orders ≤ k exist and are bounded continuous. We
similarly denote by Cbp,k ([0, T ], Rn ) the space of bounded functions whose partial
derivatives with respect to t, of orders ≤ p, and with respect to x, of order ≤
k, exist and are bounded continuous.
Theorem 4.3. Let Condition 4.4 hold, and assume further that :
• U is compact;
• b, σ and f are in Cb1,2 ([0, T ], Rn );
• g ∈ Cb3 (Rn ).
Then the DPE (2.18) with the terminal data V (T, ·) = g has a unique solution
V ∈ Cb1,2 ([0, T ] × Rn ).
4.2 Examples of control problems with explicit

solutions
4.2.1 Optimal portfolio allocation
We now apply the verification theorem to a classical example in finance, which
was introduced by Merton [10, 11], and generated a huge literature since then.
Consider a financial market consisting of a non-risky asset S 0 and a risky
one S. The dynamics of the price processes are given by
dSt0 = St0 rdt and dSt = St [µdt + σdWt ] .
Here, r, µ and σ are some given positive constants, and W is a one-dimensional

Brownian motion.
The investment policy is defined by an F−adapted process π = {πt , t ∈
[0, T ]}, where πt represents the amount invested in the risky asset at time t;
4.2. Examples 55
The remaining wealth (Xt − πt ) is invested in the risky asset. Therefore, the
liquidation value of a self-financing strategy satisfies
dSt dS 0
dXtπ = πt + (Xtπ − πt ) 0t
St St
= (rXt + (µ − r)πt ) dt + σπt dWt . (4.5)
Such a process π is said to be admissible if it lies in U0 = H2 which will be

refered to as the set of all admissible portfolios. Observe that, in view of the
particular form of our controlled process X, this definition agrees with (2.4).
Let γ be an arbitrary parameter in (0, 1) and define the power utility func-
tion :
U (x) := xγ for x≥0.
The parameter γ is called the relative risk aversion coefficient.

The objective of the investor is to choose an allocation of his wealth so as to
maximize the expected utility of his terminal wealth, i.e.
V (t, x) := sup E U (XTt,x,π ) ,

π∈U0
where X t,x,π is the solution of (4.5) with initial condition Xtt,x,π = x.

The dynamic programming equation corresponding to this problem is :
∂w
(t, x) + sup Au w(t, x) = 0, (4.6)
∂t u∈R
where Au is the second order linear operator :
∂w 1 ∂2w
Au w(t, x) := (rx + (µ − r)u) (t, x) + σ 2 u2 (t, x).
∂x 2 ∂x2
We next search for a solution of the dynamic programming equation of the form
v(t, x) = xγ h(t). Plugging this form of solution into the PDE (4.6), we get the
following ordinary differential equation on h :
u2

u 1
0 = h0 + γh sup r + (µ − r) + (γ − 1)σ 2 2 (4.7)
u∈R x 2 x

1
= h0 + γh sup r + (µ − r)δ + (γ − 1)σ 2 δ 2 (4.8)
δ∈R 2
1 (µ − r)2

= h0 + γh r + , (4.9)
2 (1 − γ)σ 2
where the maximizer is :

µ−r
û := x.
(1 − γ)σ 2
Since v(T, ·) = U (x), we seek for a function h satisfying the above ordinary
differential equation together with the boundary condition h(T ) = 1. This
induces the unique candidate:
1 (µ − r)2

a(T −t)
h(t) := e with a := γ r + .
2 (1 − γ)σ 2
Hence, the function (t, x) 7−→ xγ h(t) is a classical solution of the HJB equation
(4.6). It is easily checked that the conditions of Theorem 4.1 are all satisfied in
this context. Then V (t, x) = xγ h(t), and the optimal portfolio allocation policy
is given by the linear control process:
µ−r
π̂t = X π̂ .
(1 − γ)σ 2 t
4.2.2 Law of iterated logarithm for double stochastic in-

tegrals
The main object of this paragraph is Theorem 4.5 below, reported from [2],
which describes the local behavior of double stochastic integrals near the starting
point zero. This result will be needed in the problem of hedging under gamma
constraints which will be discussed later in these notes. An interesting feature
of the proof of Theorem 4.5 is that it relies on a verification argument. However,
the problem does not fit exactly in the setting of Theorem 4.1. Therefore, this
is an interesting exercise on the verification concept.
Given a bounded predictable process b, we define the processes
Z t Z t
b b
Yt := Y0 + br dWr and Zt := Z0 + Yrb dWr , t ≥ 0 ,
0 0
where Y0 and Z0 are some given initial data in R.

Lemma 4.4. Let λ and T be two positive parameters with 2λT < 1. Then :
h b
i h 1
i
E e2λZT ≤ E e2λZT for each predictable process b with kbk∞ ≤ 1 .
Proof. We split the argument into three steps.

1. We first directly compute that
h i
1
E e2λZT Ft = v(t, Yt1 , Zt1 ) ,
where, for t ∈ [0, T ], and y, z ∈ R, the function v is given by :

" ( Z T )!#
v(t, y, z) := E exp 2λ z + (y + Wu − Wt ) dWu
t
e E exp λ{2yWT −t + WT2 −t − (T − t)}

2λz

=
µ exp 2λz − λ(T − t) + 2µ2 λ2 (T − t)y 2 ,

=
4.2. Examples 57
where µ := [1 − 2λ(T − t)]−1/2 . Observe that
the function v is strictly convex in y, (4.10)
and
2
yDyz v(t, y, z) = 8µ2 λ3 (T − t) v(t, y, z) y 2 ≥ 0 . (4.11)
β
b b
arbitrary real parameter β, we denote by A the generator the process
2. For an
Y ,Z :
1 2 2 1
Aβ := β Dyy + y 2 Dzz
2 2
+ βyDyz .
2 2
In this step, we intend to prove that for all t ∈ [0, T ] and y, z ∈ R :
max Aβ v(t, y, z) = A1 v(t, y, z) = 0. (4.12)

|β|≤1
The second equality follows from the fact that {v(t, Yt1 , Zt1 ), t ≤ T } is a mar-
tingale . As for the first equality, we see from (4.10) and (4.11) that 1 is a
maximizer of both functions β 7−→ β 2 Dyy
2
v(t, y, z) and β 7−→ βyDyz 2
v(t, y, z) on
[−1, 1].
3. Let b be some given predictable process valued in [−1, 1], and define the
sequence of stopping times
τk := T ∧ inf t ≥ 0 : (|Ytb | + |Ztb | ≥ k , k ∈ N .

By Itô’s lemma and (4.12), it follows that :

Z τk
b b
[bDy v + yDz v] t, Ytb , Ztb dWt

v(0, Y0 , Z0 ) = v τk , Yτk , Zτk −
0
Z τk
(∂t + Abt )v t, Ytb , Ztb dt

−
0
Z τk
≥ v τk , Yτbk , Zτbk − [bDy v + yDz v] t, Ytb , Ztb dWt .

0
Taking expected values and sending k to infinity, we get by Fatou’s lemma :
v(0, Y0 , Z0 ) ≥ lim inf E v τk , Yτbk , Zτbk

k→∞
h b
i
≥ E v T, YTb , ZTb = E e2λZT ,

which proves the lemma. ♦
We are now able to prove the law of the iterated logarithm for double stochas-
tic integrals by a direct adaptation of the case of the Brownian motion. Set
1
h(t) := 2t log log for t > 0 .
t
Theorem 4.5. Let b be a predictable process valued in a bounded interval [β0 , β1 ]

RtRu
for some real parameters 0 ≤ β0 < β1 , and Xtb := 0 0 bv dWv dWu . Then :
2Xtb
β0 ≤ lim sup ≤ β1 a.s.
t&0 h(t)
Proof. We first show that the first inequality is an easy consequence of the
second one. Set β̄ := (β0 + β1 )/2 ≥ 0, and set δ := (β1 − β0 )/2. By the law of
the iterated logarithm for the Brownian motion, we have
2Xtβ̄ 2Xtb̃ 2Xtb

β̄ = lim sup ≤ δ lim sup + lim sup ,
t&0 h(t) t&0 h(t) t&0 h(t)
where b̃ := δ −1 (β̄ − b) is valued in [−1, 1]. It then follows from the second
inequality that :
2Xtb
lim sup ≥ β̄ − δ = β0 .
t&0 h(t)
We now prove the second inequality. Clearly, we can assume with no loss of
generality that kbk∞ ≤ 1. Let T > 0 and λ > 0 be such that 2λT < 1. It
follows from Doob’s maximal inequality for submartingales that for all α ≥ 0,

P max 2Xtb ≥ α = P max exp(2λXtb ) ≥ exp(λα)
0≤t≤T 0≤t≤T
h b
i
≤ e−λα E e2λXT .
In view of Lemma 4.4, this provides :

h i
1
b
P max 2Xt ≥ α ≤ e−λα E e2λXT
0≤t≤T
1
= e−λ(α+T ) (1 − 2λT )− 2 . (4.13)
We have then reduced the problem to the case of the Brownian motion, and
the rest of this proof is identical to the first half of the proof of the law of the
iterated logarithm for the Brownian motion. Take θ, η ∈ (0, 1), and set for all
k ∈ N,
αk := (1 + η)2 h(θk ) and λk := [2θk (1 + η)]−1 .
Applying (4.13), we see that for all k ∈ N,

1
P max 2Xt ≥ (1 + η) h(θ ) ≤ e−1/2(1+η) 1 + η −1 2 (−k log θ)−(1+η) .
b 2 k
0≤t≤θ k
Since k≥0 k −(1+η) < ∞, it follows from the Borel-Cantelli lemma that, for
P
almost all ω ∈ Ω, there exists a natural number K θ,η (ω) such that for all
k ≥ K θ,η (ω),
max 2Xtb (ω) < (1 + η)2 h(θk ) .
0≤t≤θ k
4.2. Examples 59
In particular, for all t ∈ (θk+1 , θk ],
h(t)
2Xtb (ω) < (1 + η)2 h(θk ) ≤ (1 + η)2 .
θ
Hence,
2Xtb (1 + η)2
lim sup < a.s.
t&0 h(t) θ
and the required result follows by letting θ tend to 1 and η to 0 along the
rationals. ♦
4.3 The verification argument for optimal stop-

ping problems
In this section, we develop the verification argument for finite horizon optimal
stopping problems. Let T > 0 be a finite time horizon, and X t,x denote the
solution of the stochastic differential equation:
Z s Z s
t,x t,x
Xs = x + b(s, Xs )ds + σ(s, Xst,x )dWs , (4.14)
t t
where b and σ satisfy the usual Lipschitz and linear growth conditions. Given
the functions k, f : [0, T ] × Rd −→ R and g : Rd −→ R, we consider the optimal
stopping problem
Z τ
V (t, x) := sup E β(t, s)f (s, Xst,x )ds + β(t, τ )g(Xτt,x ) , (4.15)
t
τ ∈T[t,T t
]
whenever this expected value is well-defined, where

Rs
k(r,Xrt,x )dr
β(t, s) := e− t , 0 ≤ t ≤ s ≤ T.
By the results of the previous chapter, the corresponding dynamic programmin

equation is:
min {−∂t v − Lv − f, v − g} = 0 on [0, T ) × Rd , v(T, .) = g, (4.16)
where L is the second order differential operator

1
:= b · Dv + Tr σσ T D2 v − kv.

Lv
2
Similar to Section 4.1, a function v will be called a supersolution (resp. subso-
lution) of (4.16) if
min {−∂t v − Lv − f, v − g} ≥ (resp. ≤) 0 and v(T, .) ≥ (resp. ≤) g.

Before stating the main result of this section, we observe that for many inter-
esting examples, it is known that the value function V does not satisfy the C 1,2
regularity which we have been using so far for the application of Itô’s formula.
Therefore, in order to state a result which can be applied to a wider class of
problems, we shall enlarge in the following remark the set of function for which
Itô’s formula still holds true.
Remark 4.6. Let v be a function in the Sobolev space W 1,2 (S). By definition,
for such a function v, there is a sequence of functions (v n )n≥1 ⊂ C 1,2 (S) such
that v n −→ v uniformly on compact subsets of S, and
k∂t v n − ∂t v m kL2 (S) + kDv n − Dv m kL2 (S) + kD2 v n − D2 v m kL2 (S) −→ 0.
Then, Itô’s formula holds true for v n for all n ≥ 1, and is inherited by v by
sending n → ∞.
Theorem 4.7. Let T < ∞ and v ∈ W 1,2 ([0, T ), Rd ). Assume further that v
and f have quadratic growth. Then:
(i) If v is a supersolution of (4.16), then v ≥ V .
(ii) If v is a solution of (4.16), then v = V and
τt∗ := inf {s > t : v(s, Xs ) = g(Xs )}
is an optimal stopping time.

Proof. Let (t, x) ∈ [0, T ) × Rd be fixed and denote βs := β(t, s).
t
(i) For an arbitrary stopping time τ ∈ T[t,T ) , we denote
:= τ ∧ inf s > t : |Xst,x − x| > n .

τn
By our regularity conditions on v, notice that Itô’s formula can be applied to it

piecewise. Then:
Z τn Z τn
v(t, x) = βτn v(τn , Xτt,x
n
)− β s (∂t + L)v(s, Xs
t,x
)ds− βs (σ T Dv)(s, Xst,x )dWs
t t
Z τn Z τn
≥ βτn v(τn , Xτn ) + βs f (s, Xs )ds − βs (σ T Dv)(s, Xst,x )dWs
t,x t,x
t t
by the supersolution property of v. Since (s, Xst,x ) is bounded on the stochastic

interval [t, τn ], this provides:
h Z τn i
v(t, x) ≥ E βτn v(τn , Xτt,x
n
) + β s f (s, Xs
t,x
)ds .
t
Notice that τn −→ τ a.s. Then, since f and v have quadratic growth, we may
pass to the limit n → ∞ invoking the dominated convergence theorem, and we
get:
h Z T i
v(t, x) ≥ E βT v(T, XTt,x ) + βs f (s, Xst,x )ds .
t
4.2. Examples 61
Since v(T, .) ≥ g by the supersolution property, this concludes the proof of (i).
(ii) Let τt∗ be the stopping time introduced in the theorem. Then, since v(T, .) =
g, it follows that τt∗ ∈ T[t,T
t
] . Set
τtn := τt∗ ∧ inf{s > t : |Xst,x − x| > n .

Observe that v > g on [t, τtn ) ⊂ [t, τt∗ ) and therefore −∂t v − Lv − f = 0 on
[t, τtn ). Then, proceeding as in the previous step, it follows from Itô’s formula
that:
h Z τtn i
v(t, x) = E βτtn v(τtn , Xτt,x
n ) +
t
β s f (s, Xs
t,x
)ds .
t
Since τtn −→ τt∗ a.s. and f, v have quadratic growth, we may pass to the limit
n → ∞ invoking the dominated convergence theorem. This leads to:
h Z T i
t,x
v(t, x) = E βT v(T, XT ) + βs f (s, Xst,x )ds ,
t
and the required result follows from the fact that v(T, .) = g. ♦
4.4 Examples of optimal stopping problems with

explicit solutions
4.4.1 Perpetual American options
The pricing problem of perpetual American put options reduces to the infinite
horizon optimal stopping problem:
sup E e−r(τ −t) (K − Sτt,s )+ ,

P (t, s) :=
t
τ ∈T[t,∞)
where K > 0 is a given exercise price, S t,s is defined by the Black-Scholes

constant coefficients model:
σ2
Sut,s := s exp r − (u − t) + σ(Wu − Wt ), u ≥ t,
2
and r ≥ 0, σ > 0 are two given constants. By the time-homogeneity of the
problem, we see that
P (t, s) = P (s) := sup E e−rτ (K − Sτ0,s )+ .

(4.17)
τ ∈T[0,∞)
In view this time independence, it follows that the dynamic programming cor-
responding to this problem is:
1
min{v − (K − s)+ , rv − rsDv − σ 2 D2 v} = 0. (4.18)
2
In order to proceed to a verification argument, we now guess a solution to the

previous obstacle problem. From the nature of the problem, we search for a
solution of this obstacle problem defined by a parameter s0 ∈ (0, K) such that:
1
p(s) = K − s for s ∈ [0, s0 ] and rp − rsp0 − σ 2 s2 p00 = 0 on [s0 , ∞).
2
We are then reduced to solving a linear second order ODE on [s0 , ∞), thus
determining v by
2
p(s) = As + Bs−2r/σ for s ∈ [s0 , ∞),
up to the two constants A and B. Notice that 0 ≤ p ≤ K. Then the constant

A = 0 in our candidate solution, because otherwise v −→ ∞ at infinity. We
finally determine the constants B and s0 by requiring our candidate solution to
be continuous and differentiable at s∗ . This provides two equations:
−2r/σ 2 −2r/σ 2 −2r/σ2 −1

Bs0 = K − s0 and s0 = −1,
B
which provide our final candidate
−2r
σ 2 s0

2rK s σ2
s0 = , p(s) = (K − s)1[0,s ] (s) + 1 [s ,∞) (s) . (4.19)

2r + σ 2 0 0
2r s0
Notice that our candidate p is not twice differentiable at s0 as p00 (s0 −) = 0 6=
p00 (s0 +). However, by Remark 4.6, Itô’s formula still applies to p, and p satisfies
the dynamic programming equation (4.18). We now show that
p = P with optimal stopping time τ ∗ := inf t > 0 : p(St0,s ) = (K − St0,s )+ .

(4.20)
Indeed, for an arbitrary stopping time τ ∈ T[0,∞) , it follows from Itô’s formula
that:
Z τ Z τ
1
p(s) = e−rτ p(Sτ0,s ) − e−rt (−rp + rsp0 + σ 2 s2 p00 )(St )dt − p0 (St )σSt dWt
0 2 0
Z τ
−rτ
≥ e t,s +
(K − Sτ ) − p0 (St )σSt dWt
0
by the fact that p is a supersolution of the dynamic programming equation.

Since p0 is bounded, there is no need to any localization to get rid of the
stochastic integral, and we directly obtain by taking expected values that p(s) ≥
E[e−rτ (K − Sτt,s )+ ]. By the arbitrariness of τ ∈ T[0,∞) , this shows that p ≥ P .
We next repeat the same argument with the stopping time τ ∗ , and we see
∗
that p(s) = E[e−rτ (K − Sτ0,s +
∗ ) ], completing the proof of (4.20).
4.4.2 Finite horizon American options

Finite horizon optimal stopping problems rarely have an explicit solution. So the
following example can be seen as a sanity check. In the context of the financial
4.2. Examples 63
market of the previous subsection, we assume the instanteneous interest rate

r = 0, and we consider an American option with payoff function g and maturity
T > 0. Then the price of the corresponding American option is given by the
optimal stopping problem:
sup E g(Sτt,s ) .

P (t, s) := (4.21)
t
τ ∈T[t,T ]
The corresponding dynamic programming equation is:

1
min v − g, −∂t v − D2 v = 0 on [0, T ) × R+

and v(T, .) = g.(4.22)
2
Assuming further that g ∈ W 1,2 and concave, we see that g is a solution of
the dynamic programming equation. Then, provided that g satisfies suitable
growth condition, we see by a verification argument that P = p.
Notice that the previous result can be obtained directly by the Jensen in-
equality together with the fact that S is a martingale.
Chapter 5
Introduction to Viscosity
Solutions
Throughout this chapter, we provide the main tools from the theory of viscosity
solutions for the purpose of our applications to stochastic control problems. For
a deeper presentation, we refer to the excellent overview paper by Crandall,
Ischii and Lions [3].
5.1 Intuition behind viscosity solutions

We consider a non-linear second order partial differential equation

(E) F x, u(x), Du(x), D2 u(x) = 0 for x ∈ O
where O is an open subset of Rd and F is a continuous map from O×R×Rd ×Sd

−→ R. A crucial condition on F is the so-called ellipticity condition :
Standing Assumption For all (x, r, p) ∈ O × R × Rd and A, B ∈ Sd :
F (x, r, p, A) ≤ F (x, r, p, B) whenever A ≥ B.
The full importance of this condition will be made clear in Proposition 5.2 below.
The first step towards the definition of a notion of weak solution to (E) is
the introduction of sub and supersolutions.
Definition 5.1. A function u : O −→ R is a classical supersolution (resp.
subsolution) of (E) if u ∈ C 2 (O) and
F x, u(x), Du(x), D2 u(x) ≥ (resp. ≤) 0 for x ∈ O .

The theory of viscosity solutions is motivated by the following result, whose

simple proof is left to the reader.
65
66 CHAPTER 5. VISCOSITY SOLUTIONS
Proposition 5.2. Let u be a C 2 (O) function. Then the following claims are
equivalent.
(i) u is a classical supersolution (resp. subsolution) of (E)
(ii) for all pairs (x0 , ϕ) ∈ O × C 2 (O) such that x0 is a minimizer (resp. maxi-
mizer) of the difference u − ϕ) on O, we have
F x0 , u(x0 ), Dϕ(x0 ), D2 ϕ(x0 ) ≥ (resp. ≤) 0 .

5.2 Definition of viscosity solutions

For the convenience of the reader, we recall the definition of the semicontinuous
envelopes. For a locally bounded function u : O −→ R, we denote by u∗
and u∗ the lower and upper semicontinuous envelopes of u. We recall that
u∗ is the largest lower semicontinuous minorant of u, u∗ is the smallest upper
semicontinuous majorant of u, and
u∗ (x) = lim
0
inf u(x0 ) , u∗ (x) = lim sup u(x0 ) .
x →x x0 →x
We are now ready for the definition of viscosity solutions. Observe that Claim
(ii) in the above proposition does not involve the regularity of u. It therefore
suggests the following weak notion of solution to (E).
Definition 5.3. Let u : O −→ R be a locally bounded function.
(i) We say that u is a (discontinuous) viscosity supersolution of (E) if
F x0 , u∗ (x0 ), Dϕ(x0 ), D2 ϕ(x0 ) ≥ 0

for all pairs (x0 , ϕ) ∈ O × C 2 (O) such that x0 is a minimizer of the difference
(u∗ − ϕ) on O.
(ii) We say that u is a (discontinuous) viscosity subsolution of (E) if
F x0 , u∗ (x0 ), Dϕ(x0 ), D2 ϕ(x0 ) ≤ 0

for all pairs (x0 , ϕ) ∈ O × C 2 (O) such that x0 is a maximizer of the difference
(u∗ − ϕ) on O.
(iii) We say that u is a (discontinuous) viscosity solution of (E) if it is both a
viscosity supersolution and subsolution of (E).

Notation We will say that F x, u∗ (x), Du∗ (x), D2 u∗ (x) ≥ 0 in the viscosity
sense whenever u∗ is a viscosity supersolution of (E). A similar notation will be
used for subsolution.
Remark 5.4. An immediate consequence of Proposition 5.2 is that any classical
solution of (E) is also a viscosity solution of (E).
Remark 5.5. Clearly, the above definition is not changed if the minimum or
maximum are local and/or strict. Also, by a density argument, the test function
can be chosen in C ∞ (O).
5.3. First properties 67
Remark 5.6. Consider the equation (E+ ): |u0 (x)| − 1 = 0 on R. Then
• The function f (x) := |x| is not a viscosity supersolution of (E+ ). Indeed

the test function ϕ ≡ 0 satisfies (f − ϕ)(0) = 0 ≤ (f − ϕ)(x) for all x ∈ R.
But |ϕ0 (0)| = 0 6≥ 1.
• The function g(x) := −|x| is a viscosity solution of (E+ ). To see this, we

concentrate on the origin which is the only critical point. The supersolu-
tion property is obviously satisfied as there is no smooth function which
satisfies the minimum condition. As for the subsolution property, we ob-
serve that whenever ϕ ∈ C 1 (R) satisfies (g − ϕ)(0) = max(g − ϕ), then
|ϕ0 (0)| ≥ 1, which is exactly the viscosity subsolution property of g.
• Similarly, the function f is a viscosity solution of the equation (E− ):

−|u0 (x)| + 1 = 0 on R.
In Section 6.1, we will show that the value function V is a viscosity solution
of the DPE (2.18) under the conditions of Theorem 2.6 (except the smoothness
assumption on V ). We also want to emphasize that proving that the value
function is a viscosity solution is almost as easy as proving that it is a classical
solution when V is known to be smooth.
The main difficulty in the theory of viscosity solutions is the interpretation
of the equation in the viscosity sense. First, by weakening the notion of solution
to the second order nonlinear PDE (E), we are enlarging the set of solutions,
and one has to guarantee that uniqueness still holds (in some convenient class
of functions). This issue will be discussed in the subsequent Section 5.4.
5.3 First properties

5.3.1 Change of variable / function
We start with two useful properties of viscosity solutions which allow to apply
standard change of variable techniques for classical solutions in the context of
viscosity solutions.
Proposition 5.7. Let u be a locally bounded (discontinuous) viscosity superso-

lution of (E). If f is a C 1 (R) function with Df 6= 0 on R, then the function v
:= f −1 ◦ u is a (discontinuous)
- viscosity supersolution, when Df > 0,
- viscosity subsolution, when Df < 0,
of the equation
K(x, v(x), Dv(x), D2 v(x)) = 0 for x ∈ O ,
where
:= F x, f (r), Df (r)p, D2 f (r)pp0 + Df (r)A .

K(x, r, p, A)
We leave the easy proof of this proposition to the reader. The next result
shows how limit operations with viscosity solutions can be performed very easily.
A particular change of function in the previous proposition is the multiplica-
tion by a scalar. Another useful tool is the addition, or convex combination, of
viscosity sub or supersolutions. Of course, this is possible only if the equation is
suitable for such operations. For this reason, the following result specializes the
discussion to the case where the nonlinearity F (x, r, p, A) is convex in (r, p, A).
By standard convex analysis, this means that F can be expressed as:
n 1 o
F (x, r, p, A) = sup − fγ (x) + kγ (x)r − bγ (x) · p − Tr σγ (x)2 A (5.1)
,
γ∈G 2
for some family of functions (fγ , kγ , bγ , σγ )γ .

Proposition 5.8. Assume F is convex in (r, p, A), with functions (fγ , kγ , bγ )
in the representation (5.1) continuous in x, and σγ Lipschitz in x, for all γ ∈ G.
Let u1 and u2 be two upper semicontinuous viscosity subsolutions of (E).
Then, λu1 + (1 − λ)u2 is a viscosity subsolution of (E).
We only provide the proof in the case where one either one of the subsolu-
tions is classical. The general case requires more technical tools which will be
developed later, and the corresponding proof is reported in Section 5.4.4.
Proof. (Assuming u2 ∈ C 2 ) Denote λ1 := λ and λ2 := 1 − λ, and u := λ1 u1 +

λ2 u2 . Let x0 in O and ϕ ∈ C 2 (O) be such that (u − ϕ)(x0 ) = maxO (u − ϕ).
Denote ψ := λ−11 (ϕ−λ2 u2 ), or equivalently ϕ = λ1 ψ+λ2 u2 . Then (u1 −ψ)(x0 ) =
maxO (u1 − ψ), and it follows from the viscosity subsolution property of u1 that
F x0 , u1 (x0 ), Dψ(x0 ), D2 ψ(x0 ) ≤ 0.

Since u2 is a classical subsolution of the equation (E), and F is convex in (r, p, A),
we now compute directly that
F x0 , u(x0 ), Dϕ(x0 ), D2 ϕ(x0 ) ≤ λ1 F x0 , u1 (x0 ), Dψ(x0 ), D2 ψ(x0 )

+λ2 F x0 , u2 (x0 ), Du2 (x0 ), D2 u2 (x0 ) ≤ 0.

Notice that, in the last simplified proof, the technical condition on the coef-
ficients (fγ , kγ , bγ , σγ )γ of the representation (5.1) of F has not been used.
5.3.2 Stability of viscosity solutions

The following result is crucial in the theory of viscosity solutions, and is in
fact the reason for the “viscosity” denomination. Loosely speaking, we now
show that limiting operations with viscosity solutions are always possible under
minimal assumptions, namely that all limiting quantities are well-defined !
5.3. First properties 69
Theorem 5.9. Let uε be a lower semicontinuous viscosity supersolution of the

equation
Fε x, uε (x), Duε (x), D2 uε (x) = 0 for x ∈ O ,

where (Fε )ε>0 is a sequence of continuous functions satisfying the ellipticity

condition. Suppose that (ε, x) 7−→ uε (x) and (ε, z) 7−→ Fε (z) are locally bounded,
and define
u(x) := lim inf uε (x0 ) and F (z) := lim sup Fε (z 0 ).

(ε,x0 )→(0,x) (ε,z 0 )→(0,z)
Then, u is a lower semicontinuous viscosity supersolution of the equation
F x, u(x), Du(x), D2 u(x) = 0 for x ∈ O.

A similar statement holds for subsolutions.

Proof. The fact that u is a lower semicontinuous function is left as an exercise
for the reader. Let ϕ ∈ C 2 (O) and x̄, be a strict minimizer of the difference
u − ϕ. By definition of u, there is a sequence (εn , xn ) ∈ (0, 1] × O such that
(εn , xn ) −→ (0, x̄) and uεn (xn ) −→ u(x̄).
Consider some r > 0 together with the closed ball B̄ with radius r, centered at
x̄. Of course, we may choose |xn − x̄| < r for all n ≥ 0. Let x̄n be a minimizer
of uεn − ϕ on B̄. We claim that
x̄n −→ x̄ and uεn (x̄n ) −→ u(x̄) as n → ∞. (5.2)
Before verifying this, let us complete the proof. We first deduce that x̄n is an
interior point of B̄ for large n, so that x̄n is a local minimizer of the difference
uεn − ϕ. Then :
Fεn x̄n , uεn (x̄n ), Dϕ(x̄n ), D2 ϕ(x̄n ) ≥ 0,

and the required result follows by taking limits and using the definition of F .
It remains to prove Claim (5.2). Recall that (xn )n is valued in the compact
set B̄. Then, there is a subsequence, still named (xn )n , which converges to some
x̃ ∈ B̄. We now prove that x̃ = x̄ and obtain the second claim in (5.2) as a
by-product. Using the fact that x̄n is a minimizer of uεn − ϕ on B̄, together
with the definition of u, we see that
0 = (u − ϕ)(x̄) = lim (uεn − ϕ) (xn )

n→∞
≥ lim sup (uεn − ϕ) (x̄n )
n→∞
≥ lim inf (uεn − ϕ) (x̄n )
n→∞
≥ (u − ϕ)(x̃) .
We now obtain (5.2) from the fact that x̄ is a strict minimizer of the difference
(u − ϕ). ♦
Observe that the passage to the limit in partial differential equations written
in the classical or the generalized sense usually requires much more technicalities,
as one has to ensure convergence of all the partial derivatives involved in the
equation. The above stability result provides a general method to pass to the
limit when the equation is written in the viscosity sense, and its proof turns out
to be remarkably simple.
A possible application of the stability result is to establish the convergence
of numerical schemes. In view of the simplicity of the above statement, the
notion of viscosity solutions provides a nice framework for such questions.
5.3.3 Parameter variables

The following result is trivial in the classical case, but needs some technicalities
when stated in the viscosity sense.
Proposition 5.10. Let A ⊂ Rd1 and B ⊂ Rd2 be two open subsets, and let
u : A × B −→ R be a lower semicontinuous viscosity supersolution of the equa-
tion :
F x, y, u(x, y), Dy u(x, y), Dy2 u(x, y) ≥ 0 on A × B,

where F is a continuous elliptic operator. Then, for all fixed x0 ∈ A, the

function v(y) := u(x0 , y) is a viscosity supersolution of the equation :
F x0 , y, v(y), Dv(y), D2 v(y) ≥ 0 on B.

A similar statement holds for the subsolution property.

Proof. Fix x0 ∈ A, set v(y) := u(x0 , y), and let y0 ∈ B and f ∈ C 2 (B) be such
that
(v − f )(y0 ) < (v − f )(y) for all y ∈ J \ {y0 } , (5.3)
where J is an arbitrary compact subset of B containing y0 in its interior. For

each integer n, define
ϕn (x, y) := f (y) − n|x − x0 |2 for (x, y) ∈ A × B ,
and let (xn , yn ) be defined by
(u − ϕn )(xn , yn ) = min(u − ϕn ) ,
I×J
where I is a compact subset of A containing x0 in its interior. We claim that
(xn , yn ) −→ (x0 , y0 ) and u(xn , yn ) −→ u(x0 , y0 ) as n −→ ∞. (5.4)

5.4. Comparison results 71
Before proving this, let us complete the proof. Since (x0 , y0 ) is an interior point
of A × B, it follows from the viscosity property of u that
F xn , yn , u(xn , yn ), Dy ϕn (xn , yn ), Dy2 ϕn (xn , yn )

0 ≤
F xn , yn , u(xn , yn ), Df (yn ), D2 f (yn ) ,

=
and the required result follows by sending n to infinity.

We now turn to the proof of (5.4). Since the sequence (xn , yn )n is valued in
the compact subset A × B, we have (xn , yn ) −→ (x̄, ȳ) ∈ A × B, after passing
to a subsequence. Observe that
u(xn , yn ) − f (yn ) ≤ u(xn , yn ) − f (yn ) + n|xn − x0 |2

= (u − ϕn )(xn , yn )
≤ (u − ϕn )(x0 , y0 ) = u(x0 , y0 ) − f (y0 ) .
Taking the limits, this provides:it follows from the lower semicontinuity of u
that
u(x̄, ȳ) − f (ȳ) ≤ lim inf u(xn , yn ) − f (yn ) + n|xn − x0 |2

n→∞
≤ lim sup u(xn , yn ) − f (yn ) + n|xn − x0 |2 (5.5)
n→∞
≤ u(x0 , y0 ) − f (y0 ).
Since u is lower semicontinu, this implies that u(x̄, ȳ)−f (ȳ)+lim inf n→∞ n|xn −
x0 |2 ≤ u(x0 , y0 ) − f (y0 ). Then, we must have x̄ = x0 , and
(v − f )(ȳ) = u(x0 , ȳ) − f (ȳ) ≤ (v − f )(y0 ),
which implies that ȳ = y0 in view of (5.3), and n|xn − x0 |2 −→ 0. We also

deduce from inequalities (5.5) that u(xn , yn ) −→ u(x0 , y0 ), concluding the proof
of (5.4). ♦
5.4 Comparison result and uniqueness

In this section, we show that the notion of viscosity solutions is consistent with
the maximum principle for a wide class of equations. Once we will have such a
result, the reader must be convinced that the notion of viscosity solutions is a
good weakening of the notion of classical solution.
We recall that the maximum principle is a stronger statement than unique-
ness, i.e. any equation satisfying a comparison result has no more than one
solution.
In the viscosity solutions literature, the maximum principle is rather called
comparison principle.
5.4.1 Comparison of classical solutions in a bounded do-

main
Let us first review the maxium principle in the simplest classical sense.
Proposition 5.11. Assume that O is an open bounded subset of Rd , and the

F (x, r, p, A) is elliptic and strictly increasing in r. Let u, v ∈
nonlinearity
C 2 cl(O) be classical subsolution and supersolution of (E), respectively, with
u ≤ v on ∂O. Then u ≤ v on cl(O).
Proof. Our objective is to prove that
M := sup (u − v) ≤ 0. (5.6)
cl(O)
Assume to the contrary that M > 0. Then since cl(O) is a compact subset of
Rd , and u − v ≤ 0 on ∂O, we have:
M = (u − v)(x0 ) for some x0 ∈ O with D(u − v)(x0 ) = 0, D2 (u − v)(x0 ) ≤ 0.

(5.7)
Then, it follows from the viscosity properties of u and v that:
F x0 , u(x0 ), Du(x0 ), D2 u(x0 ) ≤ 0 ≤ F x0 , v(x0 ), Dv(x0 ), D2 v(x0 )

≤ F x0 , u(x0 ) − M, Du(x0 ), D2 u(x0 ) ,

where the last inequality follows crucially from the ellipticity of F . This provides
the desired contradiction, under our condition that F is strictly increasing in r.
♦
The objective of this section is to mimic the previous proof in the sense of
viscosity solutions. We first start by the case of first order equations where the
beautiful trick of doubling variables allows for an immediate adaptation of the
previous argument. However, more work is needed for the second order case.
5.4.2 Comparison of viscosity solutions of first order equa-

tions
A crucial idea in the theory of viscosity solutions is to replace the maximization
problem in (5.6) by:
n n o
Mn := sup u(x) − v(y) − |x − y|2 , n ≥ 1, (5.8)
x,y∈cl(O)
2
where the concept of doubling variables separates the dependence of the func-
tions u and v on two different variables, and the penalization term involves the
parameter n which is intended to mimick the maximization problem (5.6) for
large n.
Proposition 5.12. Let O be a bounded open subset of Rd . Assume that the non-
lineary F (x, r, p) is independent of the A−variable, Lipshitz in the x−variable
uniformly in (r, p), and that there is a constant c > 0 such that
F (x, r0 , p) − F (x, r, p) ≥ c(r0 − r) for all r0 ≥ r, x ∈ cl(O), p ∈ Rd . (5.9)

Let u, v ∈ C 0 cl(O) be viscosity subsolution and supersolution of (E), respec-
tively, with u ≤ v on ∂O. Then u ≤ v on cl(O).
Proof. Suppose to the contrary that η := (u − v)(x0 ) > 0 for some x0 ∈ O, so
that the maximum value in (5.8) Mn ≥ η. Since u and v are continuous, we
may find for each n ≥ 1 a maximizer (xn , yn ) ∈ cl(O)2 of the problem (5.8):
n
Mn = u(xn ) − v(yn ) − |xn − yn |2 .
2
We shall prove later that
2εn := n|xn − yn |2 −→ 0, as n → ∞, and xn , yn ∈ O for large n. (5.10)
Observe that (5.8) implies that xn is a maximizer of the difference (u − ϕ), and
yn is a minimizer of the difference (v − ψ), where
n n
ϕ(x) := v(yn ) + |x − yn |2 , x ∈ cl(O), ψ(y) := u(xn ) − |xn − y|2 , x ∈ cl(O).
2 2
Then, it follows from the viscosity properties of u and v that

F xn , u(xn ), Dϕ(xn ) ≤ 0 ≤ F yn , v(yn ), Dψ(yn ) , for large n.
Using the Lipshitz property of F in x and the increaseness property (5.9), this
implies that

0 ≥ F xn , v(yn ) + Mn + εn , n(xn − yn ) − F yn , v(yn ), n(xn − yn )
≥ −|Fx |L∞ |xn − yn | + c(Mn + εn )
In view of (5.10) and the fact that Mn ≥ η > 0, this leads to the required
contradiction.
It remains to justify (5.10). Let x∗ ∈ cl(O) ba a maximizer of (u − v)
on cl(O), and denote by mn (x, y) the objective function in the maximization
problem (5.8). Let (x̄, ȳ) be any accumulation point of the bounded sequence
(xn , yn )n , i.e. (xnk , ynk ) −→ (x̄, ȳ) for some subsequence (nk )k . Then, it follows
from the obvious inequality Mn = mn (xn , yn ) ≥ mn (x∗ , x∗ ) that
nk
lim sup |xnk − ynk |2 ≤ u(xnk ) − v(ynk ) − (u − v)(x∗ )
k→∞ 2
≤ u(x̄) − v(ȳ) − (u − v)(x∗ ),
by the upper-semicontinuity of u and the slower-semicontinuity of v. This shows
that x̄ = ȳ, and it follows from the definition of x∗ as a maximizer of (u − v)
that
nk
lim sup |xnk − ynk |2 ≤ (u − v)(x̄) − (u − v)(x∗ ) ≤ 0,
k→∞ 2
and hence nk |xnk −ynk |2 −→ 0 as k → ∞. By the arbitrariness of the converging

subsequence (xnk , ynk )k , we obtain that n|xn − yn |2 −→ 0 as n → ∞.
Finally, suppose that there is a subsequence (xnk )k≥1 ⊂ ∂O. Then, since
u ≤ v on ∂O:
η ≤ Mnk ≤ u(xnk ) − v(ynk ) ≤ v(xnk ) − v(ynk ) + sup(u − v)

∂O

≤ ρ |xxk − ynk | −→ 0, as k → ∞,
contradicting the positivity of η. A similar argument shows that yn ∈ O for

large n. ♦
In the previous proof, one can easily see that the condition that F is uni-
formly Lipshitz in x can be weakened by exploiting the stronger convergence
of |xn − yn | to 0 in (5.10). For instance, the previous comparison result holds
true under the monotonicity condition (5.9) together with the locally Lipschitz
condition:
|F (x0 , r, p) − F (x, r, p)| ≤ C(1 + |p|)|x − x0 |.
5.4.3 Semijets definition of viscosity solutions

In order to extend the comparison result to second order equations, we first
need to develop a convenient alternative definition of viscosity solutions. For
p ∈ Rd , and A ∈ Sd , we introduce the quadratic function:
1
q p,A (y) := p · y + Ay · y, y ∈ Rd .
2
For v ∈ LSC(O), let (x0 , ϕ) ∈ O ×C 2 (O) be such that x0 is a local minimizer
of the difference (v − ϕ) in O. Then, defining p := Dϕ(x0 ) and A := D2 ϕ(x0 ),
it follows from a second order Taylor expansion that:
v(x) ≥ v(x0 ) + q p,A (x − x0 ) + ◦ |x − x0 |2 .

−
Motivated by this observation, we introduce the subjet JO v(x0 ) by
n o
−
JO v(x0 ) := (p, A) ∈ Rd × Sd : v(x) ≥ v(x0 ) + q p,A (x − x0 ) + ◦ |x − x0 |2 .
(5.11)
+
Similarly, we define the superjet JO u(x0 ) of a function u ∈ USC(O) at the point
x0 ∈ O by
n o
+
JO u(x0 ) := (p, A) ∈ Rd × Sd : u(x) ≤ u(x0 ) + q p,A (x − x0 ) + ◦ |x − x0 |2
(5.12)
Then, it can be proved that a function v ∈ LSC(O) is a viscosity supersolution
of the equation (E) if and only if
−
F (x, v(x), p, A) ≥ 0 for all (p, A) ∈ JO v(x).
The nontrivial implication of the previous statement requires to construct, for

−
every (p, A) ∈ JO v(x0 ), a smooth test function ϕ such that the difference (v −ϕ)
has a local minimum at x0 . We refer to Fleming and Soner [6], Lemma V.4.1
p211.
A symmetric statement holds for viscosity subsolutions. By continuity con-
±
siderations, we can even enlarge the semijets JO w(x0 ) to the folowing closure
n
±
J¯O w(x) := (p, A) ∈ Rd × Sd : (xn , w(xn ), pn , An ) −→ (x, w(x), p, A)
o
±
for some sequence (xn , pn , An )n ⊂ Graph(JO w) ,
± ±
where (xn , pn , An ) ∈ Graph(JO w) means that (pn , An ) ∈ JO w(xn ). The follow-
ing result is obvious, and provides an equivalent definition of viscosity solutions.
Proposition 5.13. Consider an elliptic nonlinearity F , and let u ∈ USC(O),
v ∈ LSC(O).
(i) Assume that F is lower-semicontinuous. Then, u is a viscosity subsolution
of (E) if and only if:
F (x, u(x), p, A) ≤ 0 for all x ∈ O and (p, A) ∈ J¯O

+
u(x).
(ii) Assume that F is upper-semicontinuous. Then, v is a viscosity supersolu-

tion of (E) if and only if:
−
F (x, v(x), p, A) ≥ 0 for all x ∈ O and (p, A) ∈ J¯O v(x).
5.4.4 The Crandall-Ishii’s lemma

The major difficulty in mimicking the proof of Proposition 5.11 is to derive an
analogous statement to (5.7) without involving the smoothness of u and v, as
these functions are only known to be upper- and lower-semicontinuous in the
context of viscosity solutions.
This is provided by the following result due to M. Crandall and I. Ishii. For
a symmetric matrix, we denote by |A| := sup{(Aξ) · ξ : |ξ| ≤ 1}.
Lemma 5.14. Let O be an open locally compact subset of Rd . Given u ∈
USC(O) and v ∈ LSC(O), set m(x, y) := u(x) − v(y), x, y ∈ O, and assume
that:
(m−φ) for some (x0 , y0 ) ∈ O2 , φ ∈ C 2 cl(O)2 . (5.13)

(m−φ)(x0 , y0 ) = max
2 O
Then, for each ε > 0, there exist A, B ∈ Sd such that

−
(Dx φ(x0 , y0 ), A) ∈ J¯O
+
u(x0 ), (−Dy φ(x0 , y0 ), B) ∈ J¯O v(y0 ),
and the following inequality holds in the sense of symmetric matrices in S2d :

A 0
− ε−1 + D2 φ(x0 , y0 ) I2d ≤ ≤ D2 φ(x0 , y0 ) + εD2 φ(x0 , y0 )2 .

0 −B
Proof. See Section 5.7. ♦
We will be applying Lemma 5.14 in the particular case

α
φ(x, y) := |x − y|2 for x, y ∈ O. (5.14)
2
Intuitively, sending α to ∞, we expect that the maximization of (u(x) − v(y) −
φ(x, y) on O2 reduces to the maximization of (u − v) on O as in (5.7). Then,
taking ε−1 = α, we directly compute that the conclusions of Lemma 5.14 reduce
to
−
(α(x0 − y0 ), A) ∈ J¯O
+
u(x0 ), (α(x0 − y0 ), B) ∈ J¯O v(y0 ), (5.15)
and

Id 0 A 0 Id −Id
−3α ≤ ≤ 3α . (5.16)
0 Id 0 −B −Id Id
Remark 5.15. If u and v were C 2 functions in Lemma 5.14, the first and second
order condition for the maximization problem (5.13) with the test function (5.14)
is Du(x0 ) = α(x0 − y0 ), Dv(x0 ) = α(x0 − y0 ), and
2
D u(x0 ) 0 Id −Id
≤ α .
0 −D2 v(y0 ) −Id Id
Hence, the right-hand side inequality in (5.16) is worsening the previous second
order condition by replacing the coefficient α by 3α. ♦
Remark 5.16. The right-hand side inequality of (5.16) implies that
A ≤ B. (5.17)
To see this, take an arbitrary ξ ∈ Rd , and denote by ξ T its transpose. From
right-hand side inequality of (5.16), it follows that

A 0 ξ
0 ≥ (ξ T , ξ T ) = (Aξ) · ξ − (Bξ) · ξ.
0 −B ξ
♦
Before turning to the comparison result for second order equations, let us
go back to the
Proof of Proposition 5.8 Introduce the linear maps in (r, p, A):

1
Fγ (x, r, p, A) = −fγ (x) + kγ (x)r − bγ (x) · p − Tr σγ (x)2 A for all

γ ∈ G.
2
In view of the representation (5.1), the functions u1 and u2 are viscosity subso-
lutions of the linear equations
Fγ x, ui (x), Dui (x), D2 ui (x) ≤ 0, for all γ ∈ G; i = 1, 2.

1. Denote again λ1 := λ, λ2 := 1 − λ, and set u = λ1 u1 + λ2 u2 . Let r > 0,

x0 ∈ B0 := Br (x0 ) ⊂ O, with closure B̄0 , and ϕ ∈ C 2 (O) be such that
(u − ϕ)(x0 ) = strict max(u − ϕ).

B̄0
We shall use the doubling variable technique, and thus introduce:

α
ψ(x, y) := λ1 (u1 − ϕ)(x) + λ2 (u2 − ϕ)(y) − φ(x, y), with φ(x, y) := |x − y|2 .
2
By the compactness of the closed ball B̄0 , we may find a maximizer (xα , yα ) of
φ on B̄0 × B̄0 :
Mα := sup φ = φ(xα , yα ) for all α ≥ 0.

B̄0 ×B̄0
We continue using the following claim, which will be proved later,
α|xα − yα |2 −→ 0, xα −→ x0 , Mα −→ (u − ϕ)(x0 ),
(5.18)
and u1 (xα ) −→ u1 (x0 ), u2 (yα ) −→ u2 (x0 ).
In particular, this implies that xα , yα ∈ B0 for sufficiently large α.

2. We are now in a position to apply the Crandall-Ishii Lemma 5.14. Then, we
may find matrices A1 , A2 ∈ Sd such that
Dx φ(xα , yα ) + λ1 Dϕ(xα ), A1 ∈ J¯O+

(λ1 u1 )(xα ) = λ1 J¯O
+

(u1 )(xα ),
¯− ¯−

Dy φ(xα , yα ) + λ2 Dϕ(yα ), A2 ∈ J (λ2 u2 )(yα ) = λ2 J (u2 )(yα ),
O O
with the matrix inequalities
λ1 D2 ϕ(xα )

A1 0 Id −Id 0
≤ kα + .
0 A2 −Id Id 0 λ2 D2 ϕ(yα )
As u1 and u2 are both viscosity subsolutions of the equation defined by the

nonlinearity Fγ , this implies that
0 ≥ λ1 Fγ xα , u1 (xα ), λ−1 −1

1 Dx φ(xα , yα ) + Dϕ(xα ), λ1 A1
+λ2 Fγ yα , u2 (yα ), λ−1 −1

2 Dy φ(xα , yα ) + Dϕ(yα ), λ2 A2
= −λ1 fγ (xα ) − λ2 fγ (yα ) + λ1 kγ (xα )u1 (xα ) + λ2 kγ (yα )u2 (yα )
−λ1 bγ (xα ) · Dϕ(xα ) − λ2 bγ (yα ) · Dϕ(yα )
1
− Tr σγ2 (xα )A1 + σγ2 (yα )A2 ,

2
where we used the fact that Dx φ = −Dy φ = α(x − y). Then, fγ , kγ , bγ are
continuous, (5.18) yields
0 ≥ −fγ (x0 ) + kγ (x0 )u(x0 ) − bγ (x0 ) · Dϕ(x0 )

1
− lim inf Tr σγ2 (xα )A1 + σγ2 (yα )A2 .

2 α→∞
Denoting A01 := A1 − D2 ϕ(xα ), and A02 := A2 − D2 ϕ(yα ), it follows from the

calculation in Example 5.21 below that
Tr σγ2 (xα )A1 + σγ2 (yα )A2 = Tr σγ2 (xα )D2 ϕ(xα ) + σγ2 (yα )D2 ϕ(yα )

+Tr σγ2 (xα )A01 + σγ2 (yα )A02

≤ Tr σγ2 (xα )D2 ϕ(xα ) + σγ2 (yα )D2 ϕ(yα )

+3αTr (σγ (xα ) − σγ (yα ))2

≤ Tr σγ2 (xα )D2 ϕ(xα ) + σγ2 (yα )D2 ϕ(yα )

+3Lγ α|xα − yα |2 ,
where Lγ is the Lipschitz constant of the coefficient σγ . Using again (5.18) ,

this provides
1
0 ≥ −fγ (x0 ) + kγ (x0 )u(x0 ) − bγ (x0 ) · Dϕ(x0 ) − Tr[σγ2 (x0 )D2 ϕ(x0 )].
2
The required viscosity subsolution property follows from the arbitrariness of
γ ∈ G.
3. It remains to justify (5.18). As (xα , yα )α is bounded, we may find a con-
verging sequence (xαn , yαn )n to some imiting point (x̂, ŷ). By the definition of
(xα , yα ) as the maximizer of φ, we have
(u − ϕ)(x0 , y 0 ) ≤ lim sup φ(xαn , yαn )

n→∞
2
≤ λ1 (u1 − ϕ)(x̂) + λ2 (u2 − ϕ)(ŷ) − lim inf αn xαn − yαn .
n→∞
This implies that x̂ = ŷ. Then,
(u − ϕ)(x0 ) ≤ lim inf φ(xαn , yαn )

n→∞
≤ lim sup φ(xαn , yαn ) ≤ (u − ϕ)(x̂).
n→∞
As x0 is a strict maximizer of the difference u − ϕ, we conclude that x̂ = x0 . In

particular, since any subsequence of (xα , yα ) converges to (x0 , y 0 ), we deduce
that (xα , yα ) −→ (x0 , y 0 ), and we may repeat the previous sequence of inequal-
ities with α → ∞, and obtain that all inequalities are in fact equalities. Conse-
quently, we also have α|xα −yα |2 −→ 0, and lim supα→∞ λ1 u1 (xα )+λ2 u2 (yα ) =
u(x0 ). The last convergence result, together with the upper semicontinuity of
u2 , implies that
λ1 lim inf u1 (xα ) = lim λ1 u( xα ) + λ2 u2 (yα ) − λ2 lim sup u2 (yα ) ≤ λ1 u1 (x0 ).

α→∞ α→∞ α→∞
As u1 is upper semicontinuous, this implies that u1 (xα ) −→ u1 (x0 ). By the

same argument, we obtain that u2 (yα ) −→ u2 (x0 ). ♦
5.4.5 Comparison of viscosity solutions in a bounded do-

main
We now prove a comparison result for viscosity sub- and supersolutions by
using Lemma 5.14 to mimic the proof of Proposition 5.11. The statement will
be proved under the following conditions on the nonlinearity F which will be
used at the final Step 3 of the subsequent proof.
Assumption 5.17. (i) There exists γ > 0 such that
F (x, r, p, A) − F (x, r0 , p, A) ≥ γ(r − r0 ) for all r ≥ r0 , (x, p, A) ∈ O × Rd × Sd .
(ii) There is a function $ : R+ −→ R+ with $(0+) = 0, such that

F (y, r, α(x − y), B) − F (x, r, α(x − y), A) ≤ $ α|x − y|2 + |x − y|
for all x, y ∈ O, r ∈ R and A, B satisfying (5.16).
Remark 5.18. Assumption 5.17 (ii) implies that the nonlinearity F is elliptic.
To see this, notice that for A ≤ B, ξ, η ∈ Rd , and ε > 0, we have
Aξ · ξ − (B + εId )η · η ≤ Bξ · ξ − (B + εId )η · η
= 2η · B(ξ − η) + B(ξ − η) · (ξ − η) − ε|η|2
≤ ε−1 |B(ξ − η)|2 + B(ξ − η) · (ξ − η)
|B| 1 + ε−1 |B| |ξ − η|2 .

≤
For 3α ≥ (1 + ε−1 |B|)|B|, the latter inequality implies the right-hand side of
(5.16) holds true with (A, B + εId ). For ε sufficiently small, the left-hand side
of (5.16) is also true with (A, B + εId ) if in addition α > |A| ∨ |B|. Then
F (x − α−1 p, r, p, B + εI) − F (x, r, p, A) ≤ $ α−1 (|p|2 + |p|) ,

which provides the ellipticity of F by sending α → ∞ and ε → 0. ♦

Theorem 5.19. Let O be an open bounded subset of Rd and let F be an elliptic
operator satisfying Assumption 5.17. Let u ∈ USC(O) and v ∈ LSC(O) be
viscosity subsolution and supersolution of the equation (E), respectively. Then
u ≤ v on ∂O =⇒ u ≤ v on Ō := cl(O).
Proof. As in the proof of Proposition 5.11, we assume to the contrary that
δ := (u − v)(z) > 0 for some z ∈ O. (5.19)
Step 1: For every α > 0, it follows from the upper-semicontinuity of the differ-
ence (u − v) and the compactness of Ō that
n α o
Mα := sup u(x) − v(y) − |x − y|2
O×O 2
α
= u(xα ) − v(yα ) − |xα − yα |2 (5.20)
2
for some (xα , yα ) ∈ Ō × Ō. Since Ō is compact, there is a subsequence

(xn , yn ) := (xαn , yαn ), n ≥ 1, which converges to some (x̂, ŷ) ∈ Ō × Ō. We
shall prove in Step 4 below that
x̂ = ŷ, αn |xn − yn |2 −→ 0, and Mαn −→ (u − v)(x̂) = sup(u − v). (5.21)

O
Then, since u ≤ v on ∂O and

αn
δ ≤ Mαn = u(xn ) − v(yn ) − |xn − yn |2 (5.22)
2
by (5.19), it follows from the first claim in (5.21) that (xn , yn ) ∈ O × O.
Step 2: Since the maximizer (xn , yn ) of Mαn defined in (5.20) is an interior point
to O × O, it follows from Lemma 5.14 that there exist two symmetric matrices
An , Bn ∈ Sn satisfying (5.16) such that (xn , αn (xn − yn ), An ) ∈ J¯O+
u(xn ) and
−
(yn , αn (xn − yn ), Bn ) ∈ J¯O v(yn ). Then, since u and v are viscosity subsolu-
tion and supersolution, respectively, it follows from the alternative definition of
viscosity solutions in Proposition 5.13 that:
F (xn , u(xn ), αn (xn − yn ), An ) ≤ 0 ≤ F (yn , v(yn ), αn (xn − yn ), Bn ) . (5.23)
Step 3: We first use the strict monotonicity Assumption 5.17 (i) to obtain:

γδ ≤ γ u(xn ) − v(yn ) ≤ F (xn , u(xn ), αn (xn − yn ), An )
−F (xn , v(yn ), αn (xn − yn ), An ) .
By (5.23), this provides:
γδ ≤ F (yn , v(yn ), αn (xn − yn ), Bn ) − F (xn , v(yn ), αn (xn − yn ), An ) .
Finally, in view of Assumption 5.17 (ii) this implies that:
γδ ≤ $ αn |xn − yn |2 + |xn − yn | .

Sending n to infinity, this leads to the desired contradiction of (5.19) and (5.21).
Step 4: It remains to prove the claims (5.21). By the upper-semicontinuity of
the difference (u − v) and the compactness of Ō, there exists a maximizer x∗ of
the difference (u − v). Then
αn
(u − v)(x∗ ) ≤ Mαn = u(xn ) − v(yn ) − |xn − yn |2 .
2
Sending n → ∞, this provides
1
`¯ := lim sup αn |xn − yn |2 ≤ lim sup u(xαn ) − v(yαn ) − (u − v)(x∗ )
2 n→∞ n→∞
≤ u(x̂) − v(ŷ) − (u − v)(x∗ );
in particular, `¯ < ∞ and x̂ = ŷ. Using the definition of x∗ as a maximizer of

(u − v), we see that:
0 ≤ `¯ ≤ (u − v)(x̂) − (u − v)(x∗ ) ≤ 0.
Then x̂ is a maximizer of the difference (u − v) and Mαn −→ supO (u − v). ♦
We list below two interesting examples of operators F which satisfy the

conditions of the above theorem:
Example 5.20. Assumption 5.17 is satisfied by the nonlinearity
F (x, r, p, A) = γr + H(p)
for any continuous function H : Rd −→ R, and γ > 0.
In this example, the condition γ > 0 is not needed when H is a convex and
H(Dϕ(x)) ≤ α < 0 for some ϕ ∈ C 1 (O). This result can be found in [1].
Example 5.21. Assumption 5.17 is satisfied by
F (x, r, p, A) = −Tr (σσ 0 (x)A) + γr,
where σ : Rd −→ Sd is a Lipschitz function, and γ > 0. Condition (i) of
Assumption 5.17 is obvious. To see that Condition (ii) is satisfied, we consider
(A, B, α) ∈ Sd × Sd × R∗+ satisfying (5.16). We claim that
d
Tr[M M T A − N N T B] ≤
X
3α|M − N |2 = 3α (M − N )2ij .
i,j=1
To see this, observe that the matrix

!
NNT NMT
C :=
MNT MMT
is a non-negative matrix in Sd . From the right hand-side inequality of (5.16),

this implies that

T T A 0
Tr[M M A − N N B] = Tr C
0 −B

Id −Id
≤ 3αTr C
−Id Id
h i
= 3αTr (M − N )(M − N )T = 3α|M − N |2 .
5.5 Comparison in unbounded domains

When the domain O is unbounded, a growth condition on the functions u and
v is needed. Then, by using the growth at infinity, we can build on the proof of
Theorem 5.19 to obtain a comparison principle. The following result shows how
to handle this question in the case of a sub-quadratic growth. We emphasize
that the present argument can be adapted to alternative growth conditions.
The following condition differs from Assumption 5.17 only in its part (ii)
where the constant 3 in (5.16) is replaced by 4 in (5.24). Thus the following
Assumption 5.22 (ii) is slightly stronger than Assumption 5.17 (ii).
Assumption 5.22. (i) There exists γ > 0 such that

F (x, r, p, A) − F (x, r0 , p, A) ≥ γ(r − r0 ) for all r ≥ r0 , (x, p, A) ∈ O × Rd × Sd .
(ii) There is a function $ : R+ −→ R+ with $(0+) = 0, such that

F (y, r, α(x − y), B) − F (x, r, α(x − y), A) ≤ $ α|x − y|2 + |x − y|
for all x, y ∈ O, r ∈ R and A, B satisfying

Id 0 A 0 Id −Id
−4α ≤ ≤ 4α . (5.24)
.
0 Id 0 −B −Id Id
Theorem 5.23. Let F be an elliptic operator satisfying Assumption 5.22 with
F uniformly continuous in the pair (p, A). Let u ∈ USC(O) and v ∈ LSC(O)
be viscosity subsolution and supersolution of the equation (E), respectively, with
|u(x)| + |v(x)| = ◦(|x|) as |x| → ∞. Then
u ≤ v on ∂O =⇒ u ≤ v on cl(O).
Proof. We assume to the contrary that
δ := (u − v)(z) > 0 for some z ∈ Rd , (5.25)
and we work towards a contradiction. Let
Mα := sup u(x) − v(y) − φ(x, y),
x,y∈Rd
where
1
α|x − y|2 + ε|x|2 + ε|y|2 .

φ(x, y) :=
2
1. Since u(x) = ◦(|x|2 ) and v(y) = ◦(|y|2 ) at infinity, there is a maximizer
(xα , yα ) for the previous problem:
Mα = u(xα ) − v(yα ) − φ(xα , yα ).
Moreover, there is a sequence αn → ∞ such that
(xn , yn ) := (xαn , yαn ) −→ (x̂ε , ŷε ),
and, similar to Step 4 of the proof of Theorem 5.19, we can prove that x̂ε = ŷε ,
αn |xn − yn |2 −→ 0, and Mαn −→ M∞ := sup (u − v)(x) − ε|x|2 . (5.26)
x∈Rd
Notice that Mαn ≥ (u − v)(z) − φ(z, z) ≥ δ − ε|z|2 > 0, by (5.25). then, for
sufficiently small ε > 0, we have
0 < δ − ε|z|2 ≤ lim sup Mαn = lim sup {u(xn ) − v(yn ) − φ(xn , yn )}
n→∞ n→∞
≤ lim sup {u(xn ) − v(yn ))}
n→∞
≤ lim sup u(xn ) − lim inf v(yn )
n→∞ n→∞
≤ (u − v)(x̂ε ).
Since u ≤ v on ∂O, we deduce that x̂ 6∈ ∂O and therefore (xn , yn ) is a local

maximizer of u − v − φ.
2. By the Crandall-Ishii Lemma 5.14, there exist An , Bn ∈ Sn , such that
(Dx φ(xn , yn ), An ) ∈ J¯O2,+ u(tn , xn ),
(5.27)
(−Dy φ(xn , yn ), Bn ) ∈ J¯O2,− v(tn , yn ),
and

An 0 1 2
−(αn + |D2 φ(x0 , y0 )|)I2d ≤ ≤ D2 φ(xn , yn ) + D φ(xn , yn )2 .
0 −Bn αn
(5.28)
In the present situation, we immediately calculate that
Dx φ(xn , yn ) = αn (xn − yn ) + εxn , − Dy φ(xn , yn ) = αn (xn − yn ) − εyn
and

Id −Id
D2 φ(xn , yn ) = α + ε I2d ,
−Id Id
which reduces the right hand-side of (5.28) to
ε2

An 0 Id −Id
≤ (3αn + 2ε) + ε+ I2d , (5.29)
0 −Bn −Id Id αn
while the left land-side of (5.28) implies:

An 0
−3αn I2d ≤ (5.30)
0 −Bn
3. By (5.27) and the viscosity properties of u and v, we have
F (xn , u(xn ), αn (xn − yn ) + εxn , An ) ≤ 0,
F (yn , v(yn ), αn (xn − yn ) − εyn , Bn ) ≥ 0.
Using Assumption 5.22 (i) together with the uniform continuity of F in (p, A),
this implies that:

γ u(xn ) − v(yn ) ≤ F yn , u(xn ), αn (xn − yn ) − εyn , Bn

−F yn , v(yn ), αn (xn − yn ) − εyn , Bn

≤ F yn , u(xn ), αn (xn − yn ) − εyn , Bn

−F xn , u(xn ), αn (xn − yn ) + εxn , An

≤ F yn , u(xn ), αn (xn − yn ), B̃n

−F xn , u(xn ), αn (xn − yn ), Ãn + c(ε(1 + |xn | + |yn |)),
where c(.) is a modulus of continuity of F in (p, A), and Ãn := An − 2εId ,

B̃n := Bn + 2εId . By (5.29) and (5.30), we have

Ãn 0 Id −Id
−4αI2d ≤ ≤ 4α ,
0 −B̃n −Id Id
for small ε. Then, it follows from Assumption 5.22 (ii) that
γ u(xn ) − v(xn ) ≤ $ αn |xn − yn |2 + |xn − yn | + c(ε(1 + |xn | + |yn |)).

By sending n to infinity, it follows from (5.26) that:
c(ε(1 + 2|x̂ε |)) ≥ γ M∞ + ε|x̂ε |2 ≥ γM∞ ≥ γ δ − ε|z|2 ,

and we get a contradiction of (5.25) by sending ε to zero as long as
ε|x̂ε | −→ 0 as ε & 0,
which we now prove. Indeed, we have that (u−v)(x̂ε )−ε|x̂ε |2 ≥ (u−v)z −ε|z|2 ,
and therefore
(u − v)(x̂ε ) δ
ε|x̂ε | ≤ − for small ε > 0.
|x̂ε | 2|x̂ε |
Using our condition |u(x)| = ◦(|x|) and |v(x)| = ◦(|x|) at infinity, we see that
whenever |x̂ε | is unbounded in ε, the last inequality implies that ε|x̂ε | → 0. ♦
5.6 Useful applications

We conclude this section by two consequences of the above comparison results,
which are trivial properties in the context of classical solutions.
Lemma 5.24. Let O be an open interval of R, and U : O −→ R be a lower

semicontinuous viscosity supersolution of the equation DU ≥ 0 on O. Then U
is nondecreasing on O.
Proof. For each ε > 0, define W (x) := U (x) + εx, x ∈ O. Then W satisfies in
the viscosity sense DW ≥ ε in O, i.e. for all (x0 , ϕ) ∈ O × C 1 (O) such that
(W − ϕ)(x0 ) = min(W − ϕ)(x), (5.31)

x∈O
we have Dϕ(x0 ) ≥ ε. This proves that ϕ is strictly increasing in a neighborhood

V of x0 . Let (x1 , x2 ) ⊂ V be an open interval containing x0 . We intend to prove
that
W (x1 ) < W (x2 ), (5.32)
which provides the required result from the arbitrariness of x0 ∈ O.

To prove (5.32), suppose to the contrary that W (x1 ) ≥ W (x2 ), and the
consider the function v(x) = W (x2 ) which solves the equation
Dv = 0 on the open interval (x1 , x2 ).

5.6. Applications 85
together with the boundary conditions v(x1 ) = v(x2 ) = W (x2 ). Observe that
W is a lower semicontinuous viscosity supersolution of the above equation. From
the comparison result of Example 5.20, this implies that
sup (v − W ) = max {(v − W )(x1 ), (v − W )(x2 )} ≤ 0.
[x1 ,x2 ]
Hence W (x) ≥ v(x) = W (x2 ) for all x ∈ [x1 , x2 ]. Applying this inequality at
x0 ∈ (x1 , x2 ), and recalling that the test function ϕ is strictly increasing on
[x1 , x2 ], we get :
(W − ϕ)(x0 ) > (W − ϕ)(x2 ),
contradicting (5.31). ♦
Lemma 5.25. Let O be an open interval of R, and U : O −→ R be a lower
semicontinuous viscosity supersolution of the equation −D2 U ≥ 0 on O. Then
U is concave on O.
Proof. Let a < b be two arbitrary elements in O, and consider some ε > 0
together with the function
√ √ √ √
U (a) e ε(b−s) − e− ε(b−s) + U (b) e ε(s−a) − e− ε(s−a)
v(s) := √ √ for a ≤ s ≤ b.
e ε(b−a) − e− ε(b−a)
Clearly, v solves the equation
εv − D2 v = 0 on (a, b), v=U on {a, b}.
Since U is lower semicontinuous it is bounded from below on the interval [a, b].
Therefore, by possibly adding a constant to U , we can assume that U ≥ 0, so
that U is a lower semicontinuous viscosity supersolution of the above equation.
It then follows from the comparison theorem 6.6 that :
sup (v − U ) = max {(v − U )(a), (v − U )(b)} = 0.
[a,b]
Hence,
√ √ √ √
U (a) e ε(b−s) − e− ε(b−s) + U (b) e ε(s−a) − e− ε(s−a)
U (s) ≥ v(s) = √ √
e ε(b−a) − e− ε(b−a)
and by sending ε to zero, we see that

s−a
U (s) ≥ (U (b) − U (a)) + U (a)
b−a
for all s ∈ [a, b]. Let λ be an arbitrary element of the interval [0,1], and set s
:= λa + (1 − λ)b. The last inequality takes the form :

U λa + (1 − λ)b ≥ λU (a) + (1 − λ)U (b) ,
proving the concavity of U . ♦
5.7 Proof of the Crandall-Ishii’s lemma

We start with two Lemmas. We say that a function f is λ−semiconvex if
x 7−→ f (x) + (λ/2)|x|2 is convex.
Lemma 5.26. Let f : RN −→ R be a λ−semiconvex function, for some λ ∈ R,
and assume that f (x) − 12 Bx · x ≤ f (0) for all x ∈ RN . Then there exists
X ∈ SN such that
2,+ 2,−
(0, X) ∈ J f (0) ∩ J f (0) and −λIN ≤ X ≤ B.
Our second lemma requires to introduce the following notion. For a function
v : RN −→ R and λ > 0, the corresponding λ−sup-convolution is defined by:
λ
v̂ λ (x) v(y) − |x − y|2 .

:= sup
y∈RN 2
Observe that
λ 2 λ
v̂ λ (x) + v(y) − |y|2 + λx · y

|x| = sup
2 y∈RN 2
is convex, as the supremum of linear functions. Then

v̂ λ is λ − semiconvex. (5.33)
In [3], the following property is refered to as the magical property of the sup-
convolution.
Lemma 5.27. Let λ > 0, v be a bounded lower-semicontinuous function, v̂ λ
the corresponding λ−sup-convolution.
(i) If (p, X) ∈ J 2,+ v̂(x) for some x ∈ RN , then
p 1 2
(p, X) ∈ J 2,+ v x + and v̂ λ (x) = v(x + p/λ) − |p| .
λ 2λ
(ii) For all x ∈ RN , we have (0, X) ∈ J¯2,+ v̂(x)implies that (0, X) ∈ J¯2,+ v(x).
Before proving the above lemmas, we show how they imply the Crandall-
Ishii’s lemma that we reformulate in a more symmetric way.
Lemma 5.14 Let O be an open locally compact subset of Rd and u1 , u2 ∈

USC(O). We denote w(x1 , x2 ) := u1 (x1 ) + u2 (x2 ) and we assume for some
ϕ ∈ C 2 cl(O)2 and x0 = (x01 , x02 ) ∈ O × O that:
(w − ϕ)(x0 ) = max
2
(w − ϕ).
O
Then, for each ε > 0, there exist X1 , X2 ∈ Sd such that

2,+
(Dxi ϕ(x0 ), Xi ) ∈ J¯O ui (x0i ), i = 1, 2,

X1 0
and − ε−1 + D2 ϕ(x0 ) I2d ≤ ≤ D2 ϕ(x0 ) + εD2 ϕ(x0 )2 .
0 X2
5.7. Proof of Crandall-Ishii’s lemma 87
Proof. Step 1: We first observe that we may reduce the problem to the case
1
O = Rd , x0 = 0, u1 (0) = u2 (0) = 0, and ϕ(x) = Ax · x for some A ∈ Sd .
2
The reduction to x0 = 0 follows from an immediate change of coordinates.
Choose any compact subset of K ⊂ O containing the origin and set ūi = ui on
K and −∞ otherwise, i = 1, 2. Then, the problem can be stated equivalently in
terms of the functions ūi which are now defined on Rd and take values on the
extended real line. Also by defining
¯i (xi ) := ūi (xi ) − ui (0) − Dxi ϕ(0)

ū and ϕ̄(x) := ϕ(x) − ϕ(0) − Dϕ(0) · x
we may reformulate the problem equivalently with ū ¯i (xi ) = 0 and ϕ̄(x) =

1 2 2
2 D ϕ(0)x · x + ◦(|x| ). Finally, defining ¯
ϕ̄(x) := Ax · x with A := D2 ϕ(0) + ηI2d
for some η > 0, it follows that
¯1 (x1 )+ū
ū ¯2 (x2 )−ϕ̄¯(x1 , x2 ) < ū
¯1 (x1 )+ū
¯2 (x2 )−ϕ̄(x1 , x2 ) ≤ ū
¯1 (0)+ū
¯2 (0)−ϕ̄(0) = 0.
Step 2: From the reduction of the previous step, we have
2w(x) ≤ Ax · x
= A(x − y) · (x − y)Ay · y − 2Ay · (y − x)
1
≤ A(x − y) · (x − y)Ay · y + εA2 y · y + |x − y|2
ε
1
= A(x − y) · (x − y) + |x − y| + (A + εA2 )y · y
2
ε
≤ (ε−1 + |A|)|x − y|2 + (A + εA2 )y · y.
Set λ := ε−1 + |A| and B := A + εA2 . The latter inequality implies the following
property of the sup-convolution:
1
ŵλ (y) − By · y ≤ ŵ(0) = 0.
2
Step 3: Recall from (5.33) that ŵλ is λ−semiconvex. Then, it follows from
2,+ 2,−
Lemma 5.26 that there exist X ∈ S2d such that (0, X) ∈ J ŵλ (0) ∩ J ŵλ (0)
and −λI2d ≤ X ≤ B. Moreover, it is immediately checked that ŵλ (x1 , x2 ) =
ûλ1 (x1 ) + ûλ2 (x2 ), implying that X is bloc-diagonal with blocs X1 , X2 ∈ Sd .
Hence:
X1 0
−(ε−1 + |A|)I2d ≤ ≤ A + εA2
0 X2
2,+ λ
and (0, Xi ) ∈ J ûi (0) for i = 1, 2 which, by Lemma 5.27 implies that (0, Xi ) ∈
2,+
J uλi (0). ♦
We coninue by turning to the proofs of Lemmas 5.26 and 5.27. The main
tools which will be used are the following properties of any semiconvex function
ϕ : RN −→ R whose proofs are reported in [3]:
• Aleksandrov lemma: ϕ is twice differentiable a.e.
• Jensen’s lemma: if x0 is a strict maximizer of ϕ, then for every r, δ > 0,
the set

x̄ ∈ B(x0 , r) : x 7−→ ϕ(x)+p·x has a local maximum at x̄ for some p ∈ Bδ
has positive measure in RN .
Proof of Lemma 5.26 Notice that ϕ(x) := f (x) − 21 Bx · x − |x|4 has a strict
maximum at x = 0. Localizing around the origin, we see that ϕ is a semiconvex
function. Then, for every δ > 0, by the above Aleksandrov and Jensen lemmas,
there exists qδ and xδ such that
qδ , xδ ∈ Bδ , D2 ϕ(xδ ) exists, and ϕ(xδ ) + qδ · xδ = loc-max{ϕ(x) + qδ · x}.
We may then write the first and second order optimality conditions to see that:
Df (xδ ) = −qδ + Bxδ + 4|xδ |3 and D2 f (xδ ) ≤ B + 12|xδ |2 .
Together with the λ−semiconvexity of f , this provides:
Df (xδ ) = O(δ) and −λI ≤ D2 f (xδ ) ≤ B + O(δ 2 ). (5.34)
Clearly f inherits the twice differentiability of ϕ at xδ . Then
Df (xδ ), D2 f (xδ ) ∈ J 2,+ f (xδ ) ∩ J 2,− f (xδ )

and, in view of (5.34), we may send δ to zero along some subsequence and obtain
a limit point (0, X) ∈ J¯2,+ f (0) ∩ J¯2,− f (0). ♦
Proof of Lemma 5.27 (i) Since v is bounded, there is a maximizer:

λ
v̂ λ (x) = v(y) − |x − y|2 . (5.35)
2
By the definition of v̂ λ and the fact that (p, A) ∈ J 2,+ v̂(x), we have for every
x 0 , y 0 ∈ RN :
λ 0
v(y 0 ) − |x − y 0 |2 ≤ v̂(x0 )
2
1
≤ v̂(x) + p · (x0 − x) + A(x0 − x) · (x0 − x) + ◦(x0 − x)
2
λ 1
= v(y) − |x − y|2 + p · (x0 − x) + A(x0 − x) · (x0 − x) + ◦(x0 − x),
2 2
(5.36)
5.7. Proof of Crandall-Ishii’s lemma 89
where we used (5.35) in the last equality.

By first setting x0 = y 0 + y − x in (5.36), we see that:
1
v(y 0 ) ≤ v(y) + p · (y 0 − y) + A(y 0 − y) · (y 0 − y) + ◦(y 0 − y) for all y 0 ∈ RN ,
2
which means that (p, A) ∈ J 2,+ v(y).
On the other hand, setting y 0 = y in (5.36), we deduce that:
x + x0 p
λ(x0 − x) · ≥ O |x − x0 |2 ,

+ −y
2 λ
which implies that y = x + λp .
(ii) Consider a sequence (xn , pn , An ) with (xn , v̂ λ (xn ), pn , An ) −→ (x, v̂ λ (x), 0, A)
and (pn , An ) ∈ J 2,+ v̂ λ (xn ). In view of (i) and the definition of J¯2,+ v(x), it only
remains to prove that
pn
v xn + −→ v(x). (5.37)
λ
To see this, we use the upper semicontinuity of v together with (i) and the
definition of v̂ λ :
pn
v(x) ≥ lim sup v xn +
n λ
pn
≥ lim inf v xn +
n λ
λ 1
= lim v̂ (xn ) + |pn |2 = v̂ λ (x) ≥ v(x).
n 2λ
♦
Chapter 6
Dynamic Programming
Equation in Viscosity
Sense
6.1 DPE for stochastic control problems

We now turn to the stochastic control problem introduced in Section 2.1. The
chief goal of this section is to use the notion of viscosity solutions in order to
relax the smoothness condition on the value function V in the statement of
Propositions 2.4 and 2.5. Notice that the following proofs are obtained by slight
modification of the corresponding proofs in the smooth case.
Remark 6.1. Recall that the general theory of viscosity applies for nonlinear
partial differential equations on an open domain O. This indeed ensures that
the optimizer in the definition of viscosity solutions is an interior point. In the
setting of control problems with finite horizon, the time variable moves forward
so that the left boundary of the time interval is not relevant. We shall then
write the DPE on the domain S = [0, T ) × Rd . Although this is not an open
domain, the general theory of viscosity solutions is still valid.
We first recall the setting of Section 2.1. We shall concentrate on the finite
horizon case T < ∞, while keeping in mind that the infinite horizon problems
are handled by exactly the same arguments. The only reason why we exclude
T = ∞ is because we do not want to be diverted by issues related to the
definition of the set of admissible controls.
Given a subset U of Rk , we denote by U the set of all progressively measur-
able processes ν = {νt , t < T } valued in U and by U0 := U ∩ H2 . The elements
of U0 are called admissible control processes.
The controlled state dynamics is defined by means of the functions
b : (t, x, u) ∈ S × U −→ b(t, x, u) ∈ Rn
91
92 CHAPTER 6. DPE IN VISCOSITY SENSE
and
σ : (t, x, u) ∈ S × U −→ σ(t, x, u) ∈ MR (n, d)
which are assumed to be continuous and to satisfy the conditions
|b(t, x, u) − b(t, y, u)| + |σ(t, x, u) − σ(t, y, u)| ≤ K |x − y|, (6.1)

|b(t, x, u)| + |σ(t, x, u)| ≤ K (1 + |x| + |u|). (6.2)
for some constant K independent of (t, x, y, u). For each admissible control
process ν ∈ U0 , the controlled stochastic differential equation :
dXt = b(t, Xt , νt )dt + σ(t, Xt , νt )dWt (6.3)
has a unique solution X, for all given initial data ξ ∈ L2 (F0 , P) with

ν 2
E sup |Xs | < C(1 + E[|ξ|2 ])eCt for all t ∈ [0, T ] (6.4)
0≤s≤t
for some constant C. Finally, the gain functional is defined via the functions:
f, k : [0, T ) × Rd × U −→ R and g : Rd −→ R
which are assumed to be continuous, kk − k∞ < ∞, and:
|f (t, x, u)| + |g(x)| ≤ K(1 + |u| + |x|2 ),
for some constant K independent of (t, x, u). The cost function J on [0, T ] ×
Rd × U is:
"Z #
T
J(t, x, ν) := E β ν (t, s)f (s, Xst,x,ν , νs )ds + β ν (t, T )g(XTt,x,ν ) , (6.5)
t
when this expression is meaningful, where

Z s
ν
β (t, s) := exp − k(r, Xrt,x,ν , νr )dr ,
t
and {Xst,x,ν , s ≥ t} is the solution of (6.3) with control process ν and initial
condition Xtt,x,ν = x. The stochastic control problem is defined by the value
function:
V (t, x) := sup J(t, x, ν) for (t, x) ∈ S. (6.6)

ν∈U0
We recall the expression of the Hamiltonian:

1
H(., r, p, A) := sup f (., u) − k(., u)r + b(., u) · p + Tr σσ T (., u)A , (6.7)

u∈U 2
6.1. DPE for stochastic control 93
and the second order operator associated to X and β:

1
Lu v := −k(., u)v + b(., u) · Dv + Tr σσ T (., u)D2 v ,

(6.8)
2
which appears naturally in the following Itô’s formula valid for any smooth test
function v:

dβ ν (0, t)v(t, Xtν ) = β ν (0, t) (∂t + Lνt )v(t, Xtν )dt + Dv(t, Xtν ) · σ(t, Xtν , νt )dWt .
Proposition 6.2. Assume that V is locally bounded on [0, T ) × Rd . Then, the

value function V is a viscosity supersolution of the equation
−∂t V (t, x) − H t, x, V (t, x), DV (t, x), D2 V (t, x) ≥ 0

(6.9)
on [0, T ) × Rd .
Proof. Let (t, x) ∈ S and ϕ ∈ C 2 (S) be such that
0 = (V∗ − ϕ)(t, x) = min (V∗ − ϕ). (6.10)

S
Let (tn , xn )n be a sequence in S such that
(tn , xn ) −→ (t, x) and V (tn , xn ) −→ V∗ (t, x).
Since ϕ is smooth, notice that
ηn := V (tn , xn ) − ϕ(tn , xn ) −→ 0.
Next, let u ∈ U be fixed, and consider the constant control process ν = u. We

shall denote by X n := X tn ,xn ,u the associated state process with initial data
Xtnn = xn . Finally, for all n > 0, we define the stopping time :
θn := inf {s > tn : (s − tn , Xsn − xn ) 6∈ [0, hn ) × αB} ,
where α > 0 is some given constant, B denotes the unit ball of Rn , and
p
hn := |ηn |1{ηn 6=0} + n−1 1{ηn =0} .
Notice that θn −→ t as n −→ ∞.
1. From the first inequality in the dynamic programming principle of Theorem
2.3, it follows that:
" Z θn #
n n
0 ≤ E V (tn , xn ) − β(tn , θn )V∗ (θn , Xθn ) − β(tn , r)f (r, Xr , νr )dr .
tn
Now, in contrast with the proof of Proposition 2.4, the value function is not
known to be smooth, and therefore we can not apply Itô’s formula to V . The
main trick of this proof is to use the inequality V∗ ≥ ϕ on S, implied by (6.10),

so that we can apply Itô’s formula to the smooth test function ϕ:
" #
Z θn
0 ≤ ηn + E ϕ(tn , xn ) − β(tn , θn )ϕ(θn , Xθnn ) − β(tn , r)f (r, Xrn , νr )dr
tn
"Z #
θn
·
= ηn − E β(tn , r)(∂t ϕ + L ϕ − f )(r, Xrn , u)dr
tn
"Z #
θn
−E β(tn , r)Dϕ(r, Xrn )σ(r, Xrn , u)dWr ,
tn
where ∂t ϕ denotes the partial derivative with respect to t.

2. We now continue exactly along the lines of the proof of Proposition 2.5.
Observe that β(tn , r)Dϕ(r, Xrn )σ(r, Xrn , u) is bounded on the stochastic interval
[tn , θn ]. Therefore, the second expectation on the right hand-side of the last
inequality vanishes, and :
" Z θn #
ηn 1 ·
−E β(tn , r)(∂t ϕ + L ϕ − f )(r, Xr , u)dr ≥ 0.
hn hn tn
We now send n to infinity. The a.s. convergence of the random value in-
side the expectation is easily obtained by the mean value Theorem; recall
that for n ≥ N (ω) sufficiently large, θn (ω) = hn . Since the random vari-
R θn
able h−1
n t
β(tn , r)(L· ϕ − f )(r, Xrn , u)dr is essentially bounded, uniformly in
n, on the stochastic interval [tn , θn ], it follows from the dominated convergence
theorem that :
−∂t ϕ(t, x) − Lu ϕ(t, x) − f (t, x, u) ≥ 0,
which is the required result, since u ∈ U is arbitrary. ♦
We next wish to show that V satisfies the nonlinear partial differential equa-
tion (6.9) with equality, in the viscosity sense. This is also obtained by a slight
modification of the proof of Proposition 2.5.
Proposition 6.3. Assume that the value function V is locally bounded on S.
Let the function H be finite and upper semicontinuous on [0, T ) × Rd × Rd × Sd ,
and kk + k∞ < ∞. Then, V is a viscosity subsolution of the equation
−∂t V (t, x) − H t, x, V (t, x), DV (t, x), D2 V (t, x) ≤ 0

(6.11)
on [0, T ) × Rn .
Proof. Let (t0 , x0 ) ∈ S and ϕ ∈ C 2 (S) be such that
0 = (V ∗ − ϕ)(t0 , x0 ) > (V ∗ − ϕ)(t, x) for (t, x) ∈ S \ {(t0 , x0 )}.(6.12)

6.1. DPE for stochastic control 95
In order to prove the required result, we assume to the contrary that
h(t0 , x0 ) := ∂t ϕ(t0 , x0 ) + H t0 , x0 , ϕ(t0 , x0 ), Dϕ(t0 , x0 ), D2 ϕ(t0 , x0 )

< 0,
and work towards a contradiction.

1. Since H is upper semicontinuous, there exists an open neighborhood Nr :=
(t0 − r, t0 + r) × rB(t0 , x0 ) of (t0 , x0 ), for some r > 0, such that
h := ∂t ϕ + H ., ϕ, Dϕ, D2 ϕ < 0 on Nr .

(6.13)
Then it follows from (6.12) that

+
k∞
−2ηerkk := max (V ∗ − ϕ) < 0. (6.14)
∂Nη
Next, let (tn , xn )n be a sequence in Nr such that
(tn , xn ) −→ (t0 , x0 ) and V (tn , xn ) −→ V ∗ (t0 , x0 ).
Since (V − ϕ)(tn , xn ) −→ 0, we can assume that the sequence (tn , xn ) also

satisfies :
|(V − ϕ)(tn , xn )| ≤ η for all n ≥ 1. (6.15)
For an arbitrary control process ν ∈ Utn , we define the stopping time
θnν := inf{t > tn : Xttn ,xn ,ν 6∈ Nr },
and we observe that θnν , Xθtnν ,xn ,ν ∈ ∂Nr by the pathwise continuity of the

n
controlled process. Then, with βsν := β ν (tn , s), it follows from (6.14) that:
βθνnν ϕ θnν , Xθtnν ,xn ,ν ≥ 2η + βθνnν V ∗ θnν , Xθtnν ,xn ,ν .

(6.16)
n n
2. Since βtνn = 1, it follows from (6.15) and Itô’s formula that:
V (tn , xn ) ≥ −η + ϕ(tn , xn )
ν
h Z θn i
−η + E βθνnν ϕ θnν , Xθtnν ,xn ,ν − βsν (∂t + Lνs )ϕ s, Xstn ,xn ,ν ds

=
n
tn
ν
h Z θn i
−η + E βθνnν ϕ θnν , Xθtnν ,xn ,ν + βsν f (., νs ) − h s, Xstn ,xn ,ν ds

≥
n
tn
ν
h Z θn i
ν ν tn ,xn ,ν
≥ −η + E βθnν ϕ θn , Xθν + βsν f s, Xstn ,xn ,ν , νs ds
n
tn
by (6.13). Using (6.16), this provides:

ν
h Z θn i
V (tn , xn ) ≥ η + E βθνnν V ∗ θnν , Xθtnν ,xn ,ν + βsν f s, Xstn ,xn ,ν , νs ds .

n
tn
Since η > 0 does not depend on ν, it follows from the arbitrariness of ν ∈

Utn that latter inequality is in contradiction with the second inequality of the
dynamic programming principle of Theorem (2.3). ♦
As a consequence of Propositions 6.3 and 6.2, we have the main result of

this section :
Theorem 6.4. Let the conditions of Propositions 6.3 and 6.2 hold. Then, the
value function V is a viscosity solution of the Hamilton-Jacobi-Bellman equation
−∂t V − H ·, V, DV, D2 V = 0 on S.

(6.17)
The partial differential equation (6.17) has a very simple and specific depen-
dence in the time-derivative term. Because of this, it is usually referred to as a
parabolic equation.
In order to a obtain a characterization of the value function by means of
the dynamic programming equation, the latter viscosity property needs to be
complemented by a uniqueness result. This is usually obtained as a consequence
of a comparison result.
In the present situation, one may verify the conditions of Theorem 5.23. For
completeness, we report a comparison result which is adapted for the class of
equations corresponding to stochastic control problems.
Consider the parabolic equation:
∂t u + G t, x, Du(t, x), D2 u(t, x) = 0 on S,

(6.18)
where G is elliptic and continuous. For γ > 0, set
G+γ (t, x, p, A) := sup {G(s, y, p, A) : (s, y) ∈ BS (t, x; γ)} ,

G−γ (t, x, p, A) := inf {G(s, y, p, A) : (s, y) ∈ BS (t, x; γ)} ,
where BS (t, x; γ) is the collection of elements (s, y) in S such that |t−s|2 +|x−y|2
≤ γ 2 . We report, without proof, the following result from [6] (Theorem V.8.1
and Remark V.8.1).
Assumption 6.5. The above operators satisfy:
lim supε&0 {G+γε (tε , xε , pε , Aε ) − G−γε (sε , yε , pε , Bε )}
(6.19)
≤ Const (|t0 − s0 | + |x0 − y0 |) [1 + |p0 | + α (|t0 − s0 | + |x0 − y0 |)]
for all sequences (tε , xε ), (sε , yε ) ∈ [0, T ) × Rn , pε ∈ Rn , and γε ≥ 0 with :
((tε , xε ), (sε , yε ), pε , γε ) −→ ((t0 , x0 ), (s0 , y0 ), p0 , 0) as ε & 0,
and symmetric matrices (Aε , Bε ) with

Aε 0 In −In
−KI2n ≤ ≤ 2α
0 −Bε −In In
for some α independent of ε.

6.2. DPE for optimal stopping 97
Theorem 6.6. Let Assumption 6.5 hold true, and let u ∈ USC(S̄), v ∈ LSC(S̄)
be viscosity subsolution and supersolution of (6.18), respectively. Then
sup(u − v) = sup(u − v)(T, ·).
S Rn
A sufficient condition for (6.19) to hold is that f (·, ·, u), k(·, ·, u), b(·, ·, u),
and σ(·, ·, u) ∈ C 1 (S) with
kbt k∞ + kbx k∞ + kσt k∞ + kσx k∞ < ∞
|b(t, x, u)| + |σ(t, x, u)| ≤ Const(1 + |x| + |u|) ;
see [6], Lemma V.8.1.
6.2 DPE for optimal stopping problems

We first recall the optimal stopping problem considered in Section 3.1. For
0 ≤ t ≤ T ≤ ∞, the set T[t,T ] denotes the collection of all F−stopping times
with values in [t, T ]. The state process X is defined by the SDE:
dXt = µ(t, Xt )dt + σ(t, Xt )dWt , (6.20)
where µ and σ are defined on S̄ := [0, T ) × Rn , take values in Rn and Sn ,
respectively, and satisfy the usual Lipschitz and linear growth conditions so
that the above SDE has a unique strong solution satisfying the integrability of
Theorem 1.2.
For a measurable function g : Rn −→ R, satisfying E sup0≤t<T |g(Xt )| <
∞, the gain criterion is given by:
J(t, x, τ ) := E g Xτt,x 1τ <∞ for all (t, x) ∈ S, τ ∈ T[t,T ] . (6.21)

Here, X t,x denotes the unique strong solution of (3.1) with initial condition
Xtt,x = x. Then, the optimal stopping problem is defined by:
τ ∈T[t,T ]
The next result derives the dynamic programming equation for the previous
optimal stopping problem in the sense of viscosity solution, thus relaxing the
C 1,2 regularity condition in the statement of Theorem 3.4. As usual, the same
methodology allows to handle seemingly more general optimal stopping prob-
lems:
V̄ (t, x) := ¯ x, τ ),
sup J(t, (6.23)
τ ∈T[t,T ]
where
"Z #
T
¯ x, τ )
J(t, := E β(t, s)f (s, Xst,x )ds + β(t, τ )g(Xτt,x )1{τ <∞} ,
t
Z s
β(t, s) := exp − k(u, Xut,x )du .
t
Theorem 6.7. Assume that V is locally bounded, and let g : Rn −→ R be

continuous. Then V is a viscosity solution of the obstacle problem:
min {−(∂t + A)V , V − g} = 0 on S. (6.24)
Proof. (i) We first show that V is a viscosity supersolution. As in the proof of

Theorem 3.4, the inequality V − g ≥ 0 is obvious, and implies that V∗ ≥ g. Let
(t0 , x0 ) ∈ S and ϕ ∈ C 2 (S) be such that
0 = (V∗ − ϕ)(t0 , x0 ) = min(V∗ − ϕ).

S
To prove that −(∂t + A)ϕ(t0 , x0 ) ≥ 0, we consider a sequence (tn , xn )n≥1 ⊂

[t0 − h, t0 + h] × B, for some small h > 0, such that
(tn , xn ) −→ (t0 , x0 ) and V (tn , xn ) −→ V∗ (t0 , x0 ).
Let (hn )n be a sequence of positive scalars converging to zero, to be fixed later,

and introduce the stopping times
θhnn := inf t > tn : (t, Xttn ,xn ) 6∈ [tn − hn , tn + hn ] × B .

Then θhnn ∈ T[ttn ,T ] for sufficiently small hn , and it follows from (3.10) that:
h i
V (tn , xn ) ≥ E V∗ θhnn , Xθhnn .
Since V∗ ≥ ϕ, and denoting ηn := (V − ϕ)(tn , xn ), this provides

h i
ηn + ϕ(tn , xn ) ≥ E ϕ θhnn , Xθhnn where ηn −→ 0.
We continue by fixing
p
hn := |ηn |1{ηn 6=0} + n−1 1{ηn =0} ,
as in the proof of Proposition 6.2. Then, the rest of the proof follows exactly the
line of argument of the proof of Theorem 3.4 combined with that of Proposition
6.2.
(ii) We next prove that V is a viscosity subsolution of the equation (6.24). Let
(t0 , x0 ) ∈ S and ϕ ∈ C 2 (S) be such that
0 = (V ∗ − ϕ)(t0 , x0 ) = strict max (V ∗ − ϕ),

S
assume to the contrary that
(V ∗ − g)(t0 , x0 ) > 0 and −(∂t + A)ϕ(t0 , x0 ) > 0,
and let us work towards a contradiction of the weak dynamic programming

principle.
6.2. DPE for optimal stopping 99
Since g is continuous, and V ∗ (t0 , x0 ) = ϕ(t0 , x0 ), we may finds constants

h > 0 and δ > 0 so that
ϕ ≥ g + δ and − (∂t + A)ϕ ≥ 0 on Nh := [t0 , t0 + h] × hB, (6.25)
where B is the unit ball centered at x0 . Moreover, since (t0 , x0 ) is a strict
maximizer of the difference V ∗ − ϕ:
−γ := max(V ∗ − ϕ) < 0. (6.26)
∂Nh
let (tn , xn ) be a sequence in S such that

(tn , xn ) −→ (t0 , x0 ) and V (tn , xn ) −→ V ∗ (t0 , x0 ).
We next define the stopping times:
t, Xttn ,xn ∈

θn := inf t > tn : 6 Nh ,
and we continue as in Step 2 of the proof of Theorem 3.4. We denote ηn :=
V (tn , xn ) − ϕ(tn , xn ), and we compute by Itô’s formula that for an arbitrary
t
stopping rule τ ∈ T[t,T ]:
V (tn , xn ) = ηn + ϕ(tn , xn )
" #
Z τ ∧θn
= ηn + E ϕ (τ ∧ θn , Xτ ∧θn ) − (∂t + A)ϕ(t, Xt )dt ,
tn
where the diffusion term has zero expectation because the process (t, Xttn ,xn ) is
confined to the compact subset Nh on the stochastic interval [tn , τ ∧ θn ]. Since
−(∂t + A)ϕ ≥ 0 on Nh by (6.25), this provides:

V (tn , xn ) ≥ ηn + E ϕ (τ, Xτ ) 1{τ <θn } + ϕ (θn , Xθn ) 1{τ ≥θn }
≥ ηn + E (g (Xτ ) + δ) 1{τ <θn } + (V ∗ (θn , Xθn ) + γ) 1{θn ≥τ }

≥ ηn + γ ∧ δ + E g (Xτ ) 1{τ <θn } + V ∗ (θn , Xθn ) 1{θn ≥τ } ,

where we used the fact that ϕ ≥ g + δ on Nh by (6.25), and ϕ ≥ V ∗ + γ on

t
∂Nh by (6.26). Since ηn := (V − ϕ)(tn , xn ) −→ 0 as n → ∞, and τ ∈ T[t,T ] is
arbitrary, this provides the desired contradiction of (3.9). ♦
6.3 A comparison result for obstacle problems

In this section, we derive a comparison result for the obstacle problem:

min F (., u, ∂t u, Du, D2 u), u − g = 0 on [0, T ) × Rd
(6.27)
u(T, .) = g.
The dynamic programming equation of the optimal stopping problem (6.23)
corresponds to the particular case:
1
F (., u, ∂t u, Du, D2 u) = ∂t u + b · Du + Tr σσ T D2 u − ku + f.

2
Theorem 6.8. Let F be a uniformly continuous elliptic operator satisfying

Assumption 5.22. Let u ∈ USC(O) and v ∈ LSC(O) be viscosity subsolution
and supersolution of the equation (6.27), respectively, with sub-quadratic growth.
Then
u(T, .) ≤ v(T, .) =⇒ u ≤ v on [0, T ] × Rd .
Proof. This is an easy adaptation of the proof of Theorem 5.23. We adapt the
same notations so that, in the present, x stands for the pair (t, x). The only
difference appears at Step 3 which starts from the fact that
min {F (xn , u(xn ), αn (xn − yn ) + εxn , An ), u(xn ) − g(xn )} ≤ 0,

min {F (yn , v(yn ), αn (xn − yn ) − εyn , Bn ), v(yn ) − g(yn )} ≥ 0,
This leads to two cases:

- Either u(xn )−g(xn ) ≤ 0 along some subsequence. Then the inequality v(yn )−
g(yn ) ≥ 0 leads to a contradiction of (5.25).
- Or F (xn , u(xn ), αn (xn − yy ) + εxn , An ) ≤ 0, which can be combined with the
supersolution part F (yn , v(yn ), αn (xn − yy ) − εyn , Bn ) ≥ 0 exactly as in the
proof of Theorem 5.23, and leads to a contradiction of (5.25). ♦
Chapter 7
Stochastic Target
Problems
7.1 Stochastic target problems

In this section, we study a special class of stochastic target problems which
avoids to face some technical difficulties, but reflects in a transparent way the
main ideas and arguments to handle this new class of stochastic control prob-
lems.
All of the applications that we will be presenting fall into the framework of
this section. The interested readers may consult the references at the end of
this chapter for the most general classes of stochastic target problems, and their
geometric formulation.
7.1.1 Formulation
Let T > 0 be the finite time horizon and W = {Wt , 0 ≤ t ≤ T } be a d-
dimensional Brownian motion defined on a complete probability space (Ω, F, P).
We denote by F = {Ft , 0 ≤ t ≤ T } the P-augmentation of the filtration gener-
ated by W .
We assume that the control set U is a convex compact subset of Rd with
non-empty interior, and we denote by U the set of all progressively measurable
processes ν = {νt , 0 ≤ t ≤ T } with values in U .
The state process is defined as follow: given the initial data z = (x, y) ∈
Rd × R, an initial time t ∈ [0, T ], and a control process ν ∈ U, let the controlled
process Z t,z,ν = (X t,x,ν , Y t,z,ν ) be the solution of the stochastic differential
equation :
dXut,x,ν µ u, Xut,x,ν , νu du + σ u, Xut,x,ν , νu dWu ,

=
dYut,z,ν = b u, Zut,z,ν , νu du + νu · dW (u), u ∈ (t, T ) ,

101
102 CHAPTER 7. STOCHASTIC TARGET PROBLEMS
with initial data
Xtt,x,ν = x, and Ytt,x,y,ν = y.
Here, µ : S×U −→ Rd , σ : S×U −→ Sd , and b : S×R×U −→ R are continuous

functions, Lipschitz in (x, y) uniformly in (t, u). Then, all above processes are
well defined for every admissible control process ν ∈ U0 defined by
Z t
2 2

U0 := ν∈U : E |µ0 (s, νs )| + |b0 (s, νs )| + |σ0 (s, νs )| + |νs | ds .
0
Throughout this section, we assume that the the function
u 7−→ σ(t, x, u)p
has a unique fixed point for every (t, x) ∈ S̄ × R defined by:
σ(t, x, u)p = u ⇐⇒ u = ψ(t, x, p). (7.1)
For a measurable function g : Rd −→ R, we define the stochastic target problem

by:
V (t, x) := inf y ∈ R : YTt,x,y,ν ≥ g XTt,x,ν , P − a.s. for some ν ∈ U0 .

(7.2)
Remark 7.1. By introducing the subset of control processes:
ν ∈ U0 : YTt,x,y,ν ≥ g XTt,x,ν , P − a.s. ,

A(t, x, y) :=
we may re-write the value function of the stochastic target problem into:
V (t, x) = inf Y(t, x), where Y(t, x) := {y ∈ R : A(t, x, y) 6= ∅} .
The set Y(t, x) satisfies the following important property :
for all y ∈ R, y ∈ Y(t, x) =⇒ [y, ∞) ⊂ Y(t, x).
This follows from the fact that the state process X t,x,ν is independent of y and
YTt,x,y,ν is nondecreasing in y.
7.1.2 Geometric dynamic programming principle

Similar to the standard class of stochastic control and optimal stopping prob-
lems studied in the previous chapters, the main tool for the characterization of
the value function and the solution of stochastic target problems is the dynamic
programming principle. Although the present problem does not fall into the
class of problems studied in the previous chapters, the idea of dynamic pro-
gramming is the same: allow the time origin to move, and deduce a relation
between the value function at different points in time.
In these notes, we shall only use the easy direction of a more general geo-
metric dynamic programming principle.
7.1. Stochastic target 103
Theorem 7.2. Let (t, x) ∈ [0, T ] × Rd and y ∈ R such that A(t, x, y) 6= ∅.

Then, for any control process ν ∈ A(t, x, y) and stopping time τ ∈ T[t,T ] ,
Yτt,x,y,ν ≥ V τ, Xτt,x,ν , P − a.s.

(7.3)
Proof. Let z = (x, y) and ν ∈ A(t, z), and denote Zt,z,ν := (X t,x,ν , Y t,z,ν ).
t,z,ν t,x,ν
Then YT ≥ g XT P−a.s. Notice that
τ,Zτt,z,ν ,ν
ZTt,z,ν = ZT .
Then, by the definition of the set A, it follows that ν ∈ A (τ, Zτt,z,ν ), and
therefore V (τ, Xτt,x,ν ) ≤ Yτt,z,ν , P−a.s. ♦
In the next subsection, we will prove that the value function V is a viscosity
supersolution of the corresponding dynamic programming equation which will
be obtained as the infinitesimal counterpart of (7.3). The following remark
comments on the full geometric dynamic programming principle in the context
of stochastic target problems.
Remark 7.3. The statement (7.3) in Theorem 7.2 can be strengthened to the
following geometric dynamic programming principle:
n o
V (t, x) = inf y ∈ R : Y t,x,y,ν ≥ V τ, Xτt,x,ν , P − a.s. for some ν ∈ U0 .

(7.4)
Let us provide an intuitive justification of this. Denote ŷ := V (t, x). In view of
(7.3), it is easily seen that (7.4) is implied by
P Yτt,x,ŷ−η,ν > V τ, Xτt,x,ν

< 1 for all ν ∈ U0 and η > 0.
In words, there is no control process ν which allows to reach the value function
V (τ, Xτt,x,ν ) at time τ , with full probability, starting from an initial data stricly
below the value function V (t, x). To see this, suppose to the contrary that there
exist ν ∈ U0 , η > 0, and τ ∈ T[t,T ] such that:
Yτt,x,ŷ−η,ν V τ, Xτt,x,ν , P − a.s.

>
In view of Remark 7.1, this implies that Yτt,x,ŷ−η,ν ∈ Y (τ, Xτt,x,ν ), and therefore,
there exists a control ν̂ ∈ U0 such that
τ,Z t,x,ŷ−η,ν ,ν̂ τ,X t,x,ν ,ν̂

YT τ ≥ g XT τ , P − a.s.
t,x,y ∗ −η,ν
t,x,ν

Since the process X τ,Xτ ,ν̂
, Y τ,Zτ ,ν̂
depends on ν̂ only through its
realizations in the stochastic interval [t, θ], we may chose ν̂ so as ν̂ = ν on
τ,Z t,x,ŷ−η,ν ,ν̂
[t, τ ] (this is the difficult part of this proof). Then ZT τ = ZTt,x,ŷ−η,ν̂
and therefore ŷ − η ∈ Y(t, x), hence ŷ − η ≤ V (t, x). Recall that by definition
ŷ = V (t, x) and η > 0. ♦
7.1.3 The dynamic programming equation

In order to have a simpler statement and proof of the main result, we assume
in this section that
U is a closed convex subset of Rd , int(U ) 6= ∅ and 0 ∈ U. (7.5)
The formulation of the dynamic programming equation involves the notion of
support function from convex analysis.
a- Dual characterization of closed convex sets We first introduce the

support function of the set U :
δU (ξ) := sup x · ξ, for all ξ ∈ Rd .
x∈U
By definition δU is a convex function Rd . Its effective domain

Ũ := dom(δU ) = {ξ ∈ Rd : δU (ξ) < ∞}
is a closed convex cone of Rd . Since U is closed and convex by (7.5), we have
the following dual characterization:
x∈U if and only if δU (ξ) − x · ξ ≥ 0 for all ξ ∈ Ũ , (7.6)
see e.g. Rockafellar [13]. Morover, since Ũ is a cone, we may normalize the dual
variables ξ on the right hand-side:
x ∈ U if and only if δU (ξ) − x · ξ ≥ 0 for all ξ ∈ Ũ1 := {ξ ∈ Ũ : |ξ| = (7.7)
1}.
This normalization will be needed in our analysis in order to obtain a dual
characterization of int(U ). Indeed, since U has nonempty interior by (7.5), we
have:
x ∈ int(U ) if and only if inf δU (ξ) − x · ξ > 0. (7.8)
ξ∈Ũ1
b- Formal derivation of the DPE We start with a formal derivation of the

dynamic programming equation which provides the main intuitions.
To simplify the presentation, we suppose that the value function V is smooth
and that existence holds, i.e. for all (t, x) ∈ S, there is a control process ν̂ ∈ U0
such that, with z = (x, V (t, x)), we have YTt,z,ν̂ ≥ g(XTt,x,ν̂ ), P−a.s. Then it
follows from the geometric dynamic programming of Theorem 7.2 that, P−a.s.
Z t+h Z t+h
t,z,ν t,z,ν̂
t,x,ν̂
Yt+h = v(t, x) + b s, Zs , ν̂s ds + ν̂s · dWs ≥ V t + h, Xt+h .
t t
By Itô’s formula, this implies that

Z t+h
−∂t V (s, Xst,x,ν̂ ) + H s, Zst,z,ν̂ , DV (s, Xst,x,ν̂ ), D2 V (s, Xst,x,ν̂ ), ν̂s ds

0 ≤
t
Z t+h
N νs s, Xst,x,ν̂ , DV (s, Xst,x,ν̂ ) · dWs ,

+ (7.9)
t
where we introduced the functions:

1
:= b(t, x, y, u) − µ(t, x, u) · p − Tr σ(t, x, u)2 A (7.10)

H(t, x, y, p, A, u) ,
2
N u (t, x, p) := u − σ(t, x, u)p. (7.11)
We continue our intuitive derivation of the dynamic programming equation by

assuming that all terms inside the integrals are bounded (we know that this
can be achieved by localization). Then the first integral behaves like Ch, while
the second integral can be viewed as a time changed Brownian motion. By the
properties of the Brownian motion, it follows that the integrand of the stochastic
integral term must be zero at the origin:
Ntνt (t, x, DV (t, x)) = 0 or, equivalently νt = ψ t, x, DV (t, x) , (7.12)

where ψ was introduced in (7.1). In particular, this implies that ψ(t, x, DV (t, x)) ∈
U . By (7.7), this is equivalent to:
δU (ξ) − ξ · ψ(t, x, DV (t, x)) ≥ 0 for all ξ ∈ Ũ1 . (7.13)
By taking expected values in (7.9), normalizing by h, and sending h to zero, we

see that:
−∂t V (t, x) + H t, x, V (t, x), DV (t, x), D2 V (t, x), ψ(t, x, DV (t, x)) ≥ (7.14)

0.
Putting (7.13) and (7.14) together, we obtain

min −∂t V + H ., V, DV, D2 V, ψ(., DV ) , inf

δU (ξ) − ξ · ψ(., DV ) ≥ 0.
ξ∈Ũ1
By using the second part of the geometric dynamic programming principle,

see Remark 7.3, we expect to prove that equality holds in the latter dynamic
programming equation.
c- The dynamic programming equation We next turn to a rigorous deriva-

tion of the dynamic programmin equation.
Theorem 7.4. Assume that V is locally bounded, and let the maps H and ψ
be continuous. Then V is a viscosity supersolution of the dynamic programming
equation on S:

min −∂t V + H ., V, DV, D2 V, ψ(., DV ) , inf δU (ξ) − ξ · ψ(., DV ) = 0

ξ∈Ũ1
Assume further that ψ is locally Lipschitz-continuous, and U has non-empty

interior. Then V is a viscosity solution of the above DPE on S.
Proof. As usual, we prove separately the supersolution and the subsolution
properties.
1. Supersolution: Let (t0 , x0 ) ∈ S and ϕ ∈ C 2 (S) be such that

(strict) min(V∗ − ϕ) = (V∗ − ϕ)(t0 , x0 ) = 0,
S
and assume to the contrary that

−∂t V + H ., V, DV, D2 V, ψ(., DV ) (t0 , x0 ) < 0.

−2η := (7.15)
(1-i) By the continuity of H and ψ, we may find ε > 0 such that

−∂t V (t, x) + H t, x, y, DV (t, x), D2 V (t, x), u ≤ −η (7.16)
for (t, x) ∈ Bε (t0 , x0 ), |y − ϕ(t, x)| ≤ ε, and u ∈ U s.t. |N u (t, x, p)| ≤ ε.
Notice that (7.16) is obviously true if {u ∈ U : |N u (t, x, p)| ≤ ε} = ∅, so that
the subsequent argument holds in this case as well.
Since (t0 , x0 ) is a strict minimizer of the difference V∗ − ϕ, we have
γ := min (V∗ − ϕ) > 0. (7.17)
∂Bε (t0 ,x0 )
(1-ii) Let (tn , xn )n ⊂ Bε (t0 , x0 ) be a sequence such that

(tn , xn ) −→ (t0 , x0 ) and V (tn , xn ) −→ V∗ (t0 , x0 ), (7.18)
and set yn := xn + n−1 and zn := (xn , yn ). By the definition of the problem
V (tn , xn ), there exists a control process ν̂ n ∈ U0 such that the process Z n :=
n
Z tn ,zn ,ν̂ satisfies YTn ≥ g(XTn ), P−a.s. Consider the stopping times
θn0 := inf {t > tn : (t, Xtn ) 6∈ Bε (t0 , x0 )} ,
θn := θn0 ∧ inf {t > tn : |Ytn − ϕ(t, Xtn )| ≥ ε}
Then, it follows from the geometric dynamic programming principle that
n n

Yt∧θ n
≥ V t ∧ θn , Xt∧θ n
.
Since V ≥ V∗ ≥ ϕ, and using (7.17) and the definition of θn , this implies that
n n

Yt∧θn
≥ ϕ t ∧ θn , Xt∧θ n
+ 1{t=θn } γ1{θn =θn0 } + ε1{θn <θn0 }
n

≥ ϕ t ∧ θn , Xt∧θ n
+ (γ ∧ ε)1{t=θn } . (7.19)
(1-iii) Denoting cn := V (tn , xn ) − ϕ(tn , xn ) − n−1 , we write the process Y n as
Z t Z t
Ytn = cn + ϕ(tn , xn ) + b(s, Zsn , ν̂sn )ds + ν̂sn · dWs .
tn tn
Plugging this into (7.19) and applying Itô’s formula, we then see that:
Z t∧θn Z t∧θn
n
(ε ∧ γ)1{t=θn } ≤ cn + δsn ds + N ν̂s (s, Xsn , Dϕ(s, Xsn )) · dWs
tn tn
Z t∧θn
≤ Mn := cn + δsn 1An (s)ds (7.20)
tn
Z t∧θn
n
+ N ν̂s (s, Xsn , Dϕ(s, Xsn )) · dWs
tn
where
δsn := −∂t ϕ(s, Xsn ) + H s, Zsn , Dϕ(s, Xsn ), D2 ϕ(s, Xsn ), ν̂s

and
An := {s ∈ [tn , tn + θn ] : δsn > −η} .

n
By (7.16), observe that the diffusion term ζsn := N ν̂s (s, Xsn , Dϕ(s, Xsn )) in
(7.20) satisfies |ζsn | ≥ η for all s ∈ An . Then, by introducing the exponential
local martingale Ln defined by
Lntn = 1 and dLnt = Lnt |ζtn |−2 ζtn · dWt ,
we see that the process M n Ln is a local martingale which is bounded from below
by the constant ε ∧ γ. Then M n Ln is a supermartingale, and it follows from
(7.20) that
ε ∧ γ ≤ E Mθnn Lnθn ≤ Mtnn Lntn = cn ,

which can not happen because cn −→ 0. Hence, our starting point (7.22) can
not happen, and the proof of the supersolution property is complete.
2. Subsolution: Let (t0 , x0 ) ∈ S and ϕ ∈ C 2 (S) be such that
(strict) max(V ∗ − ϕ) = (V ∗ − ϕ)(t0 , x0 ) = 0, (7.21)

S
and assume to the contrary that

2η := −∂t ϕ + H ., ϕ, Dϕ, D2 ϕ, ψ(., ϕ) (t0 , x0 ) > 0,
(7.22)
and inf ξ∈Ũ1 δU (ξ) − ξ · ψ(., Dϕ) (t0 , x0 ) > 0.
(2-i) By the continuity of H and ψ, and the characterization of int(U ) in (7.8),

it follows from (7.22) that

−∂t ϕ + H ., y, Dϕ, D2 ϕ, ψ(., Dϕ) ≥ η and ψ(., Dϕ) ∈ U
for (t, x) ∈ Bε (t0 , x0 ) and |y − ϕ(t, x)| ≤ ε (7.23)
Also, since (t0 , x0 ) is a strict maximizer in (7.21), we have
−ζ := max (V ∗ − ϕ) < 0, (7.24)

∂p Bε (t0 ,x0 )
where ∂p Bε (t0 , x0 ) := {t0 + ε} × cl (Bε (t0 , x0 )) ∪ [t0 , t0 + ε) × ∂Bε (x0 ) denotes

the parabolic boundary of Bε (t0 , x0 ).
(2-ii) Let (tn , xn )n be a sequence in S which converges to (t0 , x0 ) and such that
V (tn , xn ) → V ∗ (t0 , x0 ). Set yn = V (tn , xn ) − n−1 and observe that
γn := yn − ϕ(tn , xn ) −→ 0. (7.25)
Let Z n := (X n , Y n ) denote the controlled state process associated to the Marko-

vian control ν̂tn = ψ(t, Xtn , Dϕ(t, Xtn )) and the initial condition Ztnn = (xn , yn ).
Since ψ is locally Lipschitz-continuous, the process Z n is well-defined. We next
define the stopping times
θn0 := inf {s ≥ tn : (s, Xsn ) ∈
/ Bε (t0 , x0 )} ,
θn := θn0 ∧ inf {s ≥ tn : |Y n (s) − ϕ(s, Xsn )| ≥ ε} .
By the first line in (7.23), (7.25) and a standard comparison theorem, it follows
that Yθnn − ϕ(θn , Xθnn ) ≥ ε on {|Yθnn − ϕ(θn , Xθnn )| ≥ ε} for n large enough. Since
V ≤ V ∗ ≤ ϕ, we then deduce from (7.24) and the definition of θn that

Yθnn − V (θn , Xθnn ) ≥ 1{θn <θn0 } Yθnn − ϕ θn , Xθnn

+1{θn =θn0 } Yθnn0 − V ∗ θn0 , Xθnn0

= ε1{θn <θn0 } + 1{θn =θno } Yθnn0 − V ∗ θn0 , Xθnn0

≥ ε1{θn <θn0 } + 1{θn =θn0 } Yθnn0 + ζ − ϕ θn0 , Xθnn0

≥ ε ∧ ζ + 1{θn =θn0 } Yθnn0 − ϕ θn0 , Xθnn0 .
We continue by using Itô’s formula:

!
Z θn
Yθnn −V (θn , Xθnn ) ≥ ε ∧ ζ + 1{θn =θn0 } γn + α(s, Xsn , Ysn )ds
tn
where the drift term α(·) ≥ η is defined in (7.23) and the diffusion coefficient
vanishes by the definition of the function ψ in (7.1). Since ε, ζ > 0 and γn → 0,
this implies that
Y n (θn ) ≥ V (θn , X n (θn )) for sufficiently large n.
Recalling that the initial position of the process Y n is yn = V (tn , xn ) − n−1 <
V (tn , xn ), this is clearly in contradiction with the second part of the geometric
dynamic programming principle discussed in Remark 7.3. ♦
7.1.4 Application: hedging under portfolio constraints

As an application of the previous results, we now study the problem of super-
hedging under portfolio constraints in the context of the Black-Scholes model.
a- Formulation We consider a financial market consisting of d + 1 assets.

The first asset X 0 is nonrisky, and is normalized to unity. The d next assets are
risky with price process X = (X 1 , . . . , X d )T defined by the Blac-Scholes model:
dXt = Xt ? σdWt ,
where σ is a constant symmetric nondegenrate matrix in Rd , and x ? σ is the
square matrix in Rd with entries (x ? σ)i,j = xi σi,j .
Remark 7.5. We observe that the normalization of the first asset to unity
does not entail any loss of generality as we can always reduce to this case by
discounting or, in other words, by taking the price process of this asset as a
numéraire.
Also, the formulation of the above process X as a martingale is not a re-
striction as our subsequent superhedging problem only involves the underlying
probability measure through the corresponding zero-measure sets. Therefore,
under the no-arbitrage condition (or more precisely, no free-lunch with vanish-
ing risk), we can reduce the model to the above martingale case by an equivalent
change of measure. ♦
Under the self-financing condition, the liquidation value of the portfolio is
defined by the controlled state process:
dYtπ = σπt · dWt ,
where π is the control process, with πti representing the amount invested in the
i−th risky asset X i at time t.
We introduce portfolio constraints by imposing that the portfolio process π
must be valued in a subset U of Rd . We shalll assume that
U is closed convex subset of Rd , int(U ) = ∅, and 0 ∈ U. (7.26)
We then define the controls set by Uo as in the previous sections, and we defined
the superhedging problem under portfolio constraints by the stochastic target
problem:
V (t, x) := inf y : YTt,y,π ≥ g(XTt,x ), P − a.s. for some π ∈ U0 , (7.27)

where g : Rd+ −→ R+ is a non-negative LSC function with linear growth.

We shall provide an explicit solution of this problem by only using the su-
persolution claim from Theorem 7.4. This will provide a minorant of the su-
perhedging cost V . To provide that this minorant is indeed the desired value
function, we will use a verification argument.
b- Deriving a minorant of the superhedging cost First, since 0 ≤ g(x) ≥

C(1+|x|) for some constant C > 0, we deduce that 0 ≤ V ≥ C(1+|x|), the right
hand-side inequality is easily justified by the buy-and-hold strategy suggested
by the linear upper bound. Then, by a direct application of the first part of
Theorem 7.4, we know that the LSC envelope V∗ of V is a supersolution of the
DPE:
−∂t V∗ − 21 Tr[(x ? σ)2 D2 V∗ ] ≥ 0 (7.28)

δU (ξ) − ξ · (x ? DV∗ ) ≥ 0 for all ξ ∈ Ũ . (7.29)
Notice that (7.29) is equivalent to:
the map λ 7−→ h(λ) := λδU (ξ) − V∗ (t, x ? eλξ ) is nondecreasing, (7.30)
where eλξ is the vector of Rd with entries (eλξ )i = eλξi . Then h(1) ≥ h(0)
provides:
V∗ (t, x) ≥ sup V∗ x ? eξ − δU (ξ).

ξ∈Ũ
We next observe that V∗ (T, .) ≥ g (just use the definition of V , and send t % T ).
Then, we deuce from the latter inquality that
V∗ (T, x) ≥ ĝ(x) := sup g x ? eξ − δU (ξ) for all x ∈ Rd+ .

(7.31)
ξ∈Ũ
In other words, in order to superhedge the derivative security with final payoff
g(XT ), the constraints on the portfolio require that one hedges the derivative
security with larger payoff ĝ(XT ). The function ĝ is called the face-lifted payoff,
and is the smallest majorant of g which satisfies the gradient constraint x ?
Dg(x) ∈ U for all x ∈ Rd+ .
Combining (7.31) with (7.28), it follows from the comparison result for the
linear Black-Scholes PDE that
V (t, x) ≥ V∗ (t, x) ≥ v(t, x) := E ĝ(XTt,x ) for all (t, x) ∈ S. (7.32)

c- Explicit solution Our objective is now to prove that V = v. To see this

consider the Black-Scholes hedging strategy π̂ of the derivative security ĝ(XTt,x ):
Z T
v(t, x) + π̂s · σdWs = ĝ(XTt,x ).
t
Since ĝ has linear growth, it follows that π̂ ∈ H2 . We also observe that the
random variable ln XTt,x is gaussian, so that the function v can be written in:
2
w−x+ 1 σ 2 (T −t)

Z
1 − 21 √2
w σ T −t
v(t, x) = ĝ(e ) p e dw.
2πσ 2 (T − t)
Under this form, it is clear that v is a smooth function. Then the above hedging
portfolio is given by:
π̂s := Xst,x ? DV (s, Xst,x )
Notice that, for all ξ ∈ Ũ ,

h λξ
i
λδU (ξ) − v(t, xeλξ ) = E λδU (ξ) − ĝ XTt,xe
is nondecreasing in λ by applying (7.30) to ĝ which, by definition satisfies x ?

Dg(x) ∈ U for all x ∈ Rd+ . Then, x ? Dg(x) ∈ U , and therefore the above
replicating portfolio π̂ takes values in U . Since ĝ ≥ g, we deduce from (7.31)
that v ≥ V .
7.2. Quantile stochastic target problems 111
7.2 Stochastic target problem with controlled

probability of success
In this section, we extend the model presented above to the case where the
target has to be reached only with a given probability p:
V̂ (t, x, p) := inf y ∈ R+ : P YTt,x,y,ν ≥ g(XTt,x,ν ) ≥ p for some ν ∈ U0 (7.33)

.
In order to avoid degenerate results, we restrict the analysis to the case where the
Y process takes non-negative values, by simply imposing the following conditions
on the coefficients driving its dynamics:
b(t, x, 0, u) ≥ 0 for all(t, x) ∈ S, u ∈ U. (7.34)
Notice that the above definition implies that
0 = V̂ (., 0) ≤ V̂ ≤ V̂ (., 1) = V, (7.35)
and
V̂ (., p) = 0 for p < 0 and V̂ (., p) = ∞ for p > 1 , (7.36)
with the usual convention inf ∅ = ∞.
7.2.1 Reduction to a stochastic target problem

Our first objective is to convert this problem into a (standard) stochastic target
problem, so as to apply the geometric dynamic programming arguments of the
previous section.
To do this, we introduce an additional controlled state variable:
Z s
t,p,α
Ps := p + αr · dWr , for s ∈ [t, T ], (7.37)
t
where the additional control α is an F−progressively measurable Rd −valued

RT
process satisfying the integrability condition E[ 0 |αs |2 ds] < ∞. We then set
X̂ := (X, P ), Ŝ := [0, T ) × Rd × (0, 1), Û := U × Rd , and denote by Û the
corresponding set of admissible controls. Finally, we introduce the function:
Ĝ(x̂, y) := 1{y≥g(x)} − p for y ∈ R, x̂ := (x, p) ∈ Rd × [0, 1].
Proposition 7.6. For all t ∈ [0, T ] and x̂ = (x, p) ∈ Rd × [0, 1], we have
n o
V̂ (t, x̂) = inf y ∈ R+ : Ĝ X̂Tt,x̂,ν̂ , YTt,x,y,ν ≥ 0 for some ν̂ = (ν, α) ∈ Û .
Proof. We denote by v(t, x, p) the value function apprearing on the right-hand.

We first show that V̂ ≥ y > V̂ (t, x, p), we can find ν ∈ U such that
v. For
p0 := P G XTt,x,ν , YTt,x,y,ν ≥ 0 ≥ p. By the stochastic integral representation

theorem, there exists an F-progressively measurable process α such that

Z T Z T
1{Y t,x,y,ν ≥g(X t,x,ν )} = p0 + αs · dWs = PTt,p0 ,α and E[ |αs |2 ds] < ∞.
T T
t t
α
Since p0 ≥ p, it follows that 1{Y t,x,y,ν ≥g(X t,x,ν ,)} ≥ Pt,p (T ), and therefore
T T
y ≥ v(t, x, p) from the definition of the problem v.
We next show that v ≥ V̂ . For y > v(t, x, p), we have Ĝ X̂Tt,x̂,ν̂ , YTt,x,y,ν ≥
α
0 for some ν̂ = (ν, α) ∈ Û. Since Pt,p is a martingale, it follows that
h i
P YTt,x,y,ν ≥ g XTt,x,ν = E 1{Y t,x,y,ν ≥g(X t,x,ν )} ≥ E PTt,p,α = p,

T T
which implies that y ≥ V̂ (t, x, p) by the definition of V̂ . ♦

Remark 7.7. 1. Suppose that the infimum in the definition of V̂ (t, x, p)
is
achieved and there exists a control ν ∈ U0 satisfying P YTt,x,y,ν ≥ g XTt,x,ν =

p, the above argument shows that:

h i
Pst,p,α = P YTt,x,y,ν ≥ g XTt,x,ν Fs for all s ∈ [t, T ].
2. It is easy to show that one can moreover restrict to controls α such that
the process P t,p,α takes values in [0, 1]. This is rather natural since this process
should be interpreted as a conditional probability, and this corresponds to the
natural domain [0, 1] of the variable p. We shall however avoid to introduce this
state constraint, and use th efact that the value function V̂ (·, p) is constant for
p ≤ 0 and equal ∞ for p > 1, see (7.36).
7.2.2 The dynamic programming equation

The above reduction of the problem V̂ to a stochastic target problem allows to
apply the geometric dynamic programming principle of the previous section, and
to derive the corresponding dynamic programming equation. For û = (u, α) ∈ Û
and x̂ = (x, p) ∈ Rd × [0, 1], set

µ(x, u) σ(x, u)
µ̂(x̂, û) := , σ̂(x̂, û) := .
0 αT
For (y, q, A) ∈ R × Rd+1 × Sd+1 and û = (u, α) ∈ Û ,
N̂ û (t, x̂, y, q) := u − σ̂(t, x̂, û)q = N u (t, x, qx ) − qp α for q = (qx , qp ) ∈ Rd × R,
and we assume that
u 7−→ N u (t, x, qx ) is one-to-one, with inverse function ψ(t, x, qx , .) (7.38)

Then, by a slight extension of Theorem 7.4, the corresponding dynamic pro-

gramming equation is given by:
n
0 = −∂t V̂ + sup b(., V̂ , ψ(., Dx V̂ , αDp V̂ )) − µ(., ψ(., Dx V̂ , αDp V̂ )).Dx V̂
α
1 h i
− Tr σ(., ψ(., Dx V̂ , αDp V̂ ))2 Dx2 V̂
2
1 2 2 o
− α Dp V̂ − ασ(., ψ(., Dx V̂ , αDp V̂ ))Dxp V̂
2
7.2.3 Application: quantile hedging in the Black-Scholes

model
The problem of quantile hedging was solved by Föllmer and Leukert [7] in the
general model of asset prices process (non-necessarilly Markov), by means of the
Neyman-Pearson lemma from mathematical statistics. The stochastic control
approach developed in the present section allows to solve this type of problems in
a wider generality. The objective of this section is to recover the explicit solution
of [7] in the context of a complete financial market where the underlying risky
assets prices are not affected by the control:
µ(x, u) = µ(x) and σ(x, u) = σ(x) are independent of u, (7.39)
where µ and σ are Lipschitz-continuous, and σ(x) is invertible for all x.

Notice that we will be only using the supersolution property from the results
of the previous sections.
a- The financial market The process X, representing the price process of d

risky assets, is defined by Xtt,x = x ∈ (0, ∞)d , and
dXst,x = Xst,x ? σ(Xst,x ) λ(Xst,x )ds + dWs where λ := σ −1 µ.

We assume that the coefficients µ and σ are such that X t,x ∈ (0, ∞)d P−a.s.
for all initial conditions (t, x) ∈ [0, T ] × (0, ∞)d . In order to avoid arbitrage, we
also assume that σ is invertible and that
sup |λ(x)| < ∞ where . (7.40)

x∈(0,∞)d
The drift coefficient of the controlled process Y is given by:
b(t, x, y, u) = u · λ(x). (7.41)
The control process ν is valued in U = Rd , with components νsi indicating the

dollar investment in the i−th security at time s. After the usual reduction of
the interest rates to zero, it follows from the self-financing condition that the
liquidation value of the portfolio is given by
Z s
Yst,x,y,ν = y + νr · σ(Xst,x ) λ(Xst,x )ds + dWs ,

s≥t,
t
b- The quantile hedging problem The quantile hedging problem of the

derivative security g(Xt,x (T )) is defined by the stochastic target problem with
controlled probability of success:
V̂ (t, x, p) := inf y ∈ R+ : P YTt,x,y,ν ≥ g(XTt,x ) ≥ p for some ν ∈ U0 .

We shall assume throughout that 0 ≤ g(x) ≤ C(1 + |x|) for all x ∈ Rd+ . By the
usual buy-and-hold hedging strategies, this implies that 0 ≤ V (t, x) ≤ C(1+|x|).
Under the above assumptions, the corresponding super-hedging cost V (t, x) :=
V̂ (t, x, 1) is continuous and is given by
t,x
g(XTt,x ) ,

V (t, x) = EQ
where Qt,x is the P-equivalent martingale measure defined by

!
T T
dQt,x
Z Z
1
= exp − |λ(Xst,x )|2 ds − λ(Xst,x ) · dWs .
dP 2 t t
In particular, V is a viscosity solution on [0, T ) × (0, ∞)d of the linear PDE:
1
= −∂t V − Tr σ 2 Dx2 V .

0 (7.42)
2
For later use, let us denote by
Z ·
t,x
WQ := W + λ(Xst,x )ds, s ∈ [t, T ],
t
the Qt,x -Brownian motion defined on [t, T ].
c- The viscosity supersolution property By the results of the previous

section, we have V̂∗ is a viscosity supersolution on [0, T ) × Rd+ × [0, 1] of the
equation

1 h i h i 1
0 ≤ −∂t V̂∗ − Tr σσ T Dx2 V̂∗ − inf −αλDp V̂∗ + Tr σαDxp V̂∗ + |α|2 Dp2 V̂∗ .
2 α∈Rd 2
(7.43)
The boundary conditions at p = 0 and p = 1 are immediate:
V̂∗ (·, 1) = V and V̂∗ (·, 0) = 0 on [0, T ] × Rd+ . (7.44)
We next determine the boundary condition at the terminal time t = T .
Lemma 7.8. For all x ∈ Rd+ and p ∈ [0, 1], we have V̂∗ (T, x, p) ≥ pg(x).
Proof. Let (tn , xn , pn )n be a sequence in [0, T )×Rd+ ×(0, 1) converging to (T, x, p)

with V̂ (tn , xn , pn ) −→ V̂∗ (T, x, p), and consider yn := V̂ (tn , xn , pn ) + 1/n. By
definition of the quantile hedging problem, there is a sequence (νn , αn ) ∈ Û0

such that
1{Y tn ,xn ,yn ,νn −g(X tn ,xn )≥0} ≥ PTtn ,pn ,αn .
T T
This implies that
YTtn ,xn ,yn ,νn ≥ PTtn ,pn ,αn g(XTtn ,xn ).
Taking the expectation under Qtn ,xn , this provides:

tn ,xn tn ,xn
YTtn ,xn ,yn ,νn PTtn ,pn ,αn g(XTtn ,xn )

yn ≥ EQ ≥ EQ
= E LtTn ,xn PTtn ,pn ,αn g(XTtn ,xn )

R
T RT
where we denotes LTtn ,xn := exp − tn λ(Xstn ,xn ) · dWs − 21 tn |λ(Xstn ,xn )|2 ds .
Then
yn ≥ E PTtn ,pn ,αn g(x) + E PTtn ,pn ,αn LtTn ,xn g(XTtn ,xn ) − g(x)

= pn g(x) + E PTtn ,pn ,αn LtTn ,xn g(XTtn ,xn ) − g(x)

≥ pn g(x) − E P tn ,pn ,αn Ltn ,xn g(X tn ,xn ) − g(x) ,

T T T (7.45)
where we used the fact that P tn ,pn ,αn is a nonnegative martingale. Now, since
this process is also bounded by 1, we have
E PTtn ,pn ,αn LTtn ,xn g(XTtn ,xn ) − g(x) ≤ E LtTn ,xn g(XTtn ,xn ) − g(x) −→ 0

as n → ∞, by the stability properties of the flow and the dominated conver-

gence theorem. Then, by taking limits in (7.45), we obtain that V̂∗ (T, x, p) =
limn→∞ yn ≥ pg(x), which is the required inequality. ♦
d- An explicit minorant of V̂ The key idea is to introduce the Legendre-

Fenchel dual of V∗ with respect to the p−variable in order to remove the non-
linearity in (7.43):
n o
v(t, x, q) := sup pq − V̂∗ (t, x, p) , (t, x, q) ∈ [0, T ] × (0, ∞)d × R. (7.46)
p∈R
By the definition of the function V̂ , we have

n o
v(., q) = ∞ for q < 0 and v(., q) = sup pq − V̂∗ (., p) for q > 0. (7.47)
p∈[0,1]
Using the above supersolution property of V̂∗ , we shall prove below that v is an
upper-semicontinuous viscosity subsolution on [0, T ) × (0, ∞)d × (0, ∞) of
1 1 2
−∂t v − Tr σ 2 Dx2 v − |λ| q 2 Dq2 v − Tr [σλDxq v] ≤ 0 (7.48)
2 2
with the boundary condition

+
v(T, x, q) ≤ (q − g(x)) . (7.49)
Since the above equation is linear, we deduce from the comparison result an
explicit upper bound for v given by the Feynman-Kac representation result:
h
t,x +
i
v(t, x, q) ≤ v̄(t, x, q) := EQt,x Qt,x,q
T − g(XT ) , (7.50)
on [0, T ] × (0, ∞)d × (0, ∞), where the process Qt,x,q is defined by the dynamics
dQt,x,q
s t,x Qt,x
t,x,q = λ(Xs ) · dWs and Qt,x,q (t) = q ∈ (0, ∞). (7.51)
Qs
Given the explicit representation of v̄, we can now provide a lower bound for
the primal function V̂ by using (7.47).
We next deduce from (7.50) a lower bound for the quantile hedging problem
convp
V̂ . Recall that the convex envelop V̂∗ of V̂∗ with respect to p is given by
the bi-conjugate function:
convp
V̂∗ (t, x, p) = sup pq − v(t; x; q) ,
q
and is the largest convex minorant of V̂∗ . Then, since V̂ ≥ V̂∗ , it follows from
(7.50) that:

V̂ (t, x, p) ≥ V̂∗ (t, x, p) ≥ sup pq − v̄(t, x, q) (7.52)
q
Clearly the function v̄ is convex in q and there is a unique solution q̄(t, x, p) to

the equation
∂v̄ h i
(t, x, q̄) = EQt,x Qt,x,1
t,x,q̄
≥ g(XTt,x ) = p,

T 1{ Qt,x, q̄ (T )≥g(XT )} = P QT
t,x
∂q
(7.53)
where we have used the fact that dP/dQt,x = Qt,x,1 T . Then the maximization
on the right hand-side of (7.52) can be solved by the first order condition, and
therefore:
V̂ (t, x, p) ≥ pq̄(t, x, p) − v̄ (t, x, q̄(t, x, p))
t,x
h i
= q̄(t, x, p) p − EQ Qt,x,1
T 1{q̄(t,x,p)Qt,x,1 ≥g(X t,x )}
T T
h i
Qt,x t,x
+E g(XT )1{q̄(t,x,p)Qt,x,1 ≥g(X t,x )}
T T
h i
Qt,x t,x
= E g(XT )1{q̄(t,x,p)Qt,x,1 ≥g(X t,x )} =: y(t, x, p).
T T
e- The explicit solution We finally show that the above explicit minorant
y(t, x, p) is equal to V̂ (t, x, p). By the martingale representation theorem, there
exists a control process ν ∈ U0 such that
t,x,y(t,x,p),ν
≥ g XTt,x 1{q̄(t,x,p)Qt,x,1 ≥g(X t,x )} .

YT
T T
h i
Since P q̄(t, x, p)Qt,x,1
T ≥ g(XTt,x ) = p, by (7.53), this implies that V̂ (t, x, p) =
y(t, x, p).
f- Proof of (7.48)-(7.49) First note that the fact that v is upper-semicontinuous

on [0, T ] × (0, ∞)d × (0, ∞) follows from the lower-semicontinuity of V̂∗ and the
representation in the right-hand side of (7.47), which allows to reduce the com-
putation of the sup to the compact set [0, 1]. Moreover, the boundary condition
(7.49) is an immediate consequence of the right-hand side inequality in (7.44)
and (7.47) again.
We now turn to the supersolution property inside the domain. Let ϕ be a
smooth function with bounded derivatives and (t0 , x0 , q0 ) ∈ [0, T ) × (0, ∞)d ×
(0, ∞) be a local maximizer of v − ϕ such that (v − ϕ)(t0 , x0 , q0 ) = 0.
(i) We first show that we can reduce to the case where the map q 7→ ϕ(·, q) is
strictly convex. Indeed, since v is convex, we necessarily have Dqq ϕ(t0 , x0 , q0 ) ≥
0. Given ε, η > 0, we now define ϕε,η by ϕε,η (t, x, q) := ϕ(t, x, q) + ε|q − q0 |2 +
η|q − q0 |2 (|q − q0 |2 + |t − t0 |2 + |x − x0 |2 ). Note that (t0 , x0 , q0 ) is still a local
maximizer of U − ϕε,η . Since Dqq ϕ(t0 , x0 , q0 ) ≥ 0, we have Dqq ϕε,η (t0 , x0 , q0 ) ≥
2ε > 0. Since ϕ has bounded derivatives, we can then choose η large enough
so that Dqq ϕε,η > 0. We next observe that, if ϕε,η satisfies (7.48) at (t0 , x0 , q0 )
for all ε > 0, then (7.48) holds for ϕ at this point too. This is due to the
fact that the derivatives up to order two of ϕε,η at (t0 , x0 , q0 ) converge to the
corresponding derivatives of ϕ as ε → 0.
(ii) From now on, we thus assume that the map q 7→ ϕ(·, q) is strictly convex.
Let ϕ̃ be the Fenchel transform of ϕ with respect to q, i.e.
ϕ̃(t, x, p) := sup{pq − ϕ(t, x, q)} .

q∈R
Since ϕ is strictly convex in q and smooth on its domain, ϕ̃ is strictly convex in

p and smooth on its domain. Moreover, we have
ϕ(t, x, q) = sup{pq − ϕ̃(t, x, p)} = J(t, x, q)q − ϕ̃(t, x, J(t, x, q))

p∈R
on (0, T )×(0, ∞)d ×(0, ∞) ⊂ int(dom(ϕ)), where q 7→ J(·, q) denotes the inverse
of p 7→ Dp ϕ̃(·, p), recall that ϕ̃ is strictly convex in p.
We now deduce from the assumption q0 > 0 and (7.47) that we can find
p0 ∈ [0, 1] such that v(t0 , x0 , q0 ) = p0 q0 − V̂∗ (t0 , x0 , p0 ) which, by using the very
definition of (t0 , x0 , p0 , q0 ) and v, implies that
0 = (V̂∗ − ϕ̃)(t0 , x0 , p0 ) = (local) min(V̂∗ − ϕ̃) (7.54)
and
ϕ(t0 , x0 , q0 ) = sup{pq0 − ϕ̃(t0 , x0 , p)} (7.55)

p∈R
= p0 q0 − ϕ̃(t0 , x0 , p0 ) with p0 = J(t0 , x0 , q0 ), (7.56)
where the last equality follows from (7.54) and the strict convexity of the map
p 7→ pq0 − ϕ̃(t0 , x0 , p) in the domain of ϕ̃.
We conclude the proof by discussing three alternative cases depending on
the value of p0 .
• If p0 ∈ (0, 1), then (7.54) implies that ϕ̃ satisfies (7.43) at (t0 , x0 , p0 ) and
the required result follows by exploiting the link between the derivatives of
ϕ̃ and the derivatives of its p-Fenchel transform ϕ, which can be deduced
from (7.54).
• If p0 = 1, then the first boundary condition in (7.44) and (7.54) imply
that (t0 , x0 ) is a local minimizer of V̂∗ (·, 1) − ϕ̃(·, 1) = V − ϕ̃(·, 1) such
that (V − ϕ̃(·, 1))(t0 , x0 ) = 0. This implies that ϕ̃(·, 1) satisfies (7.42) at
(t0 , x0 ) so that ϕ̃ satisfies (7.43) for α = 0 at (t0 , x0 , p0 ). We can then
conclude as in 1. above.
• If p0 = 0, then the second boundary condition in (7.44) and (7.54) imply

that (t0 , x0 ) is a local minimizer of V̂∗ (·, 0) − ϕ̃(·, 0) = 0 − ϕ̃(·, 0) such that
0 − ϕ̃(·, 0)(t0 , x0 ) = 0. In particular, (t0 , x0 ) is a local maximum point
for ϕ̃(·, 0) so that (∂t ϕ̃, Dx ϕ̃)(t0 , x0 , 0) = 0 and Dxx ϕ̃(t0 , x0 , 0) is negative
semi-definite. This implies that ϕ̃(·, 0) satisfies (7.42) at (t0 , x0 ) so that ϕ̃
satisfies (7.43) at (t0 , x0 , p0 ), for α = 0. We can then argue as in the first
case.
♦
Chapter 8
Backward SDEs and

Stochastic Control
In this chapter, we introduce the notion of backward stochastic differential equa-

tion (BSDE hereafter) which allows to relate standard stochastic control to
stochastic target problems.
More importantly, the general theory in this chapter will be developed in the
non-Markov framework. The Markovian framework of the previous chapters and
the corresponding PDEs will be obtained under a specific construction. From
this viewpoint, BSDEs can be viewed as the counterpart of PDEs in the non-
Markov framework.
However, by their very nature, BSDEs can only cover the subclass of stan-
dard stochastic control problems with uncontrolled diffusion, with corresponding
semilinear DPE. Therefore a further extension is needed in order to cover the
more general class of fully nonlinear PDEs, as those obtained as the DPE of
standard stochastic control problems. This can be achieved by means of the
notion of second order BSDEs which are very connected to second order target
problems. We refer to Soner, Zhang and Touzi [?] for this extension.
8.1 Motivation and examples

The first appearance of BSDEs was in the early work of Bismut [?] who was
extensding the Pontryagin maximum principle of optimality to the stochastic
framework. Similar to the deterministic context, this approach introduces the
so-called adjoint process defined by a stochastic differential equation combined
with a final condition. In the deterministic framework, the existence of a solution
to the adjoint equation follows from the usual theory by obvious time inversion.
The main difficulty in the stochastic framework is that the adjoint process is
required to be adapted to the given filtration, so that one can not simply solve
the existence problem by running the time clock backward.
A systematic study of BSDEs was started by Pardoux and Peng [?]. The
119
120 CHAPTER 8. BSDE AND STOCHASTIC CONTROL
motivation was also from optimal control which was an important field of inter-
est for Shige Peng. However, the natural connections with problems in finan-
cial mathematics was very quickly realized, see Elkaroui, Peng and Quenez [?].
Therefore, a large development of the theory was achieved in connection with
financial applications and crucially driven by the intuition from finance.
8.1.1 The stochastic Pontryagin maximum principle

Our objective in this section is to see how the notion of BSDE appears naturally
in the context of the Pontryagin maximum principle. Therefore, we are not
intending to develop any general theory about this important question, and we
will not make any effort in weakening the conditions for the main statement.
We will instead considerably simplify the mathematical framework in order for
the main ideas to be as transparent as possible.
Consider the stochastic control problem
V0 := sup J0 (ν) where J0 (ν) := E [g(XTν )] ,

ν∈U0
the set of control processes U0 is defined as in Section 2.1, and the controlled
state process is defined by some initial date X0 and the SDE with random
coefficients:
dXtν = b(t, Xtν , νt )dt + σ(t, Xtν , νt )dWt .
Observe that we are not emphasizing on the time origin and the position of
the state variable X at the time origin. This is a major difference between the
dynamic programming approach, developed by the American school, and the
Pontryagin maximul principle approach of the Russian school.
For every u ∈ U , we define:
Lu (t, x, y, z) := b(t, x, u) · y + Tr σ(t, x, u)T z ,

so that
∂Lu (t, x, y, z) ∂Lu (t, x, y, z)
b(t, x, u) = and σ(t, x, u) = .
∂y ∂z
We also introduce the function
`(t, x, y, z) := sup Lu (t, x, y, z),

u∈U
and we will denote by H2 the space of all F−progressively measurable processes

with finite L2 ([0, T ] × Ω, dt ⊗ dP) −norm.
Theorem 8.1. Let ν̂ ∈ U0 be such that:
(i) there is a solution (Ŷ , Ẑ) in H2 of the backward stochastic differential equa-
tion:
dŶt = −∇x Lν̂t (t, X̂t , Ŷt , Ẑt )dt + Zt dWt , and ŶT = ∇g(X̂T ), (8.1)
8.1. Motivation and examples 121
where X̂ := X ν̂ ,
(ii) ν̂ satisfies the maximum principle:
Lν̂t (t, X̂t , Ŷt , Ẑt ) = `(t, X̂t , Ŷt , Ẑt ). (8.2)
(iii) The functions g and `(t, ., y, z) are concave, for fixed t, y, z, and
∇x Lν̂t (t, X̂t , Ŷt , Ẑt ) = ∇x `(t, X̂t , Ŷt , Ẑt ) (8.3)
Then V0 = J0 (ν̂), i.e. ν̂ is an optimal control for the problem V0 .

Proof. For an arbitrary ν ∈ U0 , we compute that
h i
J0 (ν̂) − J0 (ν) = E g(X̂T ) − g(XTν )
h i
≥ E (X̂T − XTν ) · ∇g(X̂T )
h i
= E (X̂T − XTν ) · ŶT
by the concavity assumption on g. Using the dynamics of X̂ and Ŷ , this pro-

vides:
"Z #
T
ν

J0 (ν̂) − J0 (ν) ≥ E d (X̂T − XT ) · ŶT
0
hZ T
b(t, X̂t , ν̂t ) − b(t, Xtν , νt ) · Ŷt dt

= E
0
−(X̂t − Xtν ) · ∇x Lν̂t (t, X̂t , Ŷt , Ẑt )dt
h T i i
+Tr σ(t, X̂t , ν̂t ) − σ(t, Xtν , νt ) Ẑt dt
hZ T
= E Lν̂t (t, X̂t , Ŷt , Ẑt ) − Lνt (t, Xt , Ŷt , Ẑt )
0
i
−(X̂t − Xtν ) · ∇x Lν̂t (t, X̂t , Ŷt , Ẑt ) dt ,
where the diffusion terms have zero expectations because the processes Ŷ and
Ẑ are in H2 . By Conditions (ii) and (iii), this implies that
hZ T
J0 (ν̂) − J0 (ν) ≥ E `(t, X̂t , Ŷt , Ẑt ) − `(t, Xt , Ŷt , Ẑt )
0
i
−(X̂t − Xtν ) · ∇x `(t, X̂t , Ŷt , Ẑt ) dt
≥ 0
by the concavity assumption on `. ♦
Let us comment on the conditions of the previous theorem.

- Condition (ii) provides a feedback definition to ν̂. In particular, ν̂t is

a function of (t, X̂t , Ŷt , Ẑt ). As a consequence, the forward SDE defining X̂
depends on the backward component (Ŷ , Ẑ). This is a situation of forward-
backward stochastic differential equation which will not be discussed in these
notes.
- Condition (8.3) in (iii) is satisfied under natural smoothness conditions. In
the economic literature, this is known as the envelope theorem.
- Condition (i) states the existence of a solution to the BSDE (8.1), which
will be the main focus of the subsequent section.
8.1.2 BSDEs and stochastic target problems

Let us go back to a subclass of the stochastic target problems studied in Chapter
7 defined by taking the state process X independent of the control Z which is
assumed to take values in Rd . For simplicity, let X = W . Then the stochastic
target problem is defined by
V0 := inf Y0 : YTZ ≥ g(WT ), P − a.s. for some Z ∈ H2 ,

where the controlled process Y satisfies the dynamics:
dYtZ = b(t, Wt , Yt , Zt )dt + Zt · dWt . (8.4)
If existence holds for the latter problem, then there would exist a pair (Y, Z) in
H2 such that
Z T

Y0 + b(t, Wt , Yt , Zt )dt + Zt · dWt ≥ g(WT ), P − a.s.
0
If in addition equality holds in the latter inequality then (Y, Z) is a solution of

the BSDE defined by (8.4) and the terminal condition YT = g(WT ), P−a.s.
8.1.3 BSDEs and finance

In the Black-scholes model, we know that any derivative security can be perfectly
hedged. The corresponding superhedging problem reduces to a hedging problem,
and an optimal hedging portfolio exists and is determined by the martingale
representation theorem.
In fact, this goes beyond the Markov framework to which the stochastic
target problems are restricted. To see this, consider a financial market with
interest rate process {rt , t ≥ 0}, and s risky assets with price process defined by
dSt = St ? (µt dt + σt dWt ).
Then, under the self-financing condition, the liquidation value of the portfolio
is defined by
dYtπ = rt Ytπ dt + πt σt (dWt + λt dt) , (8.5)

8.2. Wellposedness of BSDEs 123
where the risk premium process λt := σt−1 (µt − rt 1) is assumed to be well-

defined, and the control process πt denotes the vector of holdings amounts in
the d risky assets at each point in time.
Now let G be a random variable indicating the random payoff of a contract.
G is called a contingent claim. The hedging problem of G consists in searching
for a portfolio strategy π̂ such that
YTπ̂ = G, P − a.s. (8.6)
We are then reduced to a problem of solving the BSDE (8.5)-(8.6). This problem
Rt
can be solved very easily if the process λ is so that the process {Wt + 0 λs ds, t ≥
0} is a Brownian motion under the so-called equivalent probability measure
Q. Under this condition, it suffices to get rid of the linear term in (8.5) by
discounting, then π̂ is obtained by the martingale representation theorem in the
present Brownian filtration under the equivalent measure Q.
We finally provide an example where the dependence of Y in the control
variable Z is nonlinear. The easiest example is to consider a financial market
with different lending and borrowing rates rt ≤ rt . Then the dynamics of
liquidation value of the portfolio (8.6) is replaced by the following SDE:
dYt = πt · σt (dWt + λt dt)(Yt − πt · 1)+ rt − (Yt − πt · 1)− rt (8.7)
As a consequence of the general results of the subsequent section, we will obtain

the existence of a hedging process π̂ such that the corresponding liquidation
value satisfies (8.7) together with the hedging requirement (8.6).
8.2 Wellposedness of BSDEs

Throughout this section, we consider a d−dimensional Brownian motion W on
a complete probability space (Ω, F, P), and we denote by F = FW the corre-
sponding augmented filtration.
Given two integers n, d ∈ N, we consider the mapping
f : [0, T ] × Ω × Rn × Rn×d −→ R,
that we assume to be P⊗B(Rn+nd )−measurable, where P denotes the σ−algebra

generated by predictable processes. In other words, for every fixed (y, z) ∈
Rn × Rn×d , the process {ft (y, z), t ∈ [0, T ]} is F−predictable.
Our interest is on the BSDE:
dYt = −ft (Yt , Zt )dt + Zt dWt and YT = ξ, P − a.s. (8.8)
where ξ is some given FT −measurable r.v. with values in Rn .

We will refer to (8.8) as BSDE(f, ξ). The map f is called the generator. We
may also re-write the BSDE (8.8) in the integrated form:
Z T Z T
Yt = ξ + fs (Ys , Zs )ds − Zs dWs , t ≤ T, , P − a.s. (8.9)
t t
8.2.1 Martingale representation for zero generator

When the generator f ≡ 0, the BSDE problem reduces to the martingale repre-
sentation theorem in the present Brownian filtration. More precisely, for every
ξ ∈ L2 (Rn , FT ), there is a unique pair process (Y, Z) in H2 (Rn ×Rn×d ) satisfying
(8.8):
Z t
Yt := E[ξ|Ft ] = E[ξ] + Zs dWs
0
Z T
= ξ− Zs dWs .
t
Here, for a subset E of Rk , k ∈ N, we denoted by H2 (E) the collection of all

F−progressively measurable L2 ([0, T ] × Ω, Leb ⊗ P)−processes with values in E.
We shall frequently simply write H2 keeping the reference to E implicit.
Let us notice that Y is a uniformly integrable martingale. Moreover, by the
Doob’s maximal inequality, we have:

2 2
≤ 4E |YT |2 = 4kZk2H2 .

kY kS 2 := E sup |Yt | (8.10)
t≤T
Hence, the process Y is in the space of continuous processes with finite S 2 −norm.
For later use, we report the following necessary and sufficient condition for
a martingale to be uniformly integrabile.
Lemma 8.2. Let M = {Mt , t ∈ [0, T )} be a scalar local martingale. Then, M
is uniformly integrable if and only if

lim λP sup |Mt | > λ = 0.
λ→∞ t≤T
Proof. Denote by Θ the collection of all F−stopping times, and
Θ(M ) := {θ ∈ Θ : M.∧θ is a martingale} .
1. We first prove that
limn→∞ E|Mθn | = supn≥1 E|Mθn | = supθ∈Θ(M ) E|Mθ | = supθ∈Θ E|Mθ |

for all (θn )n≥1 ⊂ Θ(M ) with θn −→ ∞, P − a.s.
(8.11)
To see this, let (θn ) be such a sequence, then it follows from Fatou’s lemma that
E|Mθ | ≤ lim inf E|Mθ∧θn | ≤ lim inf E|Mθn | for all θ ∈ Θ,

n→∞ n→∞
by the Jensen inequality. Then
E|Mθ | ≤ lim inf E|Mθn | ≤ lim sup E|Mθn |

n→∞ n→∞
≤ sup E|Mθn | ≤ sup E|Mθ | ≤ sup E|Mθ |.
n≥1 θ∈Θ(M ) θ∈Θ
and the required result follows from the arbitrariness of θ ∈ Θ.

2. For every λ > 0, the stopping time Tλ := inf{t : |Mt | > λ} ∈ Θ(M ), and
h i
E|MTλ | = λP sup |Mt | > λ + E |M∞ |1{supt<T |Mt |≤λ} .
t<T
Since E|M∞ | ≤ lim inf n E|Mn | = M0 < ∞, the second term on the right hand-
side convergence to E|M∞ | as λ → ∞. Since the left hand-side term is non-
decreasing in λ, we deduce that

lim E|MTλ | = p + E|M∞ | where p := lim λP sup |Mt | > λ . (8.12)
λ→∞ λ→∞ t<T
3. Observe that Tλ ∈ Θ(M ) and Tλ → ∞ a.s. when λ → ∞. Then, it follows

from (8.11) and (8.12) that
lim E|Mθn | = p + E|M∞ | for all sequence (θn )n satisfying (8.11).

λ→∞
Then p = 0 iff Mθn −→ M∞ in L1 for all sequence (θn )n satisfying (8.11), which
is now equivalent to the uniform integrability of M . ♦
8.2.2 BSDEs with affine generator

We next consider a scalr BSDE (n = 1) with generator
ft (y, z) := at + bt y + ct · z, (8.13)
where a, b, c are F−progressively measurable processes with values in R, R and

RT
Rd , respectively. We also assume that b, c are bounded and E[ 0 |at |2 dt] < ∞.
This case is easily handled by reducing to the zero generator case. How-
ever, it will play a crucial role for the understanding of BSDEs with generator
quadratic in z, which will be the focus of the next chapter.
First, by introducing the equivalent probability Q ∼ P defined by the density
Z T !
1 T
Z
dQ
= exp ct · dWt − |ct |2 dt ,
dP 0 2 0
Rt
it follows from the Girsanov theorem that the process Bt := Wt − 0 cs ds defines
a Brownian motion under Q. By formulating the BSDE under Q:
dYt = −(at + bt Yt )dt + Zt · dBt ,
we have reduced to the case where the generator does not depend on z. We
next get rid of the linear term in y by introducing:
Rt Rt Rt
bs ds bs ds bs ds
Y t := Yt e 0 so that dY t = −at e 0 dt + Zt e 0 dBt .
Finally, defining
Z t Ru
bs ds
Y t := Y t + au e 0 du,
0
we arrive at a BSDE with zero generator for Y t which can be solved by the
martingale representation theorem under the equivalent probability measure Q.
Of course, one can also express the solution under P:
" Z T #

t t
Yt = E ΓT ξ + Γs as dsFt , t ≤ T,

t
where
Z s Z s Z s
1
Γts := exp bu du − |cu |2 du + cu · dWu , 0 ≤ t ≤ s ≤ T. (8.14)
t 2 t t
8.2.3 The main existence and uniqueness result

The following result was proved by Pardoux and Peng [?].
Theorem 8.3. Assume that {ft (0, 0), t ∈ [0, T ]} ∈ H2 and, for some constant
C > 0,
|ft (y, z) − ft (y 0 , z 0 )| ≤ C(|y − y 0 | + |z − z 0 |), dt ⊗ dP − a.s.
for all t ∈ [0, T ] and (y, z), (y 0 , z 0 ) ∈ Rn × Rn×d . Then, for every ξ ∈ L2 , there
is a unique solution (Y, Z) ∈ S 2 × H2 to the BSDE(ξ, f ).
Proof. Denote S = (Y, Z), and introduce the equivalent norm in the correspond-
ing H2 space:
"Z #
T
kSkα := E eαt (|Yt |2 + |Zt |2 )dt .
0
where α will be fixed later. We consider the operator
φ : s = (y, z) ∈ H2 7−→ S s = (Y s , Z s )
defined by:
Z T Z T
Yts = ξ+ fu (yu , zu )du − Zus · dWu , t ≤ T.
t t
1. First, since |fu (yu , zu )| ≤ |fu (0, 0)| + C(|yu | + |zu |), we see that the pro-
cess {fu (yu , zu ), u ≤ T } is in H2 . Then S s is well-defined by the martingale
representation theorem and S s = φ(s) ∈ H2 .
0 0
2. For s, s0 ∈ H2 , denote δs := s−s0 , δS := S s −S s and δf := ft (S s )−ft (S s ).
Since δYT = 0, it follows from Itô’s formula that:
Z T Z T
αt 2 αu 2
eαu 2δYu · δfu − α|δYu |2 du

e |δYt | + e |δZu | du =
t t
Z T
−2 eαu (δZu )T δYu · dWu .
t
In the remaining part of this step, we prove that

Z .
M. := eαu (δZu )T δYu · dWu is a uniformly integrable martingale.(8.15)
0
so that we deduce from the previous equality that

" Z T # "Z #
T
αt 2 αu 2 αu 2

E e |δYt | + e |δZu | du = E e 2δYu · δfu − α|δYu | du .
t t
(8.16)
To prove (8.15), we set V := supt≤T |Mt |, and we verify the condition of Lemma
8.2:

λP [V > λ] = λE 1{1<λ−1 V } ≤ E V 1{V >λ}
which converges to zero, provided that V ∈ L1 . To check that the latter condi-
tion hold true, we estimate by the Burkholder-Davis-Gundy inequality that:
 !1/2 
Z T
E[V ] ≤ CE  e2αu |δYu |2 |δZu |2 du 
0
 !1/2 
Z T
≤ C 0 E  sup |δYu | |δZu |2 du 
u≤T 0
"Z #!
T
C0

2 2
≤ E sup |δYu | +E |δZu | du < ∞.
2 u≤T 0
3. We now continue estimating (8.16) by using the Lipschitz property of the

generator:
h Z T i
E eαt |δYt |2 + eαu |δZu |2 du
t Z
h T i
≤ E eαu −α|δYu |2 + C2|δYu |(|δyu | + |δzu |) du
"Zt #
T
eαu −α|δYu |2 + C ε2 |δYu |2 + ε−2 (|δyu | + |δzu |)2 du

≤ E
t
for any ε > 0. Choosing Cε2 = α, we obtain:

" Z T # "Z #
T 2
C
E eαt |δYt |2 + eαu |δZu |2 du ≤ E eαu (|δyu | + |δzu |)2 du
t t α
2
C
≤ 2 kδsk2α .
α
This provides
C2 C 2T
kδZk2α ≤ 2 kδsk2α and kδY k2α dt ≤ 2 kδsk2α
α α
where we abused notatation by writing kδY kα and kδZkα although these pro-
cesses do not have the dimension required by the definition. Finally, these two
estimates imply that
r
2C 2
kδSkα ≤ (1 + T )kδskα .
α
By choosing α > 2(1 + T )C 2 , it follows that the map φ is a contraction on H2 ,
and that there is a unique fixed point.
4. It remain to prove that Y ∈ S 2 . This is easily obtained by first estimating:
"Z # !
T Z t 2
2 2 2
E sup |Yt | ≤ C |Y0 | + E |ft (Yt , Zt )| dt + E sup Zs · dWs ,

t≤T 0 t≤T 0
and then using the Lipschitz property of the generator and the Burkholder-
Davis-Gundy inequality. ♦
Remark 8.4. Consider the Picard iterations:

(Y 0 , Z 0 ) = (0, 0), and
Z T Z T
Ytk+1 =ξ+ fu (Yuk , Zuk )du + Zuk+1 · dWu .
t t
Then, S k = (Y k , Z k ) −→ (Y, Z) in H2 . Moreover, since

2 k
2C
kS k kα ≤ (1 + T ) ,
α
it follows that k kS k kα < ∞, and we conclude by the Borel-Cantelli lemma
P
that the convergence (Y k , Z k ) −→ (Y, Z) also holds dt ⊗ dP−a.s.
8.3 Comparison and stability

Theorem 8.5. Let n = 1, and let (Y i , Z i ) be the solution of BSDE(f i , ξ i ) for
some pair (ξ i , f i ) satisfying the conditions of Theorem 8.3, i = 0, 1. Assume
that
ξ1 ≥ ξ0 and ft1 (Yt0 , Zt0 ) ≥ ft0 (Yt0 , Zt0 ), dt ⊗ dP − a.s. (8.17)
8.3. Comparison and stability 129
Then Yt1 ≥ Yt0 , t ∈ [0, T ], P−a.s.

Proof. We denote
δY := Y 1 − Y 0 , δZ := Z 1 − Z 0 , δ0 f := f 1 (Y 0 , Z 0 ) − f 0 (Y 0 , Z 0 ),
and we compute that
d(δYt ) = − (αt δYt + βt · δZt + δ0 ft ) dt + δZt · dWt , (8.18)
where
ft1 (Yt1 , Zt1 ) − ft1 (Yt0 , Zt1 )
αt := 1{δYt 6=0} ,
δYt
and, for j = 1, . . . , d,

ft1 Yt0 , Zt1 ⊕j−1 Zt0 − ft1 Yt0 , Zt1 ⊕j Zt0
βtj := 1{δZ 0,j 6=0} ,
δZt0,j t
where δZ 0,j denotes the j−th component of δZ, and for every z 0 , z 1 ∈ Rd ,
z 1 ⊕j z 0 := z 1,1 , . . . , z 1,j , z 0,j+1 , . . . , z 0,d for 0 < j < n, z 1 ⊕0 z 0 := z 0 ,
z 1 ⊕n z 0 := z 1 .
Since f 1 is Lipschitz-continuous, the processes α and β are bounded. Solving
the linear BSDE (8.18) as in subsection 8.2.2, we get:
" Z T #

t t
δYt = E ΓT δYT + Γu δ0 fu duFt , t ≤ T,

t
where the process Γt is defined as in (8.14) with (δ0 f, α, β) substituted to (a, b, c).
Then Condition (8.17) implies that δY ≥ 0, P−a.s. ♦
Our next result compares the difference in absolute value between the solu-
tions of the two BSDEs, and provides a bound which depends on the difference
between the corresponding final datum and the genrators. In particular, this
bound provide a transparent information about the nature of conditions needed
to pass to limits with BSDEs.
Theorem 8.6. Let (Y i , Z i ) be the solution of BSDE(f i , ξ i ) for some pair
(ξ i , f i ) satisfying the conditions of Theorem 8.3, i = 0, 1. Then:
kY 1 − Y 0 k2S 2 + kZ 1 − Z 0 k2H2 ≤ C kξ 1 − ξ 0 k2L2 + k(f 1 − f 0 )(Y 0 , Z 0 )k2H2 ,

where C is a constant depending only on T and the Lipschitz constant of f 1 .

Proof. We denote δξ := ξ 1 − ξ 0 , δY := Y 1 − Y 0 , δf := f 1 (Y 1 , Z 1 ) − f 0 (Y 0 , Z 0 ),
and ∆f := f 1 − f 0 . Given a constant β to be fixed later, we compute by Itô’s
formula that:
Z T
eβt |δYt |2 = eβT |δξ|2 + eβu 2δYu · δfu − |δZu |2 − β|δYu |2 du

t
Z T
+2 eβu δZuT δYu · dWu .
t
By the same argument as in the proof of Theorem 8.3, we see that the stochastic
integral term has zero expectation. Then
" Z T #
βt 2 βT 2 βu 2 2

e |δYt | = Et e |δξ| + e 2δYu · δfu − |δZu | − β|δYu | du , (8.19)
t
where Et := E[.|Ft ]. We now estimate that, for any ε > 0:
2δYu · δfu ≤ ε−2 |δYu |2 + ε2 |δfu |2

2
≤ ε−2 |δYu |2 + ε2 C(|δYu | + |δZu |) + |∆fu (Yu0 , Zu0 )|
≤ ε−2 |δYu |2 + 3ε2 C 2 (|δYu |2 + |δZu |2 ) + |∆fu (Yn0 , Zu0 )|2 .

We then choose ε2 := C 2 /6 and β := ε−2 + 1/2, and plug the latter estimate in
(8.19). This provides:
"Z # " #
T
C 2 T βu
Z
βt 2 2 βT 2 1 0 2
e |δYt | + Et |δZu | du ≤ Et e |δξ| + e |δfu (Y , Zu )| du ,
t 2 0
which implies the required inequality by taking the supremum over t ∈ [0, T ] and
using the Doob’s maximal inequality for the martingale {Et [eβT |δξ|2 ], t ≤ T }.
♦
8.4 BSDEs and stochastic control

We now turn to the question of controlling the solution of a family of BSDEs
in the scalr case n = 1. Let (ξν , fν )ν∈U be a family of coefficients, where U is
some given set of controls. We assume that the coefficients (ξν , fν )ν∈U satisfy
the conditions of the existence and uniqueness theorem 8.3, and we consider the
following stochastic control problem:
V0 := sup Y0ν , (8.20)

ν∈U
where (Y ν , Z ν ) is the solution of BSDE(ξ ν , f ν ).

The above stochastic control problem boils down to the standard control
problems of Section 2.1 when the generators f α are all zero. When the gen-
erators f ν are affine in (y, z), the problem (8.20) can also be recasted in the
standard framework, by discounting and change of measure.
The following easy result shows that the above maximization problem can
be solved by maximizing the coefficients (ξ α , f α ):
ft (y, z) := ess sup ftν (y, z), ξ := ess sup ξ ν . (8.21)

ν∈U ν∈U
The notion of essential supremum is recalled in the Appendix of this chapter.

We will asume that the coefficients (f, ξ) satisfy the conditions of the existence
result of Theorem 8.3, and we will denote by (Y, Z) the corresponding solution.
8.4. BSDEs and stochastic control 131
A careful examination of the statement below shows a great similarity with

the verification result in stochastic control. In the present non-Markov frame-
work, this remarkable observation shows that the notion of BSDEs allows to
mimic the stochastic control methods developed previous chapters in the Markov
case.
Proposition 8.7. Assume that the coefficients (ξ, f ) and (ξν , fν ) satisfy the
conditions of Theorem 8.3, for all ν ∈ U. Assume further that there exists some
ν̂ ∈ U such that
ft (y, z) = f ν̂ (y, z) and ξ = ξ ν̂ .
Then V0 = Y0ν̂ and Yt = ess supν∈U Ytν , t ∈ [0, T ], P−a.s.
Proof. The P−a.s. inequality Y ≤ Y ν , for all ν ∈ U, is a direct consequence
of the comparison result of Theorem 8.5. Hence Yt ≤ supν∈U Ytν , P−a.s. To
conclude, we notice that Y and Y ν̂ are two solutions of the same BSDE, and
therefore must coincide, by uniqueness. ♦
The next result characterizes the solution of a standard stochastic control

problem in terms of a BSDE. Here, again, we emphasize that, in the present non-
Markov framework, the BSDE is playing the role of the dynamic programming
equation whose scope is restricted to the Markov case.
Let
" Z T #
Pν ν ν ν
U0 := inf E β0,T ξ + βu,T `u (νu )du ,
ν∈U 0
where
dPν
RT RT RT
1 2
:= e 0 λt (νt )·dWt − 2 0 |λt (νt )| dt ν
and βt,T := e− t
ku (νu )du
.
dP FT

We assume that all coefficients involved in the above expression satisfy the
required conditions for the problem to be well-defined.
We first notice that for every ν ∈ U, defining
" Z T #

ν Pν ν ν ν
Yt := E βt,T ξ + βu,T `u (νu )duFt

t
is the first component of the solution (Y , Z ν ) of the affine BSDE:

ν
dYtν = −ftν (Ytν , Ztν )dt + Ztν dWt , YTν = ξ ν

with ftν (y, z) := `t (νt ) − kt (νt )y + λt (νt )z. In view of this observation, the
following result is a direct application of Proposition 8.7.
Proposition 8.8. Assume that the coefficients
ξ := ess sup ξ ν and ft (y, z) := ess sup ftν (y, z)
ν∈U ν∈U
satisfy the conditions of Theorem 8.3, and let (Y, Z) be the corresponding solu-
tion. Then U0 = Y0 .
8.5 BSDEs and semilinear PDEs

In this section, we specialize the discussion to the so-called Markov BSDEs in
the one-dimensional case n = 1. This class of BSDEs corresponds to the case
where
ft (y, z) = F (t, Xt , y, z) and ξ = g(XT ),
where F : [0, T ] × Rd × R × Rd −→ R and g : Rd −→ R are measurable, and X
is a Markov diffusion process defined by some initial data X0 and the SDE:
dXt = µ(t, Xt )dt + σ(t, Xt )dWt . (8.22)
Here µ and σ are continuous and satisfy the usual Lipschitz and linear growth
condtions on order to ensure existence and uniqueness of a strong solution to
the SDE SDE-MarkovBSDE, and
f, g have polynomial growth in x
and f is uniformly Lipschitz in (y, z).
Then, it follows from Theorem 8.3 that the above Markov BSDE has a unique
solution.
We next move the time origin by considering the solution {Xst,x , s ≥ t} of
(8.22) with initial data Xtt,x = x. The corresponding solution of the BSDE
dYs = −F (s, Xst,x , Ys , Zs )ds + Zs dWs , YT = g XTt,x

(8.23)
will be denote by (Y t,x , Z t,x ).
Proposition 8.9. The process {(Yst,x , Zst,x ) , s ∈ [t, T ]} is adapted to the filtra-
tion
Fst := σ (Wu − Wt , u ∈ [t, s]) , s ∈ [t, T ].
In particular, u(t, x) := Ytt,x is adeterministic function and
s,X t,x
Yst,x = Ys s = u s, Xst,x , for all s ∈ [t, T ], P − a.s.

Proof. The first claim is obvious, and the second one follows from the fact that
s,X t,x
Xrt,x = Xr s . ♦
Proposition 8.10. Let u be the function defined in Proposition 8.9, and assume
that u ∈ C 1,2 ([0, T ), Rd ). Alors
−∂t u − Au − f (., u, σ T Du) = 0 on [0, T ) × Rd .
Proof. This an easy application of Itô’s lemma together with the usual localiza-
tion technique. ♦
CONCLUDE WITH NONLINEAR FEYNMAC-KAC

8.5. BSDEs and semilinear PDEs 133
8.6 Appendix: essential supremum

The notion of essential supremum has been introduced in probability in order to
face maximization problem over an infinite family Z. The problem arises when
Z is not countable because then the supremum is not measurable, in general.
Theorem 8.11. Let Z be a family of r.v. Z : Ω −→ R ∪ {∞} on a probability
space (Ω, F, P). Then there exists a unique (a.s.) r.v. Z̄ : Ω −→ R ∪ {∞} such
that:
(a) Z̄ ≥ Z, a.s. for all Z ∈ Z,
(b) For all r.v. Z 0 satisfying (a), we have Z̄ ≤ Z 0 , a.s.
Moreover, there exists a sequence (Zn )n∈N ⊂ Z such that Z̄ = supn∈N Zn .
The r.v. Z̄ is called the essential supremum of the family Z, and denoted by
ess sup Z.
Proof. The uniqueness of Z̄ is an immediate consequence of (b). To prove
existence, we consider the set D of all countable subsets of Z. For all D ∈ D,
we define ZD := sup{Z : Z ∈ D}, and we introduce the r.v. ζ := sup{E[ZD ] :
D ∈ D}.
1. We first prove that there exists D∗ ∈ D such that ζ = E[ZD∗ ]. To see this, let
(Dn )n ⊂ D be a maximizing sequence, i.e. E[ZDn ] −→ ζ, then D∗ := ∪n Dn ∈ D
satisfies E[ZD∗ ] = ζ. We denote Z̄ := ZD∗ .
2. It is clear that the r.v. Z̄ satisfies (b). To prove that property (a) holods
true, we consider an arbitrary Z ∈ Z together with the countable family D :=
D∗ ∪ {Z} ⊂ D. Then ZD = Z ∨ Z̄, and ζ = E[Z̄] ≤ E[Z ∨ Z̄] ≤ ζ. Consequently,
Z ∨ Z̄ = Z̄, and Z ≤ Z̄, a.s. ♦
Bibliography
[1] G. Barles. Solutions de Viscosité des Équations de Hamilton-Jacobi.

Mathématiques & Applications. Springer-Verlag, 1994.
[2] P. Cheridito, M. Soner, and N. Touzi. Small time path behavior of double stochas-
tic integrals, and application to stochastic control. Annals of Applied Probability,
15:2472–2495, 2005.
[3] M. G. Crandall, H. Ishii, and P.-L. Lions. User’s guide to viscosity solutions of
second order partial differential equations. Bull. Amer. Math. Soc., 27:1–67, 1992.
[4] J. Dugundji. Topology. Allyn and Bacon series in Advanced Mathematics, Allyn
and Bacon edt.
[5] N. ElKaroui and M. Jeanblanc-Picqué. Contrôle de processus de markov.
Séminaire de Probabilités Springer Lecture Notes in Math. 1321,, XXII:508–541,
1988.
[6] W. H. Fleming and H. M. Soner. Controlled Markov Processes and Viscosity
Solutions. Springer-Verlag, 1993.
[7] H. Foellmer and P. Leukert. Quantile hedging. Finance and Stochastics, 3:251–
273.
[8] I. Karatzas and S. Shreve. Brownian Motion and Stochastic Calculus. Springer-
Verlag, New-York Second Edition, 1991.
[9] I. Karatzas and S. Shreve. Methods of Mathematical Finance. Springer-Verlag,
New-York Second Edition, 1999.
[10] R. C. Merton. Lifetime portfolio selection under uncertainty: the continuous-time
model. Review of Economic Statistics, 51:247–257, 1969.
[11] R. C. Merton. Optimum consumption and portfolio rules in a continuous-time
model. Journal of Economic Theory, 3:373–413, 1971.
[12] P. J. Reny. On the existence of pure and mixed strategy nash equilibria in dis-
continuous games. Econometrica, 67:1029–1056, 1999.
[13] R. T. Rockafellar. Convex Analysis. Princeton University Press, 1970.
135

Master LN

Uploaded by

Copyright:

Available Formats

Master LN

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Master LN

Uploaded by

Copyright:

Available Formats

STOCHASTIC CONTROL,

AND APPLICATION TO FINANCE

Ecole Polytechnique Paris

This version: 10 November 2018

1 Conditional Expectation and Linear Parabolic PDEs 5

2 Stochastic Control and Dynamic Programming 21

3 Optimal Stopping and Dynamic Programming 37

3.4.2 Infinite horizon optimal stopping . . . . . . . . . . . . . . 43

4 Solving Control Problems by Verification 49

5 Introduction to Viscosity Solutions 63

6 Dynamic Programming Equation in Viscosity Sense 89

7 Stochastic Target Problems 99

8 Backward SDEs and Stochastic Control 117

Throughout this chapter, (Ω, F, F, P ) is a filtered probability space with filtra-

1.1 Stochastic differential equations with ran-

dXt = bt (Xt )dt + σt (Xt )dWt , t ∈ [0, T ], (1.1)

Definition 1.1. A strong solution of (1.1) is an F−progressively measurable

for some constant C = C(T, K) depending on T and K.

By the Lipschitz-continuity of b and σ in x, uniformly in t, we have |bt (x)|2 ≤

since X ∈ H2 , and b(., 0) ∈ L2 ([0, T ]).

ing bs := bs (Xs ) and σs := σs (Xs ). We directly estimate:

and we conclude by using the Gronwall lemma. ♦

The following exercise shows that the Lipschitz-continuity condition on the

∞. Then for all t0 ∈ [t, T ]:

δσ := σ(X) − σ(X 0 ). We first decompose:

Then the required estimate follows from the Gronwall inequality.

and the required estimate follows from the Gronwall inequality. ♦

1.2 Markov solutions of SDEs

The two following properties are obvious:

1.3 Connection with linear partial differential

(Exercise !). The linear differential operator A is called the generator of X. It

C 1,2 ([0, T ) × Rn ). Then v solves the partial differential equation:

1.3.2 Cauchy problem and the Feynman-Kac representa-

by the PDE satisfied by v in (1.8). Then:

by the terminal condition satisfied by v in (1.8). Moreover, since k is bounded

The above Feynman-Kac representation formula has an important numerical

and is remarkably independent of the dimension d of the variable X !

1.3.3 Representation of the Dirichlet problem

Au − ku + f = 0 on D and u = g on ∂D, (1.12)

where ∂D denotes the boundary of D, f and k are continuous functions from

Similarly to the the representation result of the Cauchy problem obtained in

Theorem 1.8. Let u be a C 2 −solution of the Dirichlet problem (1.12). Assume

Then, we have the representation:

Exercise 1.9. Provide a proof of Theorem 1.8 by imitating the arguments in

1.4 The stochastic control approach to the Black-

dSt = St ? (µt dt + σt dWt ) , t ∈ [0, T ],

where, for two vectors x, y ∈ Rd , we denote x ? y the vector of Rd with compo-

We assume that the MR (d, d)−matrix σt is invertible for every t ∈ [0, T ]

1.4.2 Portfolio and wealth process

Let X̃ π be the discounted wealth process

Then, by an immediate application of Itô’s formula, we see that

dX̃t = π̃t · σt (λt dt + dWt ) , 0 ≤ t ≤ T, (1.13)

where π̃t := e−rt πt . We still need to place further technical conditions on π,

it follows from the Girsanov theorem that the process

is a Brownian motion under the equivalent probability measure

In terms of the Q Brownian motion B, the discounted price process satisfies

dS̃t = S̃t ? σt dBt , t ∈ [0, T ],

Definition 1.10. An admissible portfolio process π = {θt , t ∈ [0, T ]} is an