Chapter 4

Mathematical Preliminaries

We want to study functional equations of the form

v(x) = max fF (x; y) + v(y)g

y2 (x)

where r is the period return function (such as the utility function) and is the
constraint set. Note that for the neoclassical growth model x = k; y = k 0 and
F (k; k 0 ) = U (f (k) k 0 ) and (k) = fk 0 2 R :0 k f (k)g
In order to so we de…ne the following operator T

(T v) (x) = max fF (x; y) + v(y)g

y2 (x)

This operator T takes the function v as input and spits out a new function T v:
In this sense T is like a regular function, but it takes as inputs not scalars z 2 R
or vectors z 2 Rn ; but functions v from some subset of possible functions. A
solution to the functional equation is then a …xed point of this operator, i.e. a
function v such that
v = Tv
We want to …nd out under what conditions the operator T has a …xed point
(existence), under what conditions it is unique and under what conditions we
can start from an arbitrary function v and converge, by applying the operator T
1
repeatedly, to v : More precisely, by de…ning the sequence of functions fvn gn=0
recursively by v0 = v and vn+1 = T vn we want to ask under what conditions
limn!1 vn = v :
In order to make these questions (and the answers to them) precise we have
to de…ne the domain and range of the operator T and we have to de…ne what
we mean by lim : This requires the discussion of complete metric spaces. In the
next subsection I will …rst de…ne what a metric space is and then what makes
a metric space complete.
Then I will state and prove the contraction mapping theorem. This theorem
states that an operator T; de…ned on a metric space, has a unique …xed point if

71
72 CHAPTER 4. MATHEMATICAL PRELIMINARIES

this operator T is a contraction (I will obviously …rst de…ne what a contraction

is). Furthermore it assures that from any starting guess v repeated applications
of the operator T will lead to its unique …xed point.
Finally I will prove a theorem, Blackwell’s theorem, that provides su¢ cient
condition for an operator to be a contraction. We will use this theorem to prove
that for the neoclassical growth model the operator T is a contraction and hence
the functional equation of our interest has a unique solution.

4.1 Complete Metric Spaces

De…nition 15 A metric space is a set S and a function d : S S ! R such
that for all x; y; z 2 S

1. d(x; y) 0
2. d(x; y) = 0 if and only if x = y
3. d(x; y) = d(y; x)
4. d(x; z) d(x; y) + d(y; z)

The function d is called a metric and is used to measure the distance between
two elements in S: The second property is usually referred to as symmetry,
the third as triangle inequality (because of its geometric interpretation in R
Examples of metric spaces (S; d) include1

Example 16 S = R with metric d(x; y) = jx yj

1 if x 6= y
Example 17 S = R with metric d(x; y) =
0 otherwise
1
Example 18 S = l1 = fx = fxgt=0 jxt 2 R; all t 0 and supt jxt j < 1g with
metric d(x; y) = supt jxt yt j

Example 19 Let X Rl and S = C(X) be the set of all continuous and

bounded functions f : X ! R: De…ne the metric d : C(X) C(X) ! R as
d(f; g) = supx2X jf (x) g(x)j: Then (S; d) is a metric space
1 A function f : X ! R is said to be bounded if there exists a constant K > 0 such that

jf (x)j < K for all x 2 X:

Let S be any subset of R: A number u 2 R is said to be an upper bound for the set S if
s u for all s 2 S: The supremum of S; sup(S) is the smallest upper bound of S:
Every set in R that has an upper bound has a supremum (imposed by the completeness
axiom). For sets that are unbounded above some people say that the supremum does not
exist, others write sup(S) = 1: We will follow the second convention.
Also note that sup(S) = max(S); whenever the latter exists. What the sup buys us is that
it always exists even when the max does not. A simle example
1
S= :n2N
n
For this example sup(S) = 0 whereas max(S) does not exist.
4.2. CONVERGENCE OF SEQUENCES 73

A few remarks: the space l1 (with corresponding norm) will be important

when we discuss the welfare theorems as naturally consumption allocations for
models with in…nitely lived consumers are in…nite sequences. Why we want to
require these sequences to be bounded will become clearer later.
The space C(X) with norm d as de…ned above will be used immediately as
we will de…ne the domain of our operator T to be C(X); i.e. T uses as inputs
continuous and bounded functions.
Let us prove that some of the examples are indeed metric spaces. For the
…rst example the result is trivial.

1 if x 6= y
Claim 20 S = R with metric d(x; y) = is a metric space
0 otherwise

Proof. We have to show that the function d satis…es all three properties in
the de…nition. The …rst three properties are obvious. For the forth property: if
x = z; the result follows immediately. So suppose x 6= z: Then d(x; z) = 1: But
then either y 6= x or y 6= z (or both), so that d(x; y) + d(y; z) 1

Claim 21 l1 together with the sup-metric is a metric space

Proof. Take arbitrary x; y; z 2 l1 : From the basic triangle inequality on R

we have that jxt yt j jxt j+jyt j: Hence, since supt jxt j < 1 and supt jyt j < 1;
we have that supt jxt yt j < 1: Property 1 is obvious. If x = y (i.e. xt = yt for
all t), then jxt yt j = 0 for all t; hence supt jxt yt j = 0: Suppose x 6= y: Then
there exists T such that xT 6= yT ; hence jxT yT j > 0; hence supt jxt yt j > 0
Property 3 is obvious since jxt yt j = jyt xt j; all t: Finally for property 4.
we note that for all t

jxt zt j jxt yt j + jyt zt j

Since this is true for all t; we can apply the sup to both sides to obtain the
result (note that the sup on both sides is …nite).

Claim 22 C(X) together with the sup-norm is a metric space

Proof. Take arbitrary f; g 2 C(X): f = g means that f (x) = g(x) for all
x 2 X: Since f; g are bounded, supx2X jf (x)j < 1 and supx2X jf (x)j < 1; so
supx2X jf (x) g(x)j < 1: Property 1. through 3. are obvious and for property
4. we use the same argument as before, including the fact that f; g 2 C(X)
implies that supx2X jf (x) g(x)j < 1:

4.2 Convergence of Sequences

The next de…nition will make precise the meaning of statements of the form
limn!1 vn = v : For an arbitrary metric space (S; d) we have the following
de…nition.
74 CHAPTER 4. MATHEMATICAL PRELIMINARIES

De…nition 23 A sequence fxn g1 n=0 with xn 2 S for all n is said to converge

to x 2 S; if for every " > 0 there exists a N" 2 N such that d(xn ; x) < " for all
n N" : In this case we write limn!1 xn = x:

This de…nition basically says that a sequence fxn g1

n=0 converges to a point
if we, for every distance " > 0 we can …nd an index N" so that the sequence of
xn is not more than " away from x after the N" element of the sequence. Also
note that, in order to verify that a sequence converges, it is usually necessary
to know the x to which it converges in order to apply the de…nition directly.
1
Example 24 Take S = R with d(x; y) = jx yj: De…ne fxn g1 n=0 by xn = n :
Then limn!1 xn = 0: This is straightforward to prove, using the de…nition.
Take any " > 0: Then d(xn ; 0) = n1 : By taking N" = 2" we have that for n N" ;
d(xn ; 0) = n1 1 " 2
N" = 2 < " (if N" = " is not an integer, take the next biggest
integer).

For easy examples of sequences it is no problem to guess the limit. Note that
the limit of a sequence, if it exists, is always unique (you should prove this for
yourself). For not so easy examples this may not work. There is an alternative
criterion of convergence, due to Cauchy.2

De…nition 25 A sequence fxn g1 n=0 with xn 2 S for all n is said to be a Cauchy

sequence if for each " > 0 there exists a N" 2 N such that d(xn ; xm ) < " for all
n; m N"

Hence a sequence fxn g1

n=0 is a Cauchy sequence if for every distance " > 0
we can …nd an index N" so that the elements of the sequence do not di¤er by
more than by ":
1
Example 26 Take S = R with d(x; y) = jx yj: De…ne fxn g1 n=0 by xn = n :
This sequence is a Cauchy sequence. Again this is straightforward to prove. Fix
" > 0 and take any n; m 2 N: Without loss of generality assume that m > n:
Then d(xn ; xm ) = n1 1 1 2
m < n : Pick N" = " and we have that for n; m N" ;
1 1 "
d(xn ; 0) < n N" = 2 < ": Hence the sequence is a Cauchy sequence.

So it turns out that the sequence in the last example both converges and is a
Cauchy sequence. This is not an accident. In fact, one can prove the following

Theorem 27 Suppose that (S; d) is a metric space and that the sequence fxn g1
n=0
converges to x 2 S: Then the sequence fxn g1
n=0 is a Cauchy sequence.

"
Proof. Since fxn g1 n=0 converges to x; there exists M 2 such that d(xn ; x) < 2
"

for all n M 2 : Therefore if n; m

" N" we have that d(xn ; xm ) d(xn ; x) +
d(xm ; x) < 2" + 2" = " (by the de…nition of convergence and the triangle inequal-
ity). But then for any " > 0; pick N" = M 2" and it follows that for all n; m N"
we have d(xn ; xm ) < "
2 Augustin-Louis Cauchy (1789-1857) was the founder of modern analysis. He wrote about

800 (!) mathematical papers during his scienti…c life.

4.2. CONVERGENCE OF SEQUENCES 75

1 if x 6= y
Example 28 Take S = R with d(x; y) = : De…ne fxn g1n=0 by
0 otherwise
xn = n1 . Obviously d(xn ; xm ) = 1 for all n; m 2 N: Therefore the sequence is
not a Cauchy sequence. It then follows from the preceding theorem (by taking
the contrapositive) that the sequence cannot converge. This example shows that,
whenever discussing a metric space, it is absolutely crucial to specify the metric.
This theorem tells us that every convergent sequence is a Cauchy sequence.
The reverse does not always hold, but it is such an important property that
when it holds, it is given a particular name.
De…nition 29 A metric space (S; d) is complete if every Cauchy sequence fxn g1
n=0
with xn 2 S for all n converges to some x 2 S:
Note that the de…nition requires that the limit x has to lie within S: We are
interested in complete metric spaces since the Contraction Mapping Theorem
deals with operators T : S ! S; where (S; d) is required to be a complete metric
space. Also note that there are important examples of complete metric spaces,
but other examples where a metric space is not complete (and for which the
Contraction Mapping Theorem does not apply).
Example 30 Let S be the set of all continuous, strictly decreasing functions
on [1; 2] and let the metric on S be de…ned as d(f; g) = supx2[1;2] jf (x) g(x)j:
I claim that (S; d) is not a complete metric space. This can be proved by an
example of a sequence of functions ffn g1
n=0 that is a Cauchy sequence, but does
1
not converge within S: De…ne fn : [0; 1] ! R by fn (x) = nx : Obviously all fn
are continuous and strictly decreasing on [1; 2]; hence fn 2 S for all n: Let us
…rst prove that this sequence is a Cauchy sequence. Fix " > 0 and take N" = 2" :
Suppose that m; n N" and without loss of generality assume that m > n: Then
1 1
d(fn ; fm ) = sup
x2[1;2] nx mx
1 1
= sup
x2[1;2] nx mx
m n
= sup
x2[1;2] mnx
n
m n 1 m
= =
mn n
1 1 "
= <"
n N" 2
Hence the sequence is a Cauchy sequence. But since for all x 2 [1; 2]; limn!1 fn (x) =
0; the sequence converges to the function f; de…ned as f (x) = 0; for all x 2 [1; 2]:
But obviously, since f is not strictly decreasing, f 2= S: Hence (S; d) is not a
complete metric space. Note that if we choose S to be the set of all continu-
ous and decreasing (or increasing) functions on R; then S; together with the
sup-norm, is a complete metric space.
76 CHAPTER 4. MATHEMATICAL PRELIMINARIES
qP
L
Example 31 Let S = RL and d(x; y) = L l=1 jxl yl jL : (S; d) is a complete
metric space. This is easily proved by proving the following three lemmata (which
is left to the reader).

1. Every Cauchy sequence fxn g1 L

n=0 in R is bounded

2. Every bounded sequence fxn g1 L 1

n=0 in R has a subsequence fxni gi=0 con-
L
verging to some x 2 R (Bolzano-Weierstrass Theorem)
3. For every Cauchy sequence fxn g1 L 1
n=0 in R ; if a subsequence fxni gi=0
L 1
converges to x 2 R ; then the entire sequence fxn gn=0 converges to x 2
RL :

Example 32 This last example is very important for the applications we are
interested in. Let X RL and C(X) be the set of all bounded continuous
functions f : X ! R with d being the sup-norm. Then (C(X); d) is a complete
metric space.

Proof. (This follows SLP, pp. 48) We already proved that (C(X); d) is a
metric space. Now we want to prove that this space is complete. Let ffn g1 n=0
be an arbitrary sequence of functions in C(X) which is Cauchy. We need to
establish the existence of a function f 2 C(X) such that for all " > 0 there
exists N" satisfying supx2X jfn (x) f (x)j < " for all n N" :
We will proceed in three steps: a) …nd a candidate for f; b) establish that the
sequence ffn g1n=0 converges to f in the sup-norm and c) show that f 2 C(X):

1. Since ffn g1n=0 is Cauchy, for each " > 0 there exists M" such that supx2X jfn (x)
fm (x)j < " for all n; m M" : Now …x a particular x 2 X: Then ffn (x)g1 n=0
is just a sequence of numbers. Now

jfn (x) fm (x)j sup jfn (y) fm (y)j < "

y2X

Hence the sequence of numbers ffn (x)g1 n=0 is a Cauchy sequence in R:

Since R is a complete metric space, ffn (x)g1 n=0 converges to some number,
call it f (x): By repeating this argument for all x 2 X we derive our
candidate function f ; it is the pointwise limit of the sequence of functions
ffn g1
n=0 :

2. Now we want to show that ffn g1 n=0 converges to f as constructed above.

Hence we want to argue that d(fn ; f ) goes to zero as n goes to in…nity.
Fix " > 0: Since ffn g1 n=0 is Cauchy, it follows that there exists N" such
that d(fn ; fm ) < " for all n; m N" : Now …x x 2 X: For any m n N"
we have (remember that the norm is the sup-norm)

jfn (x) f (x)j jfn (x) fm (x)j + jfm (x) f (x)j

d(fn ; fm ) + jfm (x) f (x)j
"
+ jfm (x) f (x)j
2
4.3. THE CONTRACTION MAPPING THEOREM 77

But since ffn g1

n=0 converges to f pointwise, we have that jfm (x) f (x)j <
"
2 for all m N " (x), where N" (x) is a number that may (and in general
does) depend on x. But then, since x 2 X was arbitrary, jfn (x) f (x)j < "
for all n N" (the key is that this N" does not depend on x). Therefore
supx2X jfn (x) f (x)j = d(fn ; f ) " and hence the sequence ffn g1n=0
converges to f:

3. Finally we want to show that f 2 C(X); i.e. that f is bounded and

continuous. Since ffn g1 n=0 lies in C(X); all fn are bounded, i.e. there is
a sequence of numbers fKn g1 n=0 such that supx2X jfn (x)j Kn : But by
the triangle inequality, for arbitrary n

sup jf (x)j = sup jf (x) fn (x) + fn (x)j

x2X x2X
sup jf (x) fn (x)j + sup jfn (x)j
x2X x2X
sup jf (x) fn (x)j + Kn
x2X

But since ffn g1n=0 converges to f; there exists N" such that supx2X jf (x)
fn (x)j < " for all n N" : Fix an " and take K = KN" + 2": It is obvious
that supx2X jf (x)j K: Hence f is bounded. Finally we qprove continuity
L L
PL
of f: Let us choose the metric on R to be jjx yjj = l=1 jxl yl jL .
We need to show that for every " > 0 and every x 2 X there exists a
("; x) > 0 such that if jjx yjj < ("; x) then jf (x) f (y)j < ", for
all x; y 2 X: Fix " and x: Pick a k large enough so that d(fk ; f ) < 3"
(which is possible as ffn g1n=0 converges to f ): Choose ("; x) > 0 such
that jjx yjj < ("; x) implies jfk (x) fk (y)j < 3" : Since all fn 2 C(X);
fk is continuous and hence such a ("; x) > 0 exists. Now

jf (x) f (y)j jf (x) fk (x)j + jfk (x) fk (y)j + jfk (y) f (y)j
d(f; fk ) + jfk (x) fk (y)j + d(fk ; f )
" " "
+ + ="
3 3 3

4.3 The Contraction Mapping Theorem

Now we are ready to state the theorem that will give us the existence and
uniqueness of a …xed point of the operator T; i.e. existence and uniqueness of
a function v satisfying v = T v : Let (S; d) be a metric space. Just to clarify,
an operator T (or a mapping) is just a function that maps elements of S into
some other space. The operator that we are interested in maps functions into
functions, but the results in this section apply to any metric space. We start
with a de…nition of what a contraction mapping is.
78 CHAPTER 4. MATHEMATICAL PRELIMINARIES

De…nition 33 Let (S; d) be a metric space and T : S ! S be a function map-

ping S into itself. The function T is a contraction mapping if there exists a
number 2 (0; 1) satisfying

d(T x; T y) d(x; y) for all x; y 2 S

The number is called the modulus of the contraction mapping

A geometric example of a contraction mapping for S = [0; 1]; d(x; y) = jx yj

is contained in SLP, p. 50. Note that a function that is a contraction mapping
is automatically a continuous function, as the next lemma shows

Lemma 34 Let (S; d) be a metric space and T : S ! S be a function mapping

S into itself. If T is a contraction mapping, then T is continuous.

Proof. Remember from the de…nition of continuity we have to show that

for all s0 2 S and all " > 0 there exists a ("; s0 ) such that whenever s 2
S; d(s; s0 ) < ("; s0 ); then d(T s; T s0 ) < ": Fix an arbitrary s0 2 S and " > 0
and pick ("; s0 ) = ": Then

d(T s; T s0 ) d(s; s0 ) ("; s0 ) = " < "

We now can state and prove the contraction mapping theorem. Let by
vn = T n v0 2 S denote the element in S that is obtained by applying the
operator T n-times to v0 ; i.e. the n-th element in the sequence starting with an
arbitrary v0 and de…ned recursively by vn = T vn 1 = T (T vn 2 ) = = T n v0 :
Then we have

Theorem 35 Let (S; d) be a complete metric space and suppose that T : S ! S

is a contraction mapping with modulus : Then a) the operator T has exactly
one …xed point v 2 S and b) for any v0 2 S; and any n 2 N we have
n
d(T n v0 ; v ) d(v0 ; v )

A few remarks before the proof. Part a) of the theorem tells us that there
is a v 2 S satisfying v = T v and that there is only one such v 2 S: Part
b) asserts that from any starting guess v0 ; the sequence fvn g1 n=0 as de…ned
recursively above converges to v at a geometric rate of : This last part is
important for computational purposes as it makes sure that we, by repeatedly
applying T to any (as crazy as can be) initial guess v0 2 S, will eventually
converge to the unique …xed point and it gives us a lower bound on the speed
of convergence. But now to the proof.
Proof. First we prove part a) Start with an arbitrary v0 : As our candidate
for a …xed point we take v = limn!1 vn : We …rst have to establish that the
sequence fvn g1n=0 in fact converges to a function v : We then have to show that
this v satis…es v = T v and we then have to show that there is no other v^
that also satis…es v^ = T v^
4.3. THE CONTRACTION MAPPING THEOREM 79

Since by assumption T is a contraction

d(vn+1 ; vn ) = d(T vn ; T vn 1 ) d(vn ; vn 1 )

2
= d(T vn 1 ; T vn 2 ) d(vn 1 ; vn 2)
n
= = d(v1 ; v0 )

where we used the way the sequence fvn g1n=0 was constructed, i.e. the fact that
vn+1 = T vn : For any m > n it then follows from the triangle inequality that

d(vm ; vn ) d(vm ; vm 1 ) + d(vm 1 ; vn )

d(vm ; vm 1 ) + d(vm 1 ; vm 2 ) + + d(vn+1 ; vn )
m m 1 n
d(v1 ; v0 ) + d(v1 ; v0 ) + d(v1 ; v0 )
n m n 1
= + + + 1 d(v1 ; v0 )
n
d(v1 ; v0 )
1
By making n large we can make d(vm ; vn ) as small as we want. Hence the
sequence fvn g1
n=0 is a Cauchy sequence. Since (S; d) is a complete metric space,
the sequence converges in S and therefore v = limn!1 vn is well-de…ned.
Now we establish that v is a …xed point of T; i.e. we need to show that
T v = v : But

Tv = T lim vn = lim T (vn ) = lim vn+1 = v

n!1 n!1 n!1

Note that the fact that T (limn!1 vn ) = limn!1 T (vn ) follows from the conti-
nuity of T:3
Now we want to prove that the …xed point of T is unique. Suppose there
exists another v^ 2 S such that v^ = T v^ and v^ 6= v : Then there exists c > 0 such
that d(^v ; v ) = a: But

0 < a = d(^
v ; v ) = d(T v^; T v ) d(^
v; v ) = a

a contradiction. Here the second equality follows from the fact that we assumed
that both v^; v are …xed points of T and the inequality follows from the fact
that T is a contraction.
We prove part b) by induction. For n = 0 (using the convention that T 0 v =
v) the claim automatically holds. Now suppose that
k
d(T k v0 ; v ) d(v0 ; v )

We want to prove that

k+1
d(T k+1 v0 ; v ) d(v0 ; v )
3 Almost by de…nition. Since T is continuous for every " > 0 there exists a (") such that

d(vn v ) < (") implies d(T (vn ) T (v )) < ": Hence the sequence fT (vn )g1n=0 converges and
limn!1 T (vn ) is well-de…ned. We showed that limn!1 vn = v : Hence both limn!1 T (vn )
and limn!1 vn are well-de…ned. Then obviously limn!1 T (vn ) = T (v ) = T (limn!1 vn ):
80 CHAPTER 4. MATHEMATICAL PRELIMINARIES

But
k+1
d(T k+1 v0 ; v ) = d(T T k v0 ; T v ) d(T k v0 ; v ) d(v0 ; v )

where the …rst inequality follows from the fact that T is a contraction and the
second follows from the induction hypothesis.
The following corollary, which I will state without proof, will be very useful
in establishing properties (such as continuity, monotonicity, concavity) of the
unique …xed point v and the associated policy correspondence.

Corollary 36 Let (S; ) be a complete metric space, and let T : S ! S be a

contraction mapping with …xed point v 2 S: If S 0 is a closed subset of S and
T (S 0 ) S 0 ; then v 2 S 0 : If in addition T (S 0 ) S 00 S 0 ; then v 2 S 00 :

The Contraction Theorem is is extremely useful in order to establish that our

functional equation of interest has a unique …xed point. It is, however, not very
operational as long as we don’t know how to determine whether a given operator
is a contraction mapping. There is some good news, however. Blackwell, in 1965
provided su¢ cient conditions for an operator to be a contraction mapping. It
turns out that these conditions can be easily checked in a lot of applications.
Since they are only su¢ cient however, failure of these conditions does not imply
that the operator is not a contraction. In these cases we just have to look
somewhere else. Here is Blackwell’s theorem.

Theorem 37 Let X RL and B(X) be the space of bounded functions f :

X ! R with the d being the sup-norm. Let T : B(X) ! B(X) be an operator
satisfying

1. Monotonicity: If f; g 2 B(X) are such that f (x) g(x) for all x 2 X;

then (T f ) (x) (T g) (x) for all x 2 X:
2. Discounting: Let the function f + a; for f 2 B(X) and a 2 R+ be de…ned
by (f + a)(x) = f (x) + a (i.e. for all x the number a is added to f (x)).
There exists 2 (0; 1) such that for all f 2 B(X); a 0 and all x 2 X

[T (f + a)](x) [T f ](x) + a

If these two conditions are satis…ed, then the operator T is a contraction

with modulus :

Proof. In terms of notation, if f; g 2 B(X) are such that f (x) g(x) for
all x 2 X; then we write f g: We want to show that if the operator T satis…es
conditions 1. and 2. then there exists 2 (0; 1) such that for all f; g 2 B(X)
we have that d(T f; T g) d(f; g):
Fix x 2 X: Then f (x) g(x) supy2X jf (y) g(y)j: But this is true for all
x 2 X: So using our notation we have that f g + d(f; g) (which means that for
any value of x 2 X; adding the constant d(f; g) to g(x) gives something bigger
than f (x):
4.3. THE CONTRACTION MAPPING THEOREM 81

But from f g + d(f; g) it follows by monotonicity that

Tf T [g + d(f; g)]
T g + d(f; g)

where the last inequality comes from discounting. Hence we have

Tf Tg d(f; g)

Switching the roles of f and g around we get

(T f T g) d(g; f ) = d(f; g)

(by symmetry of the metric). Combining yields

(T f ) (x) (T g) (x) d(f; g) for all x 2 X

(T g) (x) (T f ) (x) d(f; g) for all x 2 X

Therefore
sup j(T f ) (x) (T g) (x)j = d(T f; T g) d(f; g)
x2X

and T is a contraction mapping with modulus :

Note that do not require the functions in B(X) to be continuous. It is
straightforward to prove that (B(X); d) is a complete metric space once we
proved that (B(X); d) is a complete metric space. Also note that we could
restrict ourselves to continuous and bounded functions and Blackwell’s theorem
obviously applies. Note however that Blackwells theorem requires the metric
space to be a space of functions, so we lose generality as compared to the
Contraction mapping theorem (which is valid for any complete metric space).
But for our purposes it is key that, once Blackwell’s conditions are veri…ed we
can invoke the CMT to argue that our functional equation of interest has a
unique solution that can be obtained by repeated iterations on the operator T:
We can state an alternative version of Blackwell’s theorem

Theorem 38 Let X RL and B(X) be the space of bounded functions f :

X ! R with the d being the sup-norm. Let T : B(X) ! B(X) be an operator
satisfying

1. Monotonicity: If f; g 2 B(X) are such that f (x) g(x) for all x 2 X;

then (T f ) (x) (T g) (x) for all x 2 X:
2. Discounting: Let the function f + a; for f 2 B(X) and a 2 R+ be de…ned
by (f + a)(x) = f (x) + a (i.e. for all x the number a is added to f (x)).
There exists 2 (0; 1) such that for all f 2 B(X); a 0 and all x 2 X

[T (f a)](x) [T f ](x) + a

If these two conditions are satis…ed, then the operator T is a contraction

with modulus :
82 CHAPTER 4. MATHEMATICAL PRELIMINARIES

The proof is identical to the …rst theorem and hence omitted.

As an application of the mathematical structure we developed let us look
back at the neoclassical growth model. The operator T corresponding to our
functional equation was

T v(k) = max fU (f (k) k 0 ) + v(k 0 )g

0 k0 f (k)

De…ne as our metric space (B[0; 1); d) the space of bounded functions on [0; 1)
with d being the sup-norm. We want to argue that this operator has a unique
…xed point and we want to apply Blackwell’s theorem and the CMT. So let us
verify that all the hypotheses for Blackwell’s theorem are satis…ed.

1. First we have to verify that the operator T maps B[0; 1) into itself (this
is very often forgotten). So if we take v to be bounded, since we assumed
that U is bounded, then T v is bounded. Note that you may be in big
trouble here if U is not bounded.4

2. How about monotonicity. It is obvious that this is satis…ed. Suppose

v w: Let by gv (k) denote an optimal policy (need not be unique) corre-
sponding to v: Then for all k 2 (0; 1)

T v(k) = U (f (k) gv (k)) + v(gv (k))

U (f (k) gv (k)) + w(gv (k))
max
0
fU (f (k) k 0 ) + w(k 0 )g
0 k f (k)

= T w(k)

Even by applying the policy gv (k) (which need not be optimal for the
situation in which the value function is w) gives higher T w(k) than T v(k):
Choosing the policy for w optimally does only improve the value (T v) (k):

3. Discounting. This also straightforward

T (v + a)(k) = max fU (f (k) k 0 ) + (v(k 0 ) + a)g

0 k0 f (k)

= max fU (f (k) k 0 ) + v(k 0 )g + a

0 k0 f (k)

= T v(k) + a
4 Somewhat surprisingly, in many applications the problem is that u is not bounded below;

unboundedness from above is sometimes easy to deal with.

We made the assumption that f 2 C 2 f 0 > 0; f 00 < 0; limk&0 f 0 (k) = 1 and
limk!1 f 0 (k) = 1 : Hence there exists a unique k ^ such that f (k)
^ = k: ^ Hence for all
kt > k^ we have kt+1 f (kt ) < kt : Therefore we can e¤ectively restrict ourselves to capital
^ Hence, even if u is not bounded above we have that for all
stocks in the set [0; max(k0 ; k)]:
feasible paths policies u(f (k) k0 ) u(f (max(k0 ; k))^ < 1; and hence by sticking a function
v into the operator that is bounded above, we get a T v that is bounded above. Lack of
boundedness from below is a much harder problem in general.
4.4. THE THEOREM OF THE MAXIMUM 83

Hence the neoclassical growth model with bounded utility satis…es the Su¢ -
cient conditions for a contraction and there is a unique …xed point to the func-
tional equation that can be computed from any starting guess v0 be repeated
application of the T -operator.
One can also prove some theoretical properties of the Howard improvement
algorithm using the Contraction Mapping Theorem and Blackwell’s conditions.
Even though we could state the results in much generality, we will con…ne our
discussion to the neoclassical growth model. Remember that the Howard im-
provement algorithm iterates on feasible policies [TBC]

4.4 The Theorem of the Maximum

An important theorem is the theorem of the maximum. It will help us to
establish that, if we stick a continuous function f into our operator T; the
resulting function T f will also be continuous and the optimal policy function
will be continuous in an appropriate sense.
We are interested in problems of the form

h(x) = max ff (x; y)g

y2 (x)

The function h gives the value of the maximization problem, conditional on the
state x: We de…ne

G(x) = fy 2 (x) : f (x; y) = h(x)g

Hence G is the set of all choices y that attain the maximum of f , given the state
x; i.e. G(x) is the set of argmax’es. Note that G(x) need not be single-valued.
In the example that we study the function f will consist of the sum of the
current return function r and the continuation value v and the constraint set
describes the resource constraint. The theorem of the maximum is also widely
used in microeconomics. There, most frequently x consists of prices and income,
f is the (static) utility function, the function h is the indirect utility function,
is the budget set and G is the set of consumption bundles that maximize utility
at x = (p; m):
Before stating the theorem we need a few de…nitions. Let X; Y be arbitrary
sets (in what follows we will be mostly concerned with the situations in which
X and Y are subsets of Euclidean spaces. A correspondence : X ) Y maps
each element x 2 X into a subset (x) of Y: Hence the image of the point x
under may consist of more than one point (in contrast to a function, in which
the image of x always consists of a singleton).

De…nition 39 A compact-valued correspondence : X ) Y is upper-hemicontinuous

at a point x if (x) 6= ? and if for all sequences fxn g in X converging to some
x 2 X and all sequences fyn g in Y such that yn 2 (xn ) for all n; there exists a
convergent subsequence of fyn g that converges to some y 2 (x): A correspon-
dence is upper-hemicontinuous if it is upper-hemicontinuous at all x 2 X:
84 CHAPTER 4. MATHEMATICAL PRELIMINARIES

A few remarks: by talking about convergence we have implicitly assumed

that X and Y (together with corresponding metrics) are metric spaces. Also,
a correspondence is compact-valued, if for all x 2 X; (x) is a compact set.
Also this de…nition requires to be compact-valued. With this additional re-
quirement the de…nition of upper hemicontinuity actually corresponds to the
de…nition of a correspondence having a closed graph. See, e.g. Mas-Colell et al.
p. 949-950 for details.

De…nition 40 A correspondence : X ) Y is lower-hemicontinuous at a

point x if (x) 6= ? and if for every y 2 (x) and every sequence fxn g in X
converging to x 2 X there exists N 1 and a sequence fyn g in Y converging to
y such that yn 2 (xn ) for all n N: A correspondence is lower-hemicontinuous
if it is lower-hemicontinuous at all x 2 X:

De…nition 41 A correspondence : X ) Y is continuous if it is both upper-

hemicontinuous and lower-hemicontinuous.

Note that a single-valued correspondence (i.e. a function) that is upper-

hemicontinuous is continuous. Now we can state the theorem of the maximum.

Theorem 42 Let X RL and Y RM ; let f : X Y ! R be a contin-

uous function, and let : X ) Y be a compact-valued and continuous cor-
respondence. Then h : X ! R is continuous and G : X ! Y is nonempty,
compact-valued and upper-hemicontinuous.

The proof is somewhat tedious and omitted here (you probably have done
it in micro anyway).
Chapter 5

Dynamic Programming

5.1 The Principle of Optimality

In the last section we showed that under certain conditions, the functional equa-
tion (F E)
v(x) = sup fF (x; y) + v(y)g
y2 (x)

has a unique solution which is approached from any initial guess v0 at geometric
speed. What we were really interested in, however, was a problem of sequential
form (SP )
1
X
t
w(x0 ) = sup F (xt ; xt+1 )
fxt+1 g1
t=0 t=0
s:t: xt+1 2 (xt )
x0 2 X given

Note that I replaced max with sup since we have not made any assumptions
so far that would guarantee that the maximum in either the functional equation
or the sequential problem exists. In this section we want to …nd out under what
conditions the functions v and w are equal and under what conditions optimal
sequential policies fxt+1 g1 t=0 are equivalent to optimal policies y = g(x) from
the recursive problem, i.e. under what conditions the principle of optimality
holds. It turns out that these conditions are very mild.
In this section I will try to state the main results and make clear what they
mean; I will not prove the results. The interested reader is invited to consult
Stokey and Lucas or Bertsekas. Unfortunately, to make our results precise
additional notation is needed. Let X be the set of possible values that the state
x can take. X may be a subset of a Euclidean space, a set of functions or
something else; we need not be more speci…c at this point. The correspondence
: X ) X describes the feasible set of next period’s states y; given that today’s

85
86 CHAPTER 5. DYNAMIC PROGRAMMING

state is x: The graph of , A is de…ned as

A = f(x; y) 2 X X : y 2 (x)g

The period return function F : A ! R maps the set of all feasible combinations
of today’s and tomorrow’s state into the reals. So the fundamentals of our
analysis are (X; F; ; ): For the neoclassical growth model F and describe
preferences and X; describe the technology.
We call any sequence of states fxt g1
t=0 a plan. For a given initial condition
x0 ; the set of feasible plans (x0 ) from x0 is de…ned as

(x0 ) = ffxt g1
t=1 : xt+1 2 (xt )g

Hence (x0 ) is the set of sequences that, for a given initial condition, satisfy all
the feasibility constraints of the economy. We will denote by x a generic element
of (x0 ): The two assumptions that we need for the principle of optimality are
basically that for any initial condition x0 the social planner (or whoever solves
the problem) has at least one feasible plan and that the total return (the total
utility, say) from all feasible plans can be evaluated. That’s it. More precisely
we have
Assumption 1: (x) is nonempty for all x 2 X
Assumption 2: For all initial conditions x0 and all feasible plans x 2 (x0 )
n
X
t
lim F (xt ; xt+1 )
n!1
t=0

exists (although it may be +1 or 1).

Assumption 1 does not require much discussion: we don’t want to deal
with an optimization problem in which the decision maker (at least for some
initial conditions) can’t do anything. Assumption 2 is more subtle. There are
various ways to verify that assumption 2 is satis…ed, i.e. various sets of su¢ cient
conditions for assumption 2 to hold. Assumption 2 holds if

1. F is bounded and 2 (0; 1): Note that boundedness of F is not enough.

1 if t even
Suppose = 1 and F (xt ; xt+1 ) = Obviously F is bounded,
1 if t odd
Pn 1 if n even
but since t=0 t F (xt ; xt+1 ) = ; the limit in assumption 2
0 if n odd
n
Pn n
1 2 + if n even
does not exist. If 2 (0; 1) then t=0 t F (xt ; xt+1 ) = n
1 2 if n odd
Pn
and therefore limn!1 t=0 t F (xt ; xt+1 ) exists and equals 1: In general
the joint assumption
Pn that F is bounded and 2 (0; 1) implies that the
sequence yn = t=0 t F (xt ; xt+1 ) is Cauchy and hence converges. In this
case lim yn = y is obviously …nite.

2. De…ne F + (x; y) = maxf0; F (x; y)g and F (x; y) = maxf0; F (x; y)g:
5.1. THE PRINCIPLE OF OPTIMALITY 87

Then assumption 2 is satis…ed if for all x0 2 X; all x 2 (x0 ); either

n
X
t
lim F + (xt ; xt+1 ) < +1 or
n!1
t=0
Xn
t
lim F (xt ; xt+1 ) < +1
n!1
t=0

or both. For example, if 2 (0; 1) and F is bounded above, then the …rst
condition is satis…ed, if 2 (0; 1) and F is bounded below then the second
condition is satis…ed.

3. Assumption 2 is satis…ed if for every x0 2 X and every x 2 (x0 ) there

are numbers (possibly dependent on x0 ; x) 2 (0; 1 ) and 0 < c < +1
such that for all t
F (xt ; xt+1 ) c t
Hence F need not be bounded in any direction for assumption 2 to be
satis…ed. As long as the returns from the sequences do not grow too fast
(at rate higher than 1 ) we are still …ne .

I would conclude that assumption 2 is rather weak (I can’t think of any

interesting economic example where assumption1 is violated, but let me know
if you come up with one). A …nal piece of notation and we are ready to state
some theorems.
De…ne the sequence of functions un : (x0 ) ! R by
n
X
t
un (x) = F (xt ; xt+1 )
t=0

For each feasible plan un gives the total discounted return (utility) up until
period n: If assumption 2 is satis…ed, then the function u : (x0 ) ! R
n
X
t
u(x) = lim F (xt ; xt+1 )
n!1
t=0

is also well-de…ned, since under assumption 2 the limit exists. The range of
u is R; the extended real line, i.e. R = R [ f 1; +1g since we allowed the
limit to be plus or minus in…nity. From the de…nition of u it follows that under
assumption 2
w(x0 ) = sup u(x)
x2 (x0 )

Note that by construction, whenever w exists, it is unique (since the supremum

of a set is always unique). Also note that the way I have de…ned w above only
makes sense under assumption 1. and 2., otherwise w is not well-de…ned.
We have the following theorem, stating the principle of optimality.
88 CHAPTER 5. DYNAMIC PROGRAMMING

Theorem 43 Suppose (X; ; F; ) satisfy assumptions 1. and 2. Then

1. the function w satis…es the functional equation (F E)

2. if for all x0 2 X and all x 2 (x0 ) a solution v to the functional equation

(F E) satis…es
n
lim v(xn ) = 0 (5.1)
n!1

then v = w

I will skip the proof, but try to provide some intuition. The …rst result
states that the supremum function from the sequential problem (which is well-
de…ned under assumption 1. and 2.) solves the functional equation. This result,
although nice, is not particularly useful for us. We are interested in solving the
sequential problem and in the last section we made progress in solving the
functional equation (not the other way around).
But result 2. is really key. It states a condition under which a solution
to the functional equation (which we know how to compute) is a solution to
the sequential problem (the solution of which we desire). Note that the func-
tional equation (F E) may (or may not) have several solution. We haven’t made
enough assumptions to use the CMT to argue uniqueness. However, only one
of these potential several solutions can satisfy (5:1) since if it does, the theo-
rem tells us that it has to equal the supremum function w (which is necessarily
unique). The condition (5:1) is somewhat hard to interpret (and SLP don’t
even try), but think about the following. We saw in the …rst lecture that for
in…nite-dimensional optimization problems like the one in (SP ) a transversality
condition was often necessary and (even more often) su¢ cient (jointly with the
Euler equation). The transversality condition rules out as suboptimal plans that
postpone too much utility into the distant future. There is no equivalent condi-
tion for the recursive formulation (as this formulation is basically a two period
formulation, today vs. everything from tomorrow onwards). Condition (5:1)
basically requires that the continuation utility from date n onwards, discounted
to period 0; should vanish in the time limit. In other words, this puts an upper
limit on the growth rate of continuation utility, which seems to substitute for
the TVC. It is not clear to me how to make this intuition more rigorous, though.
A simple, but quite famous example, shows that the condition (5:1) has
some bite. Consider the following consumption problem of an in…nitely lived
household. The household has initial wealth x0 2 X = R: He can borrow or
lend at a gross interest rate 1 + r = 1 > 1: So the price of a bond that pays o¤
one unit of consumption is q = : There are no borrowing constraints, so the
sequential budget constraint is

ct + xt+1 xt

and the nonnegativity constraint on consumption, ct 0: The household values

5.1. THE PRINCIPLE OF OPTIMALITY 89

discounted consumption, so that her maximization problem is

1
X
t
w(x0 ) = sup ct
f(ct ;xt+1 )g1
t=0 t=0

s:t: 0 ct xt xt+1
x0 given

Since there are no borrowing constraint, the consumer can assure herself in…nite
utility by just borrowing an in…nite amount in period 0 and then rolling over the
debt by even borrowing more in the future. Such a strategy is called a Ponzi-
scheme -see the hand-out. Hence the supremum function equals w(x0 ) = +1
for all x0 2 X: Now consider the recursive formulation (we denote by x current
period wealth xt ; by y next period’s wealth and substitute out for consumption
ct = xt xt+1 (which is OK given monotonicity of preferences)

v(x) = sup fx y + v(y)g

x
y

Obviously the function w(x) = +1 satis…es this functional equation (just plug
in w on the right side, since for all x it is optimal to let y tend to 1 and hence
v(x) = +1: This should be the case from the …rst part of the previous theorem.
But the function v(x) = x satis…es the functional equation, too. Using it on the
right hand side gives, for an arbitrary x 2 X

sup fx y + yg = sup x = x = v(x)

x x
y y

Note, however that the second part of the preceding theorem does not apply
for v since the sequence fxn g de…ned by xn = xn0 is a feasible plan from x0 > 0
and
lim n v(xn ) = lim n xn = x0 > 0
n!1 n!1
Note however that the second part of the theorem gives only a su¢ cient con-
dition for a solution v to the functional equation being equal to the supremum
function from (SP ); but not a necessary condition. Also w itself does not satisfy
the condition, but is evidently equal to the supremum function. So whenever
we can use the CMT (or something equivalent) we have to be aware of the fact
that there may be several solutions to the functional equation, but at most one
the several is the function that we look for.
Now we want to establish a similar equivalence between the sequential prob-
lem and the recursive problem with respect to the optimal policies/plans. The
…rst observation. Solving the functional equation gives us optimal policies
y = g(x) (note that g need not be a function, but could be a correspondence).
Such an optimal policy induces a feasible plan f^ xt+1 g1
t=0 in the following fash-
ion: x0 = x ^0 is an initial condition, x
^1 2 g(^
x0 ) and recursively x^t+1 = g(^xt ):
The basic question is how a plan constructed from a solution to the functional
equation relates to a plan that solves the sequential problem. We have the
following theorem.
90 CHAPTER 5. DYNAMIC PROGRAMMING

Theorem 44 Suppose (X; ; F; ) satisfy assumptions 1. and 2.

1. Let x 2 (x0 ) be a feasible plan that attains the supremum in the sequential
problem. Then for all t 0

w(xt ) = F (xt ; xt+1 ) + w(xt+1 )

2. Let x
^2 (x0 ) be a feasible plan satisfying, for all t 0

w(^
xt ) = F (^
xt ; x
^t+1 ) + w(^
xt+1 )

and additionally1
t
lim sup w(^
xt ) 0 (5.2)
t!1

Then x
^ attains the supremum in (SP ) for the initial condition x0 :

What does this result say? The …rst part says that any optimal plan in the
sequence problem, together with the supremum function w as value function
satis…es the functional equation for all t: Loosely it says that any optimal plan
from the sequential problem is an optimal policy for the recursive problem (once
the value function is the right one).
Again the second part is more important. It says that, for the “right”
…xed point of the functional equation w the corresponding policy g generates
a plan x^ that solves the sequential problem if it satis…es the additional limit
condition. Again we can give this condition a loose interpretation as standing
in for a transversality condition. Note that for any plan f^ xt g generated from a
policy g associated with a value function v that satis…es (5:1) condition (5:2) is
automatically satis…ed. From (5:1) we have
t
lim v(xt ) = 0
t!1

for any feasible fxt g 2 (x0 ); all x0 : Also from Theorem 32 v = w: So for any
plan f^
xt g generated from a policy g associated with v = w we have

w(^
xt ) = F (^
xt ; x
^t+1 ) + w(^
xt+1 )
t
and since limt!1 v(^
xt ) exists and equals to 0 (since v satis…es (5:1)); we have
t
lim sup v(^
xt ) = 0
t!1

and hence (5:2) is satis…ed. But Theorem 33.2 is obviously not redundant as
there may be situations in which Theorem 32.2 does not apply but 33.2 does.
1 The limit superior of a bounded sequence fx g is the in…mum of the set V of real numbers
n
v such that only a …nite number of elements of the sequence strictly exceed v: Hence it is the
largest cluster point of the sequence fxn g:
5.1. THE PRINCIPLE OF OPTIMALITY 91

Let us look at the following example, a simple modi…cation of the saving problem
from before. Now however we impose a borrowing constraint of zero.
1
X
t
w(x0 ) = max1 (xt xt+1 )
fxt+1 gt=0
t=0
xt
s:t: 0 xt+1
x0 given

Writing out the objective function yields

w0 (x0 ) = (x0 x1 ) + (x1 x2 ) + : : :

= x0

Now consider the associated functional equation

v(x) = max
0 x
fx x0 + v(x0 )g
0 x

Obviously one solution of this functional equation is v(x) = x and by Theorem

32.1 is rightly follows that w satis…es the functional equation. However, for
v condition (5:1) fails, as the feasible plan de…ned by xt = x0t shows. Hence
Theorem 32.2 does not apply and we can’t conclude that v = w (although we
have veri…ed it directly, there may be other examples for which this is not so
straightforward). Still we can apply Theorem 33.2 to conclude that certain plans
are optimal plans. Let f^ xt g be de…ned by x
^0 = x0 ; x
^t = 0 all t > 0: Then
t
lim sup w(^
xt ) = 0
t!1

and we can conclude by Theorem 33.2 that this plan is optimal for the sequential
problem. There are tons of other plans for which we can apply the same logic to
shop that they are optimal, too (which shows that we obviously can’t make any
claim about uniqueness). To show that condition (5:2) has some bite consider
the plan de…ned by x^t = x0t : Obviously this is a feasible plan satisfying

w(^
xt ) = F (^
xt ; x
^t+1 ) + w(^
xt+1 )

but since for all x0 > 0

t
lim sup w(^
xt ) = x0 > 0
t!1

Theorem 33.2 does not apply and we can’t conclude that f^ xt g is optimal (as in
fact this plan is not optimal).
So basically we have a prescription what to do once we solved our functional
equation: pick the right …xed point (if there are more than one, check the limit
condition to …nd the right one, if possible) and then construct a plan from the
92 CHAPTER 5. DYNAMIC PROGRAMMING

policy corresponding to this …xed point. Check the limit condition to make sure
that the plan so constructed is indeed optimal for the sequential problem. Done.
Note, however, that so far we don’t know anything about the number (unless
the CMT applies) and the shape of …xed point to the functional equation. This
is not quite surprising given that we have put almost no structure onto our
economy. By making further assumptions one obtains sharper characterizations
of the …xed point(s) of the functional equation and thus, in the light of the
preceding theorems, about the solution of the sequential problem.

5.2 Dynamic Programming with Bounded Re-

turns
Again we look at a functional equation of the form

v(x) = max fF (x; y) + v(y)g

y2 (x)

We will now assume that F : X X is bounded and 2 (0; 1): We will make
the following two assumptions throughout this section
Assumption 3: X is a convex subset of RL and the correspondence :
X ) X is nonempty, compact-valued and continuous.
Assumption 4: The function F : A ! R is continuous and bounded, and
2 (0; 1)
We immediately get that assumptions 1. and 2. are satis…ed and hence
the theorems of the previous section apply. De…ne the policy correspondence
connected to any solution to the functional equation as

G(x) = fy 2 (x) : v(x) = F (x; y) + v(y)g

and the operator T on C(X)

(T v) (x) = max fF (x; y) + v(y)g

y2 (x)

Here C(X) is the space of bounded continuous functions on X and we use the
sup-metric as metric. Then we have the following

Theorem 45 Under Assumptions 3. and 4. the operator T maps C(X) into

itself. T has a unique …xed point v and for all v0 2 C(X)
n
d(T n v0 ; v) d(v0 ; v)

The policy correspondence G belonging to v is compact-valued and upper-hemicontinuous

Now we add further assumptions on the structure of the return function F;

with the result that we can characterize the unique …xed point of T better.
Assumption 5: For …xed y; F (:; y) is strictly increasing in each of its L
components.
5.2. DYNAMIC PROGRAMMING WITH BOUNDED RETURNS 93

Assumption 6: is monotone in the sense that x x0 implies (x)

0
(x ):
The result we get out of these assumptions is strict monotonicity of the value
function.

Theorem 46 Under Assumptions 3. to 6. the unique …xed point v of T is

strictly increasing.

We have a similar result in spirit if we make assumptions about the curvature

of the return function and the convexity of the constraint set.
Assumption 7: F is strictly concave, i.e. for all (x; y); (x0 ; y 0 ) 2 A and
2 (0; 1)

F [ (x; y) + (1 )(x0 ; y 0 )] F (x; y) + (1 )F (x0 ; y 0 )

and the inequality is strict if x 6= x0

Assumption 8: is convex in the sense that for 2 [0; 1] and x; x0 2 X;
the fact y 2 (x); y 0 2 (x0 )

y + (1 )y 0 2 ( x + (1 )x0 )

Again we …nd that the properties assumed about F extend to the value function.

Theorem 47 Under Assumptions 3.-4. and 7.-8. the unique …xed point of v
is strictly concave and the optimal policy is a single-valued continuous function,
call it g.

Finally we state a result about the di¤erentiability of the value function,

the famous envelope theorem (some people call it the Benveniste-Scheinkman
theorem).
Assumption 9: F is continuously di¤erentiable on the interior of A:

Theorem 48 Under assumptions 3.-4. and 7.-9. if x0 2 int(X) and g(x0 ) 2

int( (x0 )); then the unique …xed point of T; v is continuously di¤ erentiable at
x0 with
@v(x0 ) @F (x0 ; g(x0 ))
=
@xi @xi
where @v(x
@xi
0)
denotes the derivative of v with respect to its i-th component, eval-
uated at x0 :

This theorem gives us an easy way to derive Euler equations from the re-
cursive formulation of the neoclassical growth model. Remember the functional
equation
v(k) = max 0
U (f (k) k 0 ) + v(k 0 )
0 k f (k)

Taking …rst order conditions with respect to k 0 (and ignoring corner solutions)
we get
U 0 (f (k) k 0 ) = v 0 (k 0 )
94 CHAPTER 5. DYNAMIC PROGRAMMING

Denote by k 0 = g(k) the optimal policy. The problem is that we don’t know v 0 :
But now we can use Benveniste-Scheinkman to obtain

v 0 (k) = U 0 (f (k) g(k))f 0 (k)

Using this in the …rst order condition we obtain

U 0 (f (k) g(k)) = v 0 (k) = U 0 (f (k 0 ) g(k 0 ))f 0 (k 0 )

= f 0 (g(k))U 0 (f (g(k)) g(g(k))

Denoting k = kt ; g(k) = kt+1 and g(g(k)) = kt+2 we obtain our usual Euler
equation
U 0 (f (kt ) kt+1 ) = f (kt+1 )U 0 (f (kt+1 ) kt+2 )