Chapter 4

Stability

Topics :

1. Basic Concepts

2. Algebraic Criteria for Linear Systems

3. Lyapunov Theory with Applications to Linear Systems

4. Stability and Control

c Claudiu C. Remsing, 2006.

Copyright
All rights reserved.

113
114 AM3.2 - Linear Control

Stability was probably the first question in classical dynamical systems which was
dealt with in a satisfactory way. Stability questions motivated the introduction of
new mathematical concepts (tools) in engineeering, particularly in control engineering.
Stability theory has been of interest to mathematicians and astronomers for a long
time and has had a stimulating impact on these fields. The specific problem of
attempting to prove that the solar system is stable accounted for the introduction of
many new methods.
Our treatment of stability will apply to (control) systems described by sets of
linear or nonlinear equations. As is to be expected, however, our most explicit results
will be obtained for linear systems.

 C.C. Remsing 115

4.1 Basic Concepts

Consider the nonlinear dynamical system described by

x = F (t, x), x Rm (4.1)

where x() is a curve in the state space Rm and F is a vector-valued mapping

having components Fi , i = 1, 2, . . ., m.

Note : We shall assume that the components Fi are continuous and satisfy stan-
dard conditions, such as having continuous first order partial derivatives so that the
solution curve of (4.1) exists and is unique for any given initial condition (state).
From a geometric point of view, the right-hand side (rhs) F can be interpreted as
a time-dependent vector field on Rm . So a (nonlinear) dynamical system is essentially
the same as (and thus can be identified with) a vector field on the state space. This
point of view is very fruitfull and extremely useful in investigating the properties of
the dynamical system (especially when the state space is a manifold).

If the functions Fi do not depend explicitly on t, then system is called

autonomous (or time-independent); otherwise, nonautonomous (or time-dependent).

Equilibrium states

4.1.1 Definition. If F (t, c) = 0 for all t, then c Rm is said to be an

equilibrium (or critical) state.

It follows at once from (4.1) that (for an equilibrium state c ) if x(t0 ) = c,

then x(t) = c for all t t0 . Thus solution curves starting at c remain there.

Clearly, by introducing new variables x0i = xi ci we can arrange for the

equilibrium state to be transferred to the origin (of the state space Rm ); we
shall assume that this has been done for any equilibrium state under consid-
eration (there may well be several for a given system ), so that we then
have
F (t, 0) = 0
116 AM3.2 - Linear Control

for all t t0 .

We shall also assume that there is no other constant solution curve in the
neighborhood of the origin, so this is an isolated equilibrium state.

4.1.2 Example. The intuitive idea of stability in a dynamical setting is

that for small perturbations from the equilibrium state at some time t0 ,
subsequent motions t 7 x(t), t t0 should not be too large. Consider a
ball resting in equilibrium on a sheet of metal bent into various shapes with
cross-sections as shown.

(i) (ii) (iii)

If frictional forces can be neglected, then small perturbations lead to :

oscillatory motion about equilibrium (case (i)) ;

the ball moving away without returning to equilibrium (case (ii));
oscillatory motion about equilibrium, unless the initial perturbation
is so large that the ball is forced to oscillate about a new equilibrium
position (case (iii)).

If friction is taken into account then the oscillatory motions steadily decrease
until the equilibrium state is returned to.

Stability

There is no single concept of stability, and many different definitions are

possible. We shall consider only the following fundamental statements.
 C.C. Remsing 117

4.1.3 Definition. An equilibrium state x = 0 is said to be :

(a) stable if for any positive scalar there exists a positive scalar
such that kx(t0 )k < implies kx(t)k < for all t t0 .

(b) asymptotically stable if it is stable and if in addition x(t) 0

as t .

(c) unstable if it is not stable; that is, there exists an > 0 such that
for every > 0 there exists an x(t0 ) with kx(t0 )k < , kx(t1 )k
for some t1 > t0 .

(d) completely unstable if there exists an > 0 such that for every
> 0 and for every x(t0 ) with kx(t0 )k < , kx(t1 )k for some
t1 > t0 .

Note : The definition (a) is often called stability in the sense of Lyapunov
(stability i.s.L.) after the Russian mathematician Aleksandr M. Lyapunov (1857-
1918), whose important work features prominently in current control theory.

In Example 4.1.2, case (i) represents stability i.s.L. if friction is ig-

nored, and asymptotic stability if friction is taken into account, whereas case
(ii) represents instability. If the metal sheet in (i) were thought to extend
indefinitely then, if friction is present, the ball would eventually return to
equilibrium no matter how large the disturbance. This is an illustration of
asymptotic stability in the large, which means that every motion converges to
a single equilibrium point (state) as t , and clearly does not apply to case
(iii). Asymptotic stability in the large implies that all motions are bounded.
Generally,

4.1.4 Definition. An equilibrium state x = 0 is said to be bounded (or

Lagrange stable) if there exists a constant M , which may depend on t0 and
x(t0 ), such that kx(t)k M for all t t0 .

Some remarks can be made at this stage.

118 AM3.2 - Linear Control

1. Regarded as a function of t in the ( n-dimensional) state space, the solu-

tion x() of (4.1) is called a trajectory (or motion). In two dimensions
we can give the definitions a simple geometric interpretation.

If the origin O is stable, then given the outer circle C, radius ,

there exists an inner circle C1 , radius 1 , such that trajectories
starting within C1 never leave C.
If O is asymptotically stable, then there is some circle C2 , radius
2 , having the same property as C1 but, in addition, trajectories
starting inside C2 tend to O as t .

2. We refer to the stability of an equilibrium state of , not the system

itself, as different equilibrium states may have different stability proper-
ties.

3. A weakness of the definition of stability i.s.L. for practical purposes is

that only the existence of some positive is required, so may be very
small compared to ; in other words, only very small disturbances from
equilibrium may be allowable.

4. In engineering applications, asymptotic stability is more desirable than

stability since it ensures eventual return to equilibrium, whereas stability
allows continuing deviations not to far from the equilibrium state.

Examples

Some further aspects of stability are now illustrated through some exam-
ples.

4.1.5 Example. We return again to the environment problem involving

rabbits anf foxes, and let the equations have the following numerical form

x 1 = 2x1 3x2

x = 2x x .
2 1 2
 C.C. Remsing 119

These equations have a single equilibrium point (state) at the origin. With
arbitrary initial numbers x1 (0) and x2 (0) of rabbits and foxes, respectively,
the solution is
!
t 15 3 15 6x2 (0) t 15
x1 (t) = x1 (0)e 2 cos t + sin t e 2 sin t
2 15 2 15 2

with a similar expression for x2 (t). Clearly, x1 (t) tends to infinity as t ,

irrespective of the initial state, so the origin is unstable.

4.1.6 Example. Consider the (initialized) dynamical system on R de-

scribed by
x = x2 , x(0) = x0 R.

It is clear that the solution exists and is unique; in fact, by integrating, we

easily obtain
1 1
= t
x x0
Hence
1
x(t) = 1
x0 t
1
so that if x0 > 0, x(t) as t x0 . The solution is said to escape to
infinity in a finite time, or to have a finite escape time. We shall henceforth
exclude this situation and assume that (4.1) has a finite solution for all finite
t t0 , for otherwise (4.1) cannot be a mathematical model of a real-life
situation.

4.1.7 Example. We demonstrate that the origin is a stable equilibrium

state for the (initialized) system described by

x = (1 2t)x, x(t0 ) = x0 R

by determining explicitly the scalar in the definition. Integrating the equa-

tion gives
2 2
x(t) = x0 ett et0 t0 .
120 AM3.2 - Linear Control

The condition |x(t)| < leads to

2 t 2
|x0 | < et et0 t0 .

2 t 1
Since t 7 et has a minimum value of e 4 when t = 12 , it follows that we
can take
1 2
= e(t0 2 ) .

In general, will depend upon , but in this example it is also a function of

the initial time. If is independent of t0 , the stability is called uniform.

4.1.8 Example. Consider the (initialized) dynamical system on R de-

scribed by

x(t) 2 if x(t) > 2



x(t) = , x(t0 ) = x0 R.


0 if x(t) 2.

The solution is easily found to be

tt0
2 + (x0 2)e if x0 > 2


x(t) =


x0 if x0 2.

The condition |x(t)| < is implied by |x0 | < 2 when 2, for then |x(t)| =
|x0 | < 2 < . When < 2, |x(t)| < is implied by |x0 | < , for then
again |x(t)| = |x0 | < . Thus according to the definition, the origin is a stable
equilibrium point (state). However, if x0 > 2 then x(t) , so for initial
perturbations x0 > 2 from equilibrium motions are certainly unstable in a
practical sense.

4.1.9 Example. Consider the equation

x = f (t)x, x(0) = x0 R
 C.C. Remsing 121

where
ln 10 if 0 t 10


f (t) =


1 if t > 10.
The solution is



10t x0 if 0 t 10
x(t) =

1010 x e10t

if t > 10.
0

Clearly, x(t) 0 as t , and the origin is asymptotically stable. However,

if x0 changes by a very small amount, say 105 , then the corresponding
change in x(t) is relatively large for example, when t = 20, the change in
x(t) is 1010 105 e1020 4.5.

Note : Examples 4.1.8 and 4.1.9 show that an equilibrium state may be stable
according to Lyapunovs definitions and yet the systems behaviour may be unsatis-
factory from a practical point of view. The converse situation is also possible, and
this has led to a definition of practical stability being coined for systems which
are unstable in Lyapunovs sense but have an acceptable performance in practice,
namely that for pre-specified deviations from equilibrium the subsequent motions also
lie within specified limits.

4.2 Algebraic Criteria for Linear Systems

We return to the general linear (time-invariant) system given by

x = Ax, x Rm (4.2)

where A Rmm and (4.2) may represent the closed or open loop system.
Provided the matrix A is nonsingular, the only equilibrium state of (4.2) is
the origin, so it is meaningful to refer to the stability of the system (4.2). If
the system is stable (at the origin) but not asymptotically stable we shall call
it neutrally stable.
122 AM3.2 - Linear Control

One of the basic results on which the development of linear system stability
theory relies is now given. The proof will be omitted.

4.2.1 Theorem. (Stability Properties of a Linear System) Consider

the linear system (4.2), and for each eigenvalue of A, suppose that m de-
notes the algebraic multiplicity of and d the geometric multiplicity of .
Then :

(a) The system is asymptotically stable if and only if A is a stability

matrix; that is, every eigenvalue of A has a negative real part.

(b) The system is neutrally stable if and only if

every eigenvalue of A has a nonpositive real part, and

at least one eigenvalue has a zero real part, and d = m for
every eigenvalue with a zero real part.

(c) The system is unstable if and only if

some eigenvalue of A has a positive real part, or

there is an eigenvalue with a zero real part and d < m .

Note : (1) Suppose all the eigenvalues of A have nonpositive real parts. One
can prove that if all eigenvalues having zero real parts are distinct, then the origin is
neutrally stable.

(2) Also, if every eigenvalue of A has a positive real part, then the system is com-
pletely unstable.

4.2.2 Example. In Example 4.1.5 the system matrix is of the form

" #
a1 a2
A=
a3 a4

where a1 , a2 , a3 , a4 > 0. It is easy to show that

det (I2 A) = ( d)
 C.C. Remsing 123

using the condition

a1 a2
=
a3 a4
Hence A has a single zero eigenvalues, so the system is neutrally stable provided
d( = a1 a4 ) is negative. (If a1 = a4 then the system is unstable.)

Note : The preceding theorem applies if A is real or complex, so the stability

determination of (4.2) can be carried out by computing the eigenvalues using one of
the powerful standard computer programs now available. However, if m is small (say
less than six), or if some of the elements of A are in parametric form, or if access to
a digital computer is not possible, then the classical results given below are useful.

Because of its practical importance the linear system stability problem has
attracted attention for a considerable time, an early study being by James C.
Maxwell (1831-1879) in connection with the governing of steam engines.
The original formulation of the problem was not of course in matrix terms,
the system model being

z (m) + k1 z (m1) + + km z = u(t). (4.3)

This is equivalent to working with the characteristic polynomial of A, which

we shall write in this section as

a() : = det (Im A) = m + a1 m1 + + am1 + am . (4.4)

The first solutions giving necessary and sufficient conditions for all the roots of
a() in (4.4) to have negative real parts were given by Augustin L. Cauchy
(1789-1857), Jacques C.F. Sturm (1803-1855), and Charles Hermite
(1822-1901).

We give here a well-known result due to Adolf Hurwitz (1859-1919)

for the case when all the coefficients ai are real. The proof will be omitted.

4.2.3 Theorem. (Hurwitz) The m m Hurwitz matrix associated

124 AM3.2 - Linear Control

with the characteristic polynomial a() of A in (4.4) is

a1 a3 a5 . . . a2m1


1 a2 a4 . . . a2m2

0 a1 a3 . . . a2m3


H := (4.5)

0 1 a2 . . . a2m4

.. .. .. ..

. . . .

0 0 0 ... am

where ar = 0, r > m. Let Hi denote the ith leading principal minor of

H. Then all the roots of a() have negative real parts ( a() is a Hurwitz
polynomial ) if and only if Hi > 0, i = 1, 2, . . ., m.

Note : A disadvantage of Theorem 4.2.3 is the need to evaluate determinants of

increasing order, and a convenient way of avoiding this is due to Edward J. Routh
(1831-1907). We will give only the test as it applies to polynomials of degree no
more than four, although the test can be extended to any polynomial.

4.2.4 Proposition. (The Routh Test) All the roots of the polynomial
a() (with real coefficients) have negative real parts precisely when the given
conditions are met.

2 + a1 + a2 : all the coefficients are positive ;

3 + a1 2 + a2 + a3 : all the coefficients are positive and a1 a2 > a3 ;

4 + a1 3 + a2 2 + a3 + a4 : all the coefficients are positive, a1 a2 > a3

and a1 a2 a3 > a21 a4 + a23 .

Note : The Hurwitz and Routh tests can be useful for determining stability
of (4.3) and (4.2) in certain cases. However, it should be noted that a practical
disadvantage of application to (4.2) is that it is very difficult to calculate accurately
the ai in (4.4). This is important because small errors in the ai can lead to large
errors in the roots of a().
 C.C. Remsing 125

4.2.5 Example. Investigate the stability of the linear system whose char-
acteristic equation is

4 + 23 + 92 + 4 + 1 = 0.

Solution : The polynomial a() = 4 + 23 + 92 + 4 + 1 has positive

coefficients, and since a1 = 2, a2 = 9, a3 = 4 and a4 = 1, the polynomial
meets the Routh conditions :

18 = a1 a2 > a3 = 4 and 72 = a1 a2 a3 > a21 a4 + a23 = 20 .

So all the roots have negative real parts, and hence the linear system is asymp-
totically stable.

Since we have assumed that the coefficients ai are real it is easy to derive
a simple necessary condition for asymptotic stability :

4.2.6 Proposition. If the coefficients ai in (4.4) are real and a() cor-
responds to an asymptotically stable system, then

ai > 0 , i = 1, 2, . . ., m.

Proof : Any complex root of a() will occur in conjugate pairs i, the
corresponding factor of a() being

( i)( + i) = 2 2 + 2 + 2 .

By Theorem 4.2.1, < 0, and similarly any real factor of a() can be
written ( + ) with > 0. Thus
Y Y
a() = ( + ) (2 2 + 2 + 2 )

and since all the coefficients above are positive, ai must also all be positive.2

Note : Of course the condition above is not a sufficient condition, but it provides a
useful initial check : if any ai are negative or zero, then a() cannot be asymptotically
stable.
126 AM3.2 - Linear Control

When we turn to linear time-varying systems

x = A(t)x , x(t0 ) = x0 (4.6)

the situation is much more complicated. In view of Theorem 4.2.1 it might

be thought that if the eigenvalues of A(t) all have negative real parts for all
t t0 , then the origin of (4.6) would be asymptotically stable. Unfortunately,
this conjecture is not true (see Exercise 73).

4.3 Lyapunov Theory

We shall develop the so-called direct method of Lyapunov in relation to the
(initialized) nonlinear autonomous dynamical system given by

x = F (x), x(0) = x0 Rm ; F (0) = 0. (4.7)

Note : Modifications needed to deal with the (nonautonomous) case

x = F (t, x), x(t0 ) = x0

are straightforward.

The aim is to determine the stability nature of the equilibrium state (at
the origin) of system without obtaining the solution x(). This of course
has been done algebraically for linear time invariant systems in section 4.2.
The essential idea is to generalize the concept of energy V for a conservative
system in mechanics, where a well-known result states that an equilibrium
point is stable if the energy is minimum. Thus V is a positive function which
has V negative in the neighborhood of a stable equilibrium point. More
generally,

4.3.1 Definition. We define a Lyapunov function V : Rm R as

follows :
V
V and all its partial derivatives are continuous ;
xi
 C.C. Remsing 127

V is positive definite; that is, V (0) = 0 and V (x) > 0 for x 6= 0 in

some neighborhood {x | kxk k} of the origin.

Consider now the (directional) derivative of V with respect to (the vector

field) F , namely

F1
V h
..
i
V := V
F = x V
xm .
x 1

Fm
V V V
= F1 + F2 + + Fm .
x1 x2 xm
A Lyapunov function V for the system (4.7) is said to be

strong if the derivative V is negative definite; that is, V (0) = 0 and

V (x) < 0 for x 6= 0 such that kxk k.

weak if the derivative V is negative semi-definite; that is, V (0) = 0

and V (x) 0 for all x such that kxk k.

Note : The definitions for positive or negative definiteness or semi-definiteness are

generalizations of those for quadratic forms. Here is the definiteness test for planar
quadratic forms : Suppose that Q = Q(x, y) is the quadratic form ax2 + 2bxy + cy2 ,
where a, b, c R. Then Q is :

positive definite a, c > 0 and b2 < ac ;

positive semi-definite a, c 0 and b2 ac ;

negative definite a, c < 0 and b2 < ac ;

negative semi-definite a, c 0 and b2 ac.

Otherwise, Q is indefinite.

The Lyapunov stability theorems

The statements of the two basic theorems of Lyapunov are remarkably

simple. The proofs will be omitted.
128 AM3.2 - Linear Control

4.3.2 Theorem. (Lyapunovs First Theorem) Suppose that there is a

strong Lyapunov function V for system . Then system is asymptotically
stable.

The conclusion of this theorem is plausible, since the values of the strong
Lyapunov function V (x(t)) must continually diminish along each orbit x =
x(t) as t increases (since V is negative definite). This means that the orbit
x = x(t) must cut across level sets V (x) = C with ever smaller values of C.
In fact, limt V (x(t)) = 0, which implies that x(t) 0 as t since
x(t) and V (x) are continuous and V has value zero only at the origin (thats
where the positive definiteness of V comes into play).

4.3.3 Theorem. (Lyapunovs Second Theorem) Suppose that there is

a weak Lyapunov function V for system . Then system is stable.

The result seems reasonable since the negative semi-definiteness of V keeps

an orbit that starts near the origin close to the origin as t increases. But orbits
dont have to cut across level sets of V .
Note : If the conditions on V in Theorem 4.3.2 hold everywhere in state space
it does not necessarily follow that the origin is asymptotically stable in the large.
For this to be the case V must have the additional property that it is radially
unbounded, which means that

V (x) for all x such that kxk .

For instance,
V = x21 + x22
is radially unbounded, but
x21
V = + x22
1 + x21
is not since, for example,

V 1 as x1 , x2 0.

A similar line of reasoning shows that if is the set of points outside a bounded
region containing the origin, and if throughout , V > 0, V 0 and V is radially
unbounded, then the origin is Lagrange stable.
 C.C. Remsing 129

4.3.4 Example. Consider a unit mass suspended from a fixed support by

a spring, z being the displacement from the equilibrium. If first the spring is
assumed to obey Hookes law, then the equation of motion is

z + kz = 0 (4.8)

where k is the spring constant. Taking x1 : = z, x2 : = z,

(4.8) becomes

x 1 = x2

x = kx .
2 1

Since the system is conservative, the total energy

1 1
E = kx21 + x22
2 2
is a Lyapunov function and it is easy to see that

E = kx1 x2 kx2 x1 = 0

so by Lyapunovs Second Theorem the origin is stable. (Of course, this is

trivial since (4.8) represents simple harmonic motion.)
Suppose now that the force exerted by the spring, instead of being linear
is some function x1 k(x1 ) satisfying

k(0) = 0 , k(x1 ) > 0 for x1 6= 0.

The total energy is now

x1
1
Z
E = x22 + k( ) d
2 0

and
E = kx1 x2 + kx1 x1 = 0.

So again by Lyapunovs Second Theorem the origin is stable for any non-
linear spring satisfying the above conditions.
130 AM3.2 - Linear Control

4.3.5 Example. Consider now the system of the previous example but
with a damping force dz added, so that the equation of motion is

z + dz + kz = 0. (4.9)

(Equation (4.9) can also be used to describe an LCR series circuit, motion of
a gyroscope, and many other problems.)
Assume first that both d and k are constant, and for simplicity let d =
1, k = 2. The system equations in state space form are

x 1 = x2

x = 2x x
2 1 2

and the total energy is

1
E = x21 + x22
2
so that
E = 2x1 x2 + x2 (2x1 x2 ) = x22

which is negative semi-definite, so by Lyapunovs Second Theorem the

origin is stable. However, now consider the function

V = 7x21 + 2x1 x2 + 3x22 .

Then
V = 4x21 4x22 .

Clearly, V is negative definite and it is easy to verify that the quadratic

form V is positive definite, so by Lyapunovs First Theorem the origin is
asymptotically stable (in fact, in the large).

This example illustrates that a suitably-chosen Lyapunov function can pro-

vide more information than the energy function. However, when V is only
negative semi-definite, the following result is often useful.
 C.C. Remsing 131

4.3.6 Proposition. Suppose that there is a (weak) Lyapunov function V

such that V does not vanish identically on any nontrivial trajectories of .
Then (the origin of ) system is asymptotically stable.

4.3.7 Example. Consider again the damped mass-spring system described

by (4.9), but now suppose that both d and k are not constant. Let k(x1 )
be as defined in Example 4.3.4 and let d(x2 ) have the property

d(x2 ) > 0 for x2 6= 0; d(0) = 0.

The state equations are

x 1 = x2

x = x k(x ) x d(x ).
2 1 1 2 2

So, if E is
x1
1
Z
E = x22 + k( ) d
2 0
then
E = x2 (x1 k dx2 ) + x1 kx 1 = x22 d 0.

Now E is positive definite and E vanishes only when x2 (t) 0, which

implies k(x1 ) 0, which in turn implies x1 (t) 0. Thus E vanishes only
on the trivial solution of (4.9), and so by Proposition 4.3.6 the origin is
asymptotically stable.

4.3.8 Example. The van der Pol equation

z +  (z 2 1)z + z = 0 (4.10)

where  is a constant, arises in a number of engineering problems. (In a

control context it can be thought of as application of nonlinear feedback

u = z +  (1 z 2 )z
132 AM3.2 - Linear Control

to the (linear) system described by

z = u.)

We shall assume that  < 0. As usual take x1 : = z, x2 : = z to transform

(4.10) into
x 1 = x2

x = x  (x2 1)x .

2 1 1 2

(The only equilibrium state of this system is the origin.) Try as a potential
Lyapunov function V = x21 + x22 which is obviously positive definite. Then

V = 2x1 x 1 + 2x2 x 2
= 2 x22 (1 x21 ).

Thus V 0 if x21 < 1, and then by Proposition 4.3.6 the origin is

asymptotically stable. It follows that all trajectories starting inside the re-
gion : x21 + x22 < 1 converge to the origin as t , and is therefore
called a region of asymptotic stability.

Note : You may be tempted to think that the infinite strip S : x21 < 1 is a region
of asymptotic stability. This is not in fact true, since a trajectory starting outside
can move inside the strip whilst continuing in the direction of decreasing V circles,
and hence lead to divergence.

In general, if a closed region

R : V (x) constant

is bounded and has V negative throughout, then region R is a region of asymp-

totic stability.

Suppose that we now take as state variables

Z t
x1 : = z, x3 : = z( ) d.
0
 C.C. Remsing 133

The corresponding state equations are

1 3


x 1 = x3  3 x1 x1

x = x .
3 1

Indeed,

x 1 = z
Z t Z t
= z d  (z 2 1)z d
0 0
 
1 3
= x3  z z
3
 
1 3
= x3  x x1 .
3 1

Hence, using V = x21 + x23 , we have

 
1
V = 2x1 x3  x31 +  x1 + 2x3 x1
3
 
1 2
2
= 2 x1 1 x1 0 if x21 < 3
3

so the region of asymptotic stability obtained by this different set of state

variables is R : x21 + x23 < 3, larger than before.

Note : In general, if the origin is an asymptotically stable equilibrium point (state),

then the total set of initial points (states) from which trajectories converge to the
origin as t is called the domain of attraction. Knowledge of this domain
is of great value in practical problems since it enables permissible deviations from
equilibrium to be determined. However, Example 4.3.8 illustrates the fact that since
a particular Lyapunov function gives only sufficient conditions for stability, the region
of asymptotic stability obtained can be expected to be only part of the domain of
attraction. Different Lyapunov functions or different sets of state variables may well
yield different stability regions.
The general problem of finding optimum Lyapunov functions, which give best
possible estimates for the domain of attraction, is a difficult one.
134 AM3.2 - Linear Control

It may be a waste of effort trying to determine the stability properties of

an equilibrium point, since the point may be unstable. The following result is
then useful :

4.3.9 Theorem. (Lyapunovs Third Theorem) Let a function V : Rm

R with V (0) = 0 have continuous first order partial derivatives. If there is
some neighborhood containing the origin in which V takes negative values,
and if in addition V is negative semi-definite, then the origin of (4.7) is not
asymptotically stable. If V is negative definite, the origin is unstable, and if
both V and V are negative definite, the origin is completely unstable.

Note : In all three Lyapunovs theorems the terms positive and negative can
be interchanged simply by using V instead of V . It is only the relative signs of the
Lyapunov function and its derivative which matter.

Application to linear systems

We now return to the linear time-invariant system

x = Ax, x Rm . (4.11)

In section 4.2 we gave algebraic criteria for determining asymptotic stability

via the characteristic equation of A. We now show how Lyapunov theory can
be used to deal directly with (4.11) by taking as a potential Lyapunov function
the quadratic form
V = xT P x (4.12)

where the matrix P Rmm is symmetric. The (directional) derivative of V

with respect to (4.11) (in fact, with respect to the vector field x 7 Ax ) is

V = x T P x + xT P x
= xT AT P x + xT P Ax
= xT Qx
 C.C. Remsing 135

where
AT P + P A = Q (4.13)

and it is easy to see that Q is also symmetric. If P and Q are both positive
definite, then by Lyapunovs First Theorem the (origin of) system (4.11)
is asymptotically stable. If Q is positive definite and P is negative definite or
indefinite, then in both cases V can take negative values in the neighborhood
of the origin so by Lyapunovs Third Theorem, (4.11) is unstable. We
have therefore proved :

4.3.10 Proposition. The matrix A Rmm is a stability matrix if and

only if for any given positive definite symmetric matrix Q, there exists a posi-
tive definite symmetric matrix P that satisfies the Lyapunov matrix equation
(4.13).

Moreover, if the matrix A is a stability matrix, then P is the unique

solution of the Lyapunov matrix equation (see Exercise 77).

It would be no use choosing P to be positive definite and calculating Q

from (4.13). For unless Q turned out to be definite or semi-definite (which
is unlikely) nothing could be inferred about asymptotic stability from the
Lyapunov theorems.

Note : Equations similar in form to (4.13) also arise in other areas of control
theory. However, it must be admitted that since a digital computer will be required
to solve (4.13) except for small values of n, so far as stability determination of (4.11)
is concerned it will be preferable instead to find the eigenvalues of A. The true value
and importance of Proposition 4.3.10 lies in its use as a theoretical tool.

Linearization

The usefulness of linear theory can be extended by use of the idea of

linearization. Suppose the components of (the vector field) F in

x = F (x), x(0) = x0 ; F (0) = 0

136 AM3.2 - Linear Control

are such that we can apply Taylors theorem to obtain

F (x) = Ax + g(x). (4.14)

F 

Here A = DF (0) : = x x=0 Rmm , g(0) = 0 Rm , and the components
of g have power series expansions in x1 , x2 , . . . , xm beginning with terms of
at least second degree. The linear system

x = Ax (4.15)

is called the linearization (or first approximation) of the given (nonlinear)

system (at the origin). We then have:

4.3.11 Theorem. (Lyapunovs Linearization Theorem) If (4.15) is

asymptotically stable or unstable, then the origin for x = F (x), where F
is given by (4.14), has the same stability property.

Proof : Consider the function

V = xT P x

where P satisfies
AT P + P A = Q

Q being an arbitrary positive definite symmetric matrix. If (4.15) is asymp-

totically stable, then by Proposition 4.3.10 P is positive definite. The
derivative of V with respect to (4.14) is

V = xT Qx + 2g T P x .

Because of the nature of g, the term 2g T P x has degree three at least, and so
for x sufficiently close to the origin, V < 0.

Exercise 66 Let a, b > 0 and consider the function

f : R R, t 7 at2 + bt3 .

Show that, for t sufficiently close to the origin (i.e. for |t| < ), we have f(t) < 0.
 C.C. Remsing 137

Hence, by Lyapunovs First Theorem, the origin of (4.14) is asymp-

totically stable.
If (4.15) is unstable, V remains negative definite but P is indefinite, so V
can take negative values and therefore satisfies the conditions of Lyapunovs
Third Theorem for instability. 2

Note : If (4.15) is stable but not asymptotically stable, Lyapunovs Lineariza-

tion Theorem provides no information about the stability of the origin of (4.14),
and other methods must be used.

Furthermore, it is clear that linearization cannot provide any information

about regions of asymptotic stability for nonlinear systems, since if the first
approximation is asymptotically stable, then it is so in the large. Thus the
extent of asymptotic stability for

x = F (x), F (0) = 0

is determined by the nonlinear terms in (4.14).

4.3.12 Example. Consider the differential equation

z + az + bz + g(z, z)
=0

or
x 1 = x2

x = bx ax g(x , x ).
2 1 2 1 2

The linear part of this system is asymptotically stable if and only if a > 0
and b > 0, so if g is any function of x1 and x2 satisfying the conditions of
Theorem 4.3.11, the origin of the system is also asymptotically stable.

4.4 Stability and Control

We now consider some stability problems associated explicitly with the control
variables.
138 AM3.2 - Linear Control

Input-output stability

Our definitions in section 4.1 referred to stability with respect to pertur-

bations from an equilibrium state. When a system is subject to inputs it is
useful to define a new type of stability.

4.4.1 Definition. The control system (with outputs) described by

x = F (t, x, u) , F (t, 0, 0) = 0

y = h(t, x, u)

is said to be bounded input-bounded output stable (b.i.b.o. stable) if

any bounded input produces a bounded output; that is, given

ku(t)k < L1 , t t0

where L1 is any positive constant, then there exists a number L2 > 0 such
that
ky(t)k < L2 , t t0

regardless of initial state x(t0 ). The problem of studying b.i.b.o. stability for
nonlinear systems is a difficult one, but we can give some results for the usual
linear time-invariant system

x = Ax + Bu(t)


(4.16)


y = Cx.

Exercise 67 Let A Rmm be a matrix having m distinct eigenvalues with

negative real parts. Show that (for all t 0)

k exp (tA)k Keat

for some constants K, a > 0. (The result remains valid for any stability matrix, but
the proof is more difficult.)
 C.C. Remsing 139

4.4.2 Proposition. If the linear system

x = Ax

is asymptotically stable, then the system described by (4.16) is b.i.b.o. stable.

Proof : Using
 Z t 
x(t) = exp(tA) x0 + exp( A)Bu( ) d
0

and properties of norms, we have

ky(t)k kCk kx(t)k

Z t
kCk k exp(tA)x0 k + kCk k exp(t )AkkBuk d.
0

If A is a stability matrix, then

k exp (tA)k K eat K , t0

for some positive constants K and a. Thus

 
1 eat
ky(t)k kCk Kkx0 k + L1 KkBk
a
 
L1 KkBk
kCk Kkx0 k + , t0
a

showing that the output is bounded, since kCk and kBk are positive numbers.2

The converse of this result holds if (A, B, C) is a minimal realization. In

other words,

4.4.3 Proposition. If the control system described by (4.16) is c.c. and

c.o. and b.i.b.o. stable, then the linear (dynamical) system

x = Ax

is asymptotically stable.
140 AM3.2 - Linear Control

We shall not give a proof.

Note : For linear time-varying systems Proposition 4.4.2 is not true, unless for
all t the norms of B(t) and C(t) are bounded and the norm of the state transition
matrix (t, t0 ) is bounded and tends to zero as t independently of t0 .

In the definition of complete controllability no restrictions were applied to

u(), but in practical situations there will clearly always be finite bounds on the
magnitudes of the control variables and on the duration of their application. It
is then intuitively obvious that this will imply that not all states are attainable.
As a trivial example, if a finite thrust is applied to a rocket for a finite time,
then there will be a limit to the final velocity which can be achieved. We give
here one formal result for linear systems.

4.4.4 Proposition. If A Rmm is a stability matrix, then the linear

control system given by

x = Ax + Bu(t)

with u() bounded is not completely controllable.

Proof : Let V be a quadratic form Lyapunov function (i.e. V = xT P x,

where P is a symmetric matrix) for the (unforced) system x = Ax. Then
(with respect to ) we have

V = V T = (Ax + Bu)T (V )
= xT AT (V ) + uT B T (V )
= xT Qx + uT B T (V )

where P and Q satisfy (the Lyapunov matrix equation)

AT P + P A = Q

and  T
V V V
V : = ...
x1 x2 xm
 C.C. Remsing 141

is the gradient of V . The term uT B T (V ) is linear in x and since u() is

bounded, it follows that for kxk sufficiently large, V = x T (V ) is negative.
This shows that x()
points into the interior of the region R : V (x) = M for
some M sufficiently large. Hence points outside this region cannot be reached,
so by definition the system is not c.c. 2

Linear feedback

Consider again the linear system (4.16). If the open loop system is unstable
(for instance, by Theorem 4.2.1, if one or more of the eigenvalues of A has
a positive real part), then an essential practical objective would be to apply
control so as to stabilize the system; that is, to make the closed loop system
asymptotically stable.
If (4.16) is c.c., then we saw (Theorem 3.1.3) that stabilization can always
be achieved by linear feedback u = Kx, since there are an infinity of matrices
K which will make A + BK a stability matrix.
If the pair (A, B) is not c.c., then we can define the weaker property that
(A, B) is stabilizable if (and only if) there exists a constant matrix K such
that A + BK is asymptotically stable.

4.4.5 Example. Return to the linear system described by

4 3 5 2

x =
1 2 3 x + 1 u(t)

2 1 8 1

in Example 3.4.4. The eigenvalues of the uncontrollable part are the roots
of the polynomial
p() = 2 7 23

which is not asymptotically stable since it has negative coefficients (Propo-

sition 4.2.6). The system is not stabilizable.

By duality (see Theorem 3.2.4) we define the pair (A, C) to be de-

tectable if (and only if) the pair (AT , C T ) is stabilizable.
142 AM3.2 - Linear Control

4.4.6 Example. Consider the linear control system (with outputs)

x = Ax + bu(t)

y = cx

where " # " #

2 2 1 h i
A= , b= , c= 1 0 .
1 1 1
This system is both stabilizable and detectable : using the feedback matrix K
and the observer matrix L given by
" #
h i 0
K = 2 0 , L=
1

the matrix A + bK then has eigenvalues 3, 2 and the matrix A + Lc has

eigenvalues 2, 1. (Other choices for K and L are also possible.)

Stabilizability and detectability are not guaranteed.

4.4.7 Example. Consider the linear control system (with outputs)

x = Ax + bu(t)

y = cx

where " # " #

0 1 1 h i
A= , b= , c= 0 1 .
0 0 0
In this case, any 1 2 feedback matrix K produces a closed loop matrix
A + bK with zero as an eigenvalue; therefore, the system is not stabilizable.
Also, any 21 matrix L yields a matrix A+Lc with zero as an eigenvalue;
therefore, the system is not detectable.

The following simple test for stabilizability holds :

 C.C. Remsing 143

4.4.8 Proposition. Suppose A Rmm and B Rm` . The pair (A, B)

is stabilizable if and only if
h i
rank sIm A B = m

for all s Ce+ : = {s | Re (s) 0}.

The proof is not difficult and will be omitted.

4.4.9 Corollary. The pair (A, B) is stabilizable if and only if

h i
rank Im A B = m

for all eigenvalues Ce+ of A.

Note : It can be proved that the pair (A, B) is c.c. if and only if
h i
rank Im A B = m

for all eigenvalues of A. (The eigenvalues at which the rank drops below m
are the so-called uncontrollable modes.) Clearly, if the pair (A, B) is c.c., then it is
stabilizable.

By duality, the following (test for detectability) is now immediate.

4.4.10 Proposition. Suppose A Rmm and C R`m . The pair

(A, C) is detectable if and only if
" #
sIm A
rank =m
C

for all s Ce+ : = {s | Re (s) 0}.

4.4.11 Corollary. The pair (A, C) is detectable if and only if

" #
Im A
rank =m
C

for all eigenvalues Ce+ of A.

144 AM3.2 - Linear Control

Note : By duality again, the pair (A, C) is c.o. if and only if

" #
Im A
rank =m
C

for all eigenvalues of A. (The eigenvalues at which the rank drops below m
are the so-called unobservable modes.) Clearly, if the pair (A, C) is c.o., then it is
detectable.

Let A Rmm , B Rm` , and C Rnm . Recall that if A is a stability

(or Hurwitz) matrix, then the usual (time-invariant) linear control system with
outputs

x = Ax + Bu
y = Cx

is b.i.b.o. stable (see Proposition 4.4.2). As we know, the converse is not

true,
" in#general.
" #(For example, the usual linear control system with A =
2 0 1 h i
,b= , and c = 0 1 is b.i.b.o. stable but not asymptotically
0 1 1
stable.)

The following interesting result is given without a proof.

4.4.12 Proposition. Suppose a usual linear control system is stabi-

lizable and detectable. is asymptotically stable if and only if it is b.i.b.o.
stable.

Application

It is interesting to consider here a simple application of the Lyapunov meth-

ods. First notice that if the linear system

x = Ax

is asymptotically stable with Lyapunov function V = xT P x, where P satisfies

AT P + P A = Q
 C.C. Remsing 145

then
V xT Qx
= T
V x Px
xT Qx
where is the minimum value of the ratio xT P x
(in fact, this is equal to the
smallest eigenvalue of QP 1 ). Integrating with respect to t gives

V (x(t)) et V (x(0)).

Since V (x(t)) 0 as t , this can be regarded as a measure of the way

in which trajectories approach the origin, so the larger the faster does
x(t) 0.
Suppose now we apply the control

u = (S Q1 )B T P x (4.17)

to (4.16), where P is the solution of

AT P + P A = Q

and S and Q1 are arbitrary skew-symmetric and positive definite symmetric

matrices, respectively. The closed loop system is thus

x = A + B(S Q1 )B T P x


(4.18)

and it is easy to verify that if V = xT P x, then the (directional) derivative

with respect to (4.18) is

V = xT Qx 2xT P BQ1 B T P x < xT Qx

since P BQ1 B T P = (P B)Q1 (P B)T is positive definite. Hence, by the argu-

ment just developed, it follows that (4.18) is more stable than the open loop
system
x = Ax

in the sense that trajectories will approach the origin more quickly.
146 AM3.2 - Linear Control

Note : (4.17) is of rather limited practical value because it requires asymptotic

stability of the open loop system, but nevertheless the power of Lyapunov theory is
apparent by the ease with which the asymptotic stability of (4.18) can be established.
This would be impossible using the classical methods requiring the calculation of the
characteristic equation of (4.18). Furthermore, the Lyapunov approach often enables
extensions to nonlinear problems to be made.

4.5 Exercises
Exercise 68 Determine the equilibrium point (other than the origin) of the system
described by
x 1 = x1 2x1 x2

x = 2x + x x .
2 2 1 2

Apply a transformation of coordinates which moves this point to the origin, and
find the new system equations. (The equations are an example of a predator-pray
population model due to Vito Volterra (1860-1940) and used in biology, and are
more general than the simple linear rabbit-fox model.)

Exercise 69 Determine whether the following polynomials are asymptotically sta-

ble :
(a) 3 + 172 + 2 + 1.
(b) 4 + 3 + 42 + 4 + 3.
(c) 4 + 63 + 22 + + 3.

Exercise 70 Determine for what range of values of k the polynomial

(3 k)3 + 22 + (5 2k) + 2

is asymptotically stable.

Exercise 71 Determine for what range of values of k R the linear (dynamical)

system
0 1 0

x =
0 0 x
1
k 1 2
 C.C. Remsing 147

is asymptotically stable. If k = 1, and a control term

0

1 u(t)

0
is added, find a linear feedback control which makes all the eigenvalues of the closed
loop system equal to 1.

Exercise 72 Verify that the eigenvalues of

" #
4 3e8t
A(t) =
e8t 0
are both constant and negative, but the solution of the linear time-varying system

x = A(t)x

diverges as t .

Exercise 73 Write the system

z + z + z 3 = 0

in state space form. Let

V = ax41 + bx21 + cx1 x2 + dx22

and choose the constants a, b, c, d such that

V = x41 x22 .

Hence investigate the stability nature of the equilibrium point at the origin.

Exercise 74 Using the function

V = 5x21 + 2x1 x2 + 2x22

show that the origin of

x 1 = x2

x = x x + (x + 2x ) x2 1


2 1 2 1 2 2

is asymptotically stable by considering the region |x2 | < 1. State the region of
asymptotic stability thus determined.
148 AM3.2 - Linear Control

Exercise 75 Investigate the stability nature of the origin of the system

2 2
x 1 = x1 x2

x = 2x x
2 1 2

using the function

V = 3x1 x22 x31 .

Exercise 76 Given the matrix

" #
1 1
A=
2 4

and taking " #

p1 p2
P =
p2 p3
solve the equation
AT P + P A = I2 .

Hence determine the stability nature of A.

Exercise 77 Integrate both sides of the matrix ODE

(t) = AW (t) + W (t)B ,

W W (0) = C

with respect to t from t = 0 to t = . Hence deduce that if A is a stability matrix,

the solution of the equation
AT P + P A = Q

can be written as Z
P = exp(tAT )Q exp(tA) dt.
0

Exercise 78 Convert the second order ODE

z + a1 z + a2 z = 0

into the state space form. Using V = xT P x with V = x22 , obtain the necessary
and sufficient conditions a1 > 0, a2 > 0 for asymptotic stability.
 C.C. Remsing 149

Exercise 79 By using the quadratic Lyapunov function V = V (x1 , x2 ) which has

derivative 2(x21 + x22 ), determine the stability of the system

x 1 = kx1 3x2

x = kx 2x

2 1 2

when k = 1. Using the same function V , obtain sufficient conditions on k for the
system to be asymptotically stable.

Exercise 80 Investigate the stability nature of the equilibrium state at the origin
for the system
4
x 1 = 7x1 + 2 sin x2 x2

x = ex1 3x 1 + 5x2 .

2 2 1

Exercise 81 Investigate the stability nature of the origin.

(a)
x 1 = x1 + x2

x = (x + x ) sin x 3x .
2 1 2 1 2

(b)
3
x 1 = x1 + x2

x = ax bx ;

a, b > 0.
2 1 2

Exercise 82 Show that the system described by

0 1 0 0

x = 1 0 x + 0 u(t)
0



0 0 1 1







y=x

is stable i.s.L. and b.i.b.o. stable, but not asymptotically stable. [It is easy to verify
that this system is not completely controllable.]
150 AM3.2 - Linear Control

Exercise 83 Consider the system described by the ODE

1
x = x + u(t), x(0) = x0 R.
t+3
Show that if u(t) 0, t 0 then the origin is asymptotically stable, but that if u()
is the unit step function, then lim x(t) = .
t

Exercise 84 A system is described by the equations

x 1 = x2 x1

x = x x x2 .

2 1 2 1

Determine the equilibrium state which is not at the origin. Transform the system
equations so that this point is transferred to the origin, and hence verify that this
equilibrium state is unstable.

Exercise 85 Determine for what range of values of the real parameter k the linear
system
0 1 0

x =
0 0 1 x

5 k k6
is asymptotically stable.

Exercise 86 A particle moves in the xy-plane so that its position in any time t is
given by
x + y + 3x = 0, y + x + 3y = 0.

Determine the stability nature of the system if (a) = 4 and (b) = 16.

Exercise 87 Use the Lyapunov function

V = 2x21 + x22

to find a region of asymptotic stability for the origin of the system

3
x 1 = x1 3x1 + x2

x = 2x .
2 1
 C.C. Remsing 151

Exercise 88 Use Lyapunovs Linearization Theorem to show that the origin

for the system
x 1 = 3x2

x = x 2x3 x


2 1 2 2

is asymptotically stable provided the real parameter is negative. Use the Lyapunov
function
1 2
x1 + 3x22


V =
2
to determine a region of asymptotic stability about the origin.

Exercise 89 A system has state equations

x 1 = x1 + x2 x3 + x31






x 2 = x1 x2







x 3 = x1 x2 x3 + u(t).

(a) If = 0 show that the system is b.i.b.o. stable for any output which is
linear in the state variables, provided > 1.
(b) If = = 1 and u() = 0, investigate the stability nature of the equi-
librium state at the origin by using the Lyapunov function

V = x21 + x22 + x23 .

Exercise 90 Investigate for stabilizability and detectability the following control

systems.
(a)
" # " #
2 0 1
x = x+ u(t)
0 1 1
h i
y = 0 1 x.

(b)
" # " #
0 0 0
x = x+ u(t)
0 1 1
h i
y = 1 1 x.