Systems & Control Letters 40 (2000) 77–82
www.elsevier.com/locate/sysconle
On the in nite-horizon LQ tracker
Enrique Barbieri ∗ , Rocio Alba-Flores
Department of Electrical Engineering and Computer Science, Tulane University, New Orleans, LA 70118, USA
Received 20 March 1999; received in revised form 31 January 2000
Abstract
The in nite-horizon tracking problem is considered in the linear-quadratic optimal control framework. Computationally,
one term in the control signal is a constant gain, stabilizing, full-state feedback found by solving an algebraic Riccati equation;
the second term involves a steady-state function vss (t) that solves an auxiliary, forced di erential equation with unknown
initial condition. In this article, a linear system of algebraic equations is derived to determine vss (0). Numerical examples
are included to illustrate the result. c 2000 Elsevier Science B.V. All rights reserved.
Keywords: Optimal; Tracking; Linear quadratic
1. Introduction and background
The development of the theory of optimal linearquadratic regulators, tracking, and servo controllers
has become standard textbook material [1,2,5 –7]. The
steady-state or in nite-horizon regulator for linear,
time-invariant (LTI) systems is particularly appealing and always used in practice since its solution, a
constant gain, stabilizing, full-state feedback control,
is obtained by solving the algebraic Riccati equation
(ARE). The steady-state tracking controller, on the
other hand, has received much less attention mainly
because for most reference signals the cost of tracking
becomes unbounded and there are additional computational diculties that are not present in the regulator.
In general, one considers the linear, time-varying
system in state-space form
ẋ(t) = A(t)x(t) + B(t)u(t);
x(t0 ) = x0 given;
(1)
where x ∈ Rn and u ∈ Rm are the state and control
vectors, respectively, and the matrices A and B are appropriately dimensioned. In the classical nite horizon
linear-quadratic tracking (LQT) problem, one seeks a
control law u(t) that forces the state x(t) to track a
desired reference trajectory xr (t) over a speci ed time
interval [t0 ; T ] while minimizing the nite-time cost
functional
J (u) = 21 [(x(T ) − xr (T ))′ Qf (x(T ) − xr (T )]
+ 12
Z
T
[(x − xr )′ Q(x − xr ) + u′ Ru] dt:
where Qf ¿0; Q¿0, and R ¿ 0 are real symmetric
matrices. Throughout, we let (·)′ denote the transpose
of the indicated quantity.
The optimal LQT control law consists of the sum
of two components:
u(t) = u∗ (t) = −K(t)x(t) + R−1 (t)B′ (t)v(t):
∗ Corresponding author.
E-mail addresses: barbieri@eecs.tulane.edu (E. Barbieri);
alba@eecs.tulane.edu (R. Alba-Flores).
(2)
t0
(3)
The rst term is a full-state feedback with Kalman gain
K(t) = R−1 (t)B′ (t)P(t);
0167-6911/00/$ - see front matter c 2000 Elsevier Science B.V. All rights reserved.
PII: S 0 1 6 7 - 6 9 1 1 ( 0 0 ) 0 0 0 0 4 - 9
(4)
78
E. Barbieri, R. Alba-Flores / Systems & Control Letters 40 (2000) 77–82
where P(t) satis es the matrix di erential Riccati
equation
− Ṗ(t) = A(t)′ P(t) + P(t)A(t)
−P(t)B(t)R−1 (t)B′ (t)P(t) + Q(t);
(5)
P(T ) = Qf :
The second term uses the auxiliary function v(t)
which is found from the solution of the following
di erential equation:
2. The steady-state LQT problem (SS LQT)
We consider the time-invariant version of the optimal tracking problem introduced in the previous section. That is, all system matrices and cost functional
parameters are constant. In addition, Qf = 0; t0 =
0, and T → ∞. Assuming that {A; B} is controllable
(stabilizable) and {A; C} (Q = C ′ C) is observable
(detectable), the solution of the Riccati equation (5)
reaches the steady-state value Pss that solves the algebraic Riccati equation (ARE)
0 = A′ Pss + Pss A − Pss BR−1 B′ Pss + Q;
(7)
′
−v̇(t) = [A(t) − B(t)K(t)] v(t) + Qxr (t);
v(T ) = Qf xr (T ):
(6)
Note that in both Eqs. (5) and (6), the boundary
condition is at the end-point T . The implementation of
the optimal tracker (3) in real time involves an optimal feedback regulator and a feed-forward controller.
Since the solution of P(t) is independent of the reference trajectory, the Riccati equation is solved o -line
and the feedback gain K(t) is stored. In many tracking problems, the reference xr (t) is known only over
the look-ahead interval [t1 ; t1 + ] where t1 ∈ [t0 ; T ].
The value of the auxiliary function v(t1 ), needed to
compute the control at t1 , is therefore obtained by integrating (6) backward-in-time in the interval [t1 ; t1 +]
with boundary condition v(t1 + ) = Qf xr (t1 + ).
In this note, we take a new look at the steady-state
linear-quadratic tracking problem (SS LQT). It is a
well-known fact that the ∞-horizon problem does
not have a solution in the strict sense because in
general the cost is unbounded. However, for applications where the reference signal is asymptotically
stable, the solution is seen to have a bounded cost.
In other cases where the control window T or the
look-ahead-window is large, the design framework
may still provide a suitable, implementable controller. Our focus here is on nding the unknown
initial condition of the steady-state auxiliary function
vss (t) which is required to implement the control at
t = 0. The result is given by a linear system of algebraic equations that avoids having to integrate the
associated di erential equation backward-in-time.
The remainder of the paper is organized as follows:
in Section 2, we de ne the SS LQT problem; in Section 3, we derive the main results; in Section 4 we
present two numerical examples; and in Section 5, we
summarize the article and provide conclusions.
and consequently, the Kalman gain also reaches the
steady-state value Kss = R−1 B′ Pss .
Let us write the solution u(t) based on (3) as follows:
u(t) = −Kss x(t) + R−1 B′ vss (t);
(8)
where the function
△
vss (t) = v(t)|T →∞
is the limiting solution of (6), that is [5]
−v̇ss (t) = [A − BKss ]′ vss (t) + Qxr (t);
vss (0) = unknown:
(9)
It is known that vss (t) is nite for any nite value of
t and moreover, it is bounded for bounded xr (t). Note
that the full-state feedback term in (8) is stabilizing,
that is, the spectrum of the closed-loop system matrix
[A − BKss ] lies in the open left-half of the complex
plane. Therefore, it follows that the control signal (8)
also renders the system state x(t) nite for any nite t
and bounded for bounded xr . Finally, as T → ∞, the
cost functional is in general unbounded and the meaning of minimality is lost, that is, strictly speaking, the
optimal control u(t) = u∗ (t) does not exist. This issue
is removed when the reference signal is asymptotically
stable.
3. The initial condition of Css (t)
In this section, we obtain a linear system of algebraic equations for vss (0) as an alternative to integrating (9) backwards in time for a large value of T . In
the examples, we use the integration approach to verify our result. Consider the Hamiltonian and co-state
E. Barbieri, R. Alba-Flores / Systems & Control Letters 40 (2000) 77–82
The examination of the ith component of X (s) given
equations given by
H=
1
2 [(x
′
− xr ) Q(x − xr )] +
79
1 ′
2 u Ru
′
+ [Ax + Bu]
(10)
by
Xi (s) =
r (s)N0i (s) + Nri (s)
n (s)n (−s)r (s)
leads to the following fact:
and
@H
= A′ (t) + Qx(t) − Qxr (t);
@x
respectively. The stationary condition
˙ =
− (t)
@H
= Ru(t) + B′ (t)
@u
leads to the control
(11)
0=
(12)
u(t) = −R−1 B′ (t)
(13)
and the state equation
ẋ(t) = Ax(t) − BR−1 B′ (t):
(14)
Combining Eq. (14) and (11), one obtains
"
# "
#"
# " #
ẋ(t)
x(t)
0
A
−BR−1 B′
+
xr (t)
=
′
˙
(t)
Q
−Q
−A
(t)
"
# " #
x(t)
0
(15)
=M
+
xr (t);
(t)
Q
where M is the Hamiltonian matrix.
Using the Laplace transform, system (15) can be
written as
#"
# "
#
"
X (s)
x0
(sI − A)
BR−1 B′
=
;
0 + QXr (s)
Q
(sI + A′ ) (s)
(16)
where X (s); (s), and Xr (s) are the Laplace transforms of x; , and xr , respectively, and 0 is the unknown vector of the co-states at t = 0. Solving for
X (s) we obtain
X (s) = M −1 x0 − M −1 L0 − M −1 LQXr (s)
1
Nr (s);
n (s)n (−s)r (s)
Proof. The result follows because the control signal
(8) renders the state vector x(t) nite for any nite
t and bounded for bounded xr (t). Therefore, the unstable roots i of the polynomial n (−s) must be cancelled by zeros in the numerator polynomial of the
expansion for Xi (s).
Let [(·)i ]′ denote the ith row of the indicated expression. Then, we prove the following main result:
Theorem 1. Consider the steady-state; linear-quadratic
tracking (SS LQT ) problem with solution
u(t) = −R−1 B′ Pss x(t) + R−1 B′ vss (t):
Select any row [(M −1 (s)L(s))i ]′ and [(M −1 (s)L(s)Q)i ]′
where
M (s) = (sI − A) − LQ
and
L(s) = BR−1 B′ (sI + A′ )−1 :
Denote by [ 1 ; 2 ; : : : ; p ] the (unstable) eigenvalues
of [ − A + BKss ]; where k has multiplicity nk ¿1 and
n1 + n2 + · · · + np = n. Then; the initial value of the
steady-state auxiliary function vss (0) is given by
vss (0) = −
1
N0 (s)
=
n (s)n (−s)
+
Proposition 1. Let Xi (s) denote the Laplace transform of the ith component xi (t) of the state vector
(i = 1; : : : ; n). Then; the numerator polynomial in the
expansion for Xi (s) must be such that the polynomial
n (−s) is cancelled exactly.
−1
1
2
(18)
and the solution to the algebraic Riccati equation (7)
is given by
(17)
where M = (sI − A) − LQ; L = BR−1 B′ (sI + A′ )−1 ;
N0 (s) and Nr (s) are vectors with polynomial entries,
r (s) is a polynomial whose roots depend on the type
of reference signal, and n (s)n (−s) is the characteristic polynomial of M, with roots consisting of the n
stable eigenvalues i of the closed-loop system matrix
[A − BKss ], and the n unstable eigenvalues i = −i .
Pss =
−1
1
0;
(19)
where, for j = 0; 1; : : : ; nk − 1; 1 is an n × n matrix
whose rows are
dj
[(M −1 (s)L(s))i ]′ |s= k ;
ds j
2 is an n × 1 vector whose entries are
dj
[(M −1 (s)L(s)Q)i ]′ Xr (s) |s= k ;
ds j
80
E. Barbieri, R. Alba-Flores / Systems & Control Letters 40 (2000) 77–82
and
0
is an n × n matrix whose rows are
j
d
[(M −1 (s))i ]′ |s= k ;
ds j
j = 0; 1; : : : ; nk − 1:
Proof. Consider the case where k ; k = 1; : : : ; n, are
distinct, that is, nk = 1; k = 1; : : : ; n. Then, from Proposition 1 and Eq. (17), we set up the n equations
{[(M −1 )i ]′ x0 − [(M −1 L)i ]′ 0 − [(M −1 LQ)i ]′ Xr (s)}
=[s −
k]
nk
=0
at s =
(20)
k
[1 s s2 · · · sn−1 ]′ . Therefore, from
we nd that
1 0 0 ··· 0
0 s 0 ··· 0
1
0 0 s2 · · · 0
X (s) =
det(M )
.. .. .. .. ..
. . . . .
0 · · · · · · · · · sn−1
×[0 − QXr (s)]:
or, equivalently,
0 x0
+
1 0
+
2
= 0:
(21)
It is clear that when k is repeated, then the additional
nk −1 equations are generated by taking nk −1 derivatives of (20). Solving for 0 from (21), we observe
that 0 can be written as the sum of two components,
one that depends on x0 and the other that depends on
the reference signal Xr . That is,
0 = −
−1
1
0 x0
−
−1
1
(22)
2:
Substituting (22) into (13) at t = 0 we get
u(0) = −R−1 B′
−1
1
0 x0
− R−1 B′
−1
1
2
(17) with x0 = 0,
′
(L )n (s)
′
(L )n (s)
..
.
(L′ )n (s)
Finally, consider the expansion for Xi (s) given by
Xi (s) =
1
si−1 (L′ )n (s)[0 − QXr (s)]
det(M )
which, when evaluated at
k,
results in
si−1 (L′ )n (s)[0 − QXr (s)]|s= k = [s −
k]
nk
:
(25)
Then, the n equations are generated by substituting
s = k into (25) and its nk − 1 derivatives. In each
such equation, we can see that the factor introduced
by si−1 can be divided out, and therefore M −1 does
not play a role in the computation of 1 and 2 .
(23)
Refs. [3,4] use Theorem 1 to construct matrices 1
and 0 in terms of easy-to-generate polynomials for
continuous- and discrete-time single-input systems in
phase-canonic form.
and comparing (23) with (8) at t = 0
u(0) = −R−1 B′ Pss x0 + R−1 B′ vss (0);
the result follows.
A special case is worth noting where M −1 (s) is not
required to compute vss (0).
Corollary 1. Consider the scalar control case and
the pair {A; b} in the standard phase canonic (controllable) form. Then
vss (0) = −
where
1
−1
1
2
(24)
is an n × n matrix whose rows are
dj
[(L(s))n ]′ |s=
ds j
and
2;
k
is an n × 1 vector whose entries are
dj
[(L(s)Q)n ]′ Xr (s) |s= k :
ds j
Proof. Since {A; b} is in phase-canonic form, then
the rst n − 1 rows of matrix L(s) are identically zero.
It can also be veri ed that the nth column of M −1 is
4. Examples
The change of variables = T − t in Eq. (9) leads
to the forward di erential equation
dvss ()
= [A − BKss ]′ vss () + Qxr (T − );
d
with initial condition vss ( = 0) = Qf xr (T ). Then,
vss (t = 0) is obtained from the graph of vss () at = T .
Given the two-input, second-order system and
quadratic-cost matrices
"
#
"
#
"
#
−1 2
1 2
2 1
A=
; B=
; Q=
;
4 3
0 1
1 2
"
#
2 0
R=
;
0 1
the unstable eigenvalues of the Hamiltonian matrix are
= [5:7669
2:7826]′ :
81
E. Barbieri, R. Alba-Flores / Systems & Control Letters 40 (2000) 77–82
Fig. 1. Example 1: Function v() in the interval ∈ [0; 10].
The polynomial matrices L(s) and M −1 (s) are given
by
"
#
0:5(19 + 9s) 2(−10 + s)
1
;
L(s) = 2
s + 2s − 11
2(2 + s)
−9 + s
M
−1
×
2
1
= 4
2s − 82s2 + 515
"
4s2 +25s−105
2(4s2 +13s−45)
2(s3 +3s2 −20s−10)
#
and, for a constant reference signal, Xr (s) =
[2=s 2=s]′ we nd
M −1 (s)L(s)QXr (s)
vss (0) = [0:8377
and
2
122:752]′
6(6s2 + 48s − 95)]′ :
Selecting the rst row of
M −1 (s)L(s)QXr (s)
0:1565]
is found and veri ed in Fig. 1.
Example 2. Consider a third-order integrator in phasecanonic form. Letting R = 1 and Q = diag(4; 0; 0), the
unstable eigenvalues of the Hamiltonian matrix are
= [0:63 + j1:09
1
=
s(2s4 − 82s2 + 515)
M −1 (s)L(s) and
= [ − 18713:8
1
from which the initial condition
2(s3 −s2 −21s+47)
×[6(13s2 − 48s + 42)
and evaluating at the unstable eigenvalues
we compute
"
#
−23339:4
5352:61
;
1=
167:125
−110:209
0:63 − j1:09
The polynomial matrix L(s) is
0
0
0
0
0
L(s) = 0
;
1
1
1
−
s3
s2
s
1:26]′ :
82
E. Barbieri, R. Alba-Flores / Systems & Control Letters 40 (2000) 77–82
Fig. 2. Example 2: Function v() in the interval ∈ [0; 40].
and, for a reference xr (t) = [0:5t 2 t 1]′ , we take the
third row of L(s) and L(s)QXr (s) from which 1 and
2 are found to be
1
1
1
1
−
2
16
13
1
1
1
1
1
1
− 2
;
2 = −4 6
1= 3
2
2
2
2
1
1
1
1
−
2
3
6
3
and vss (0) = [2
Fig. 2.
3
3:17
3
3
′
2:52] . These are veri ed in
5. Conclusions
A linear system of algebraic equations were derived to calculate the initial condition of the auxiliary function in the in nite-horizon linear-quadratic
tracking controller. At least for systems of moderate
dimensions, the algebraic equations are an alternative to having to integrate the associated di erential
equation backwards in time for a (possibly) large interval [T; 0]. A topic for further research (suggested
by one reviewer) is to examine error bounds in the
cost or a measure in the quality of tracking caused by
using vss (0) rather than v(0) in problems where T is
nite but large.
References
[1] B.D. Anderson, J.B. Moore, Optimal Control. Linear Quadratic
Methods, Prentice-Hall, Englewood Cli s, NJ, 1990.
[2] M. Athans, P.L. Falb, Optimal Control, An Introduction to the
Theory and its Applications, McGraw-Hill, New York, 1966.
[3] E. Barbieri, On the LQR problem for phase-canonic systems,
Proceedings of the American Control Conference, San
Francisco, CA, June 1993, pp. 3174 –3175.
[4] E. Barbieri, A solution to the discrete-time ARE, Proceedings
of the American Control Conference, Baltimore, MD, June
1994, pp. 1153–1154.
[5] A.E. Bryson, T. Ho, Applied Optimal Control, Hemisphere
Publishing, Washington, DC, 1975.
[6] H. Kwakernaak, R. Sivan, Linear Optimal Control Systems,
Wiley-Interscience, New York, 1972.
[7] F.L. Lewis, Optimal Control, Wiley, New York, l986.