Systems & Control Letters 40 (2000) 77–82 www.elsevier.com/locate/sysconle On the in nite-horizon LQ tracker Enrique Barbieri ∗ , Rocio Alba-Flores Department of Electrical Engineering and Computer Science, Tulane University, New Orleans, LA 70118, USA Received 20 March 1999; received in revised form 31 January 2000 Abstract The in nite-horizon tracking problem is considered in the linear-quadratic optimal control framework. Computationally, one term in the control signal is a constant gain, stabilizing, full-state feedback found by solving an algebraic Riccati equation; the second term involves a steady-state function vss (t) that solves an auxiliary, forced di erential equation with unknown initial condition. In this article, a linear system of algebraic equations is derived to determine vss (0). Numerical examples are included to illustrate the result. c 2000 Elsevier Science B.V. All rights reserved. Keywords: Optimal; Tracking; Linear quadratic 1. Introduction and background The development of the theory of optimal linearquadratic regulators, tracking, and servo controllers has become standard textbook material [1,2,5 –7]. The steady-state or in nite-horizon regulator for linear, time-invariant (LTI) systems is particularly appealing and always used in practice since its solution, a constant gain, stabilizing, full-state feedback control, is obtained by solving the algebraic Riccati equation (ARE). The steady-state tracking controller, on the other hand, has received much less attention mainly because for most reference signals the cost of tracking becomes unbounded and there are additional computational diculties that are not present in the regulator. In general, one considers the linear, time-varying system in state-space form ẋ(t) = A(t)x(t) + B(t)u(t); x(t0 ) = x0 given; (1) where x ∈ Rn and u ∈ Rm are the state and control vectors, respectively, and the matrices A and B are appropriately dimensioned. In the classical nite horizon linear-quadratic tracking (LQT) problem, one seeks a control law u(t) that forces the state x(t) to track a desired reference trajectory xr (t) over a speci ed time interval [t0 ; T ] while minimizing the nite-time cost functional J (u) = 21 [(x(T ) − xr (T ))′ Qf (x(T ) − xr (T )] + 12 Z T [(x − xr )′ Q(x − xr ) + u′ Ru] dt: where Qf ¿0; Q¿0, and R ¿ 0 are real symmetric matrices. Throughout, we let (·)′ denote the transpose of the indicated quantity. The optimal LQT control law consists of the sum of two components: u(t) = u∗ (t) = −K(t)x(t) + R−1 (t)B′ (t)v(t): ∗ Corresponding author. E-mail addresses: barbieri@eecs.tulane.edu (E. Barbieri); alba@eecs.tulane.edu (R. Alba-Flores). (2) t0 (3) The rst term is a full-state feedback with Kalman gain K(t) = R−1 (t)B′ (t)P(t); 0167-6911/00/$ - see front matter c 2000 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 - 6 9 1 1 ( 0 0 ) 0 0 0 0 4 - 9 (4) 78 E. Barbieri, R. Alba-Flores / Systems & Control Letters 40 (2000) 77–82 where P(t) satis es the matrix di erential Riccati equation − Ṗ(t) = A(t)′ P(t) + P(t)A(t) −P(t)B(t)R−1 (t)B′ (t)P(t) + Q(t); (5) P(T ) = Qf : The second term uses the auxiliary function v(t) which is found from the solution of the following di erential equation: 2. The steady-state LQT problem (SS LQT) We consider the time-invariant version of the optimal tracking problem introduced in the previous section. That is, all system matrices and cost functional parameters are constant. In addition, Qf = 0; t0 = 0, and T → ∞. Assuming that {A; B} is controllable (stabilizable) and {A; C} (Q = C ′ C) is observable (detectable), the solution of the Riccati equation (5) reaches the steady-state value Pss that solves the algebraic Riccati equation (ARE) 0 = A′ Pss + Pss A − Pss BR−1 B′ Pss + Q; (7) ′ −v̇(t) = [A(t) − B(t)K(t)] v(t) + Qxr (t); v(T ) = Qf xr (T ): (6) Note that in both Eqs. (5) and (6), the boundary condition is at the end-point T . The implementation of the optimal tracker (3) in real time involves an optimal feedback regulator and a feed-forward controller. Since the solution of P(t) is independent of the reference trajectory, the Riccati equation is solved o -line and the feedback gain K(t) is stored. In many tracking problems, the reference xr (t) is known only over the look-ahead interval [t1 ; t1 +
] where t1 ∈ [t0 ; T ]. The value of the auxiliary function v(t1 ), needed to compute the control at t1 , is therefore obtained by integrating (6) backward-in-time in the interval [t1 ; t1 +
] with boundary condition v(t1 +
) = Qf xr (t1 +
). In this note, we take a new look at the steady-state linear-quadratic tracking problem (SS LQT). It is a well-known fact that the ∞-horizon problem does not have a solution in the strict sense because in general the cost is unbounded. However, for applications where the reference signal is asymptotically stable, the solution is seen to have a bounded cost. In other cases where the control window T or the look-ahead-window is large, the design framework may still provide a suitable, implementable controller. Our focus here is on nding the unknown initial condition of the steady-state auxiliary function vss (t) which is required to implement the control at t = 0. The result is given by a linear system of algebraic equations that avoids having to integrate the associated di erential equation backward-in-time. The remainder of the paper is organized as follows: in Section 2, we de ne the SS LQT problem; in Section 3, we derive the main results; in Section 4 we present two numerical examples; and in Section 5, we summarize the article and provide conclusions. and consequently, the Kalman gain also reaches the steady-state value Kss = R−1 B′ Pss . Let us write the solution u(t) based on (3) as follows: u(t) = −Kss x(t) + R−1 B′ vss (t); (8) where the function △ vss (t) = v(t)|T →∞ is the limiting solution of (6), that is [5] −v̇ss (t) = [A − BKss ]′ vss (t) + Qxr (t); vss (0) = unknown: (9) It is known that vss (t) is nite for any nite value of t and moreover, it is bounded for bounded xr (t). Note that the full-state feedback term in (8) is stabilizing, that is, the spectrum of the closed-loop system matrix [A − BKss ] lies in the open left-half of the complex plane. Therefore, it follows that the control signal (8) also renders the system state x(t) nite for any nite t and bounded for bounded xr . Finally, as T → ∞, the cost functional is in general unbounded and the meaning of minimality is lost, that is, strictly speaking, the optimal control u(t) = u∗ (t) does not exist. This issue is removed when the reference signal is asymptotically stable. 3. The initial condition of Css (t) In this section, we obtain a linear system of algebraic equations for vss (0) as an alternative to integrating (9) backwards in time for a large value of T . In the examples, we use the integration approach to verify our result. Consider the Hamiltonian and co-state E. Barbieri, R. Alba-Flores / Systems & Control Letters 40 (2000) 77–82 The examination of the ith component of X (s) given equations given by H= 1 2 [(x ′ − xr ) Q(x − xr )] + 79 1 ′ 2 u Ru ′ +  [Ax + Bu] (10) by Xi (s) =
r (s)N0i (s) + Nri (s)
n (s)
n (−s)
r (s) leads to the following fact: and @H = A′ (t) + Qx(t) − Qxr (t); @x respectively. The stationary condition ˙ = − (t) @H = Ru(t) + B′ (t) @u leads to the control (11) 0= (12) u(t) = −R−1 B′ (t) (13) and the state equation ẋ(t) = Ax(t) − BR−1 B′ (t): (14) Combining Eq. (14) and (11), one obtains " # " #" # " # ẋ(t) x(t) 0 A −BR−1 B′ + xr (t) = ′ ˙ (t) Q −Q −A (t) " # " # x(t) 0 (15) =M + xr (t); (t) Q where M is the Hamiltonian matrix. Using the Laplace transform, system (15) can be written as #" # " # " X (s) x0 (sI − A) BR−1 B′ = ; 0 + QXr (s) Q (sI + A′ ) (s) (16) where X (s); (s), and Xr (s) are the Laplace transforms of x; , and xr , respectively, and 0 is the unknown vector of the co-states at t = 0. Solving for X (s) we obtain X (s) = M −1 x0 − M −1 L0 − M −1 LQXr (s) 1 Nr (s);
n (s)
n (−s)
r (s) Proof. The result follows because the control signal (8) renders the state vector x(t) nite for any nite t and bounded for bounded xr (t). Therefore, the unstable roots i of the polynomial
n (−s) must be cancelled by zeros in the numerator polynomial of the expansion for Xi (s). Let [(·)i ]′ denote the ith row of the indicated expression. Then, we prove the following main result: Theorem 1. Consider the steady-state; linear-quadratic tracking (SS LQT ) problem with solution u(t) = −R−1 B′ Pss x(t) + R−1 B′ vss (t): Select any row [(M −1 (s)L(s))i ]′ and [(M −1 (s)L(s)Q)i ]′ where M (s) = (sI − A) − LQ and L(s) = BR−1 B′ (sI + A′ )−1 : Denote by [ 1 ; 2 ; : : : ; p ] the (unstable) eigenvalues of [ − A + BKss ]; where k has multiplicity nk ¿1 and n1 + n2 + · · · + np = n. Then; the initial value of the steady-state auxiliary function vss (0) is given by vss (0) = − 1 N0 (s) =
n (s)
n (−s) + Proposition 1. Let Xi (s) denote the Laplace transform of the ith component xi (t) of the state vector (i = 1; : : : ; n). Then; the numerator polynomial in the expansion for Xi (s) must be such that the polynomial
n (−s) is cancelled exactly. −1 1 2 (18) and the solution to the algebraic Riccati equation (7) is given by (17) where M = (sI − A) − LQ; L = BR−1 B′ (sI + A′ )−1 ; N0 (s) and Nr (s) are vectors with polynomial entries,
r (s) is a polynomial whose roots depend on the type of reference signal, and
n (s)
n (−s) is the characteristic polynomial of M, with roots consisting of the n stable eigenvalues i of the closed-loop system matrix [A − BKss ], and the n unstable eigenvalues i = −i . Pss = −1 1 0; (19) where, for j = 0; 1; : : : ; nk − 1; 1 is an n × n matrix whose rows are dj [(M −1 (s)L(s))i ]′ |s= k ; ds j 2 is an n × 1 vector whose entries are dj  [(M −1 (s)L(s)Q)i ]′ Xr (s) |s= k ; ds j 80 E. Barbieri, R. Alba-Flores / Systems & Control Letters 40 (2000) 77–82 and 0 is an n × n matrix whose rows are j d [(M −1 (s))i ]′ |s= k ; ds j j = 0; 1; : : : ; nk − 1: Proof. Consider the case where k ; k = 1; : : : ; n, are distinct, that is, nk = 1; k = 1; : : : ; n. Then, from Proposition 1 and Eq. (17), we set up the n equations {[(M −1 )i ]′ x0 − [(M −1 L)i ]′ 0 − [(M −1 LQ)i ]′ Xr (s)} =[s − k] nk =0 at s = (20) k [1 s s2 · · · sn−1 ]′ . Therefore, from we nd that  1 0 0 ··· 0  0 s 0 ··· 0   1  0 0 s2 · · · 0 X (s) =  det(M )   .. .. .. .. .. . . . . .  0 · · · · · · · · · sn−1 ×[0 − QXr (s)]: or, equivalently, 0 x0 + 1 0 + 2 = 0: (21) It is clear that when k is repeated, then the additional nk −1 equations are generated by taking nk −1 derivatives of (20). Solving for 0 from (21), we observe that 0 can be written as the sum of two components, one that depends on x0 and the other that depends on the reference signal Xr . That is, 0 = − −1 1 0 x0 − −1 1 (22) 2: Substituting (22) into (13) at t = 0 we get u(0) = −R−1 B′ −1 1 0 x0 − R−1 B′ −1 1 2 (17) with x0 = 0,    ′  (L )n (s)    ′   (L )n (s)       ..   .     (L′ )n (s) Finally, consider the expansion for Xi (s) given by Xi (s) = 1 si−1 (L′ )n (s)[0 − QXr (s)] det(M ) which, when evaluated at k, results in si−1 (L′ )n (s)[0 − QXr (s)]|s= k = [s − k] nk : (25) Then, the n equations are generated by substituting s = k into (25) and its nk − 1 derivatives. In each such equation, we can see that the factor introduced by si−1 can be divided out, and therefore M −1 does not play a role in the computation of 1 and 2 . (23) Refs. [3,4] use Theorem 1 to construct matrices 1 and 0 in terms of easy-to-generate polynomials for continuous- and discrete-time single-input systems in phase-canonic form. and comparing (23) with (8) at t = 0 u(0) = −R−1 B′ Pss x0 + R−1 B′ vss (0); the result follows. A special case is worth noting where M −1 (s) is not required to compute vss (0). Corollary 1. Consider the scalar control case and the pair {A; b} in the standard phase canonic (controllable) form. Then vss (0) = − where 1 −1 1 2 (24) is an n × n matrix whose rows are dj [(L(s))n ]′ |s= ds j and 2; k is an n × 1 vector whose entries are dj  [(L(s)Q)n ]′ Xr (s) |s= k : ds j Proof. Since {A; b} is in phase-canonic form, then the rst n − 1 rows of matrix L(s) are identically zero. It can also be veri ed that the nth column of M −1 is 4. Examples The change of variables  = T − t in Eq. (9) leads to the forward di erential equation dvss () = [A − BKss ]′ vss () + Qxr (T − ); d with initial condition vss ( = 0) = Qf xr (T ). Then, vss (t = 0) is obtained from the graph of vss () at  = T . Given the two-input, second-order system and quadratic-cost matrices " # " # " # −1 2 1 2 2 1 A= ; B= ; Q= ; 4 3 0 1 1 2 " # 2 0 R= ; 0 1 the unstable eigenvalues of the Hamiltonian matrix are = [5:7669 2:7826]′ : 81 E. Barbieri, R. Alba-Flores / Systems & Control Letters 40 (2000) 77–82 Fig. 1. Example 1: Function v() in the interval  ∈ [0; 10]. The polynomial matrices L(s) and M −1 (s) are given by " # 0:5(19 + 9s) 2(−10 + s) 1 ; L(s) = 2 s + 2s − 11 2(2 + s) −9 + s M −1 × 2 1 = 4 2s − 82s2 + 515 " 4s2 +25s−105 2(4s2 +13s−45) 2(s3 +3s2 −20s−10) # and, for a constant reference signal, Xr (s) = [2=s 2=s]′ we nd M −1 (s)L(s)QXr (s) vss (0) = [0:8377 and 2 122:752]′ 6(6s2 + 48s − 95)]′ : Selecting the rst row of M −1 (s)L(s)QXr (s) 0:1565] is found and veri ed in Fig. 1. Example 2. Consider a third-order integrator in phasecanonic form. Letting R = 1 and Q = diag(4; 0; 0), the unstable eigenvalues of the Hamiltonian matrix are = [0:63 + j1:09 1 = s(2s4 − 82s2 + 515) M −1 (s)L(s) and = [ − 18713:8 1 from which the initial condition 2(s3 −s2 −21s+47) ×[6(13s2 − 48s + 42) and evaluating at the unstable eigenvalues we compute " # −23339:4 5352:61 ; 1= 167:125 −110:209 0:63 − j1:09 The polynomial matrix L(s) is   0 0 0    0 0 L(s) =  0 ;   1 1 1 − s3 s2 s 1:26]′ : 82 E. Barbieri, R. Alba-Flores / Systems & Control Letters 40 (2000) 77–82 Fig. 2. Example 2: Function v() in the interval  ∈ [0; 40]. and, for a reference xr (t) = [0:5t 2 t 1]′ , we take the third row of L(s) and L(s)QXr (s) from which 1 and 2 are found to be  1   1 1  1 − 2  16   13 1  1      1   1 1  1     − 2 ; 2 = −4  6  1= 3   2   2 2  2     1  1  1 1 − 2 3 6 3 and vss (0) = [2 Fig. 2. 3 3:17 3 3 ′ 2:52] . These are veri ed in 5. Conclusions A linear system of algebraic equations were derived to calculate the initial condition of the auxiliary function in the in nite-horizon linear-quadratic tracking controller. At least for systems of moderate dimensions, the algebraic equations are an alternative to having to integrate the associated di erential equation backwards in time for a (possibly) large interval [T; 0]. A topic for further research (suggested by one reviewer) is to examine error bounds in the cost or a measure in the quality of tracking caused by using vss (0) rather than v(0) in problems where T is nite but large. References [1] B.D. Anderson, J.B. Moore, Optimal Control. Linear Quadratic Methods, Prentice-Hall, Englewood Cli s, NJ, 1990. [2] M. Athans, P.L. Falb, Optimal Control, An Introduction to the Theory and its Applications, McGraw-Hill, New York, 1966. [3] E. Barbieri, On the LQR problem for phase-canonic systems, Proceedings of the American Control Conference, San Francisco, CA, June 1993, pp. 3174 –3175. [4] E. Barbieri, A solution to the discrete-time ARE, Proceedings of the American Control Conference, Baltimore, MD, June 1994, pp. 1153–1154. [5] A.E. Bryson, T. Ho, Applied Optimal Control, Hemisphere Publishing, Washington, DC, 1975. [6] H. Kwakernaak, R. Sivan, Linear Optimal Control Systems, Wiley-Interscience, New York, 1972. [7] F.L. Lewis, Optimal Control, Wiley, New York, l986.