Abstract
In this chapter, we investigate inverse optimal control problems for discrete-time dynamical systems. We pose two discrete-time inverse optimal control problems involving the computation of the parameters of discrete-time optimal control cost functions from data. The problems differ in whether the available data consists of whole or truncated state and control sequences. We present and discuss methods for solving these problems based on bilevel optimization and discrete-time versions of the minimum principle. We specifically show that minimum-principle methods reduce to solving systems of linear equations or quadratic programs under a linear parameterization of the class of cost functions, and admit conditions under which they are guaranteed to provide unique cost-function parameters. Finally, we develop a bespoke technique for solving discrete-time inverse optimal control problems with linear dynamical systems and infinite-horizon quadratic cost functions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We shall consider variations of the truncated-sequence problem in which the horizon T is either known, unknown, or known only to be finite or infinite.
- 2.
- 3.
We use \(\Vert \cdot \Vert \) to denote the Euclidean norm.
- 4.
At all other times \(k \in \{0, 1, \ldots , T-1\} \setminus \mathscr {K}\), the minimum of the Hamiltonian occurs with \(u_k\) on the boundary of the constraint \(\mathscr {U}\). Hence, \(\nabla _{u} H_k \left( x_k, u_k, \lambda _{k+1}, \theta \right) \) is potentially nonzero at times other than those in \(\mathscr {K}\).
- 5.
- 6.
- 7.
We note that this corresponds to choosing basis functions according to Assumption 3.3, which are additionally nonredundant and nontrivial. The concepts of nonredundant and nontrivial basis functions were not discussed in the context of Assumption 3.3 since they were not necessary for the development and analysis of the minimum-principle methods in Sect. 3.4, though they may be of practical concern.
References
Aghasadeghi N, Long A, Bretl T (2012) Inverse optimal control for a hybrid dynamical system with impacts. In: 2012 IEEE international conference on robotics and automation (ICRA), pp 4962–4967
Anderson BDO, Moore JB (1990) Optimal control: linear quadratic methods. Prentice Hall, Englewood Cliffs
Åström KJ, Wittenmark B (1995) Adaptive control, 2nd edn. Addison-Wesley, Reading
Basar T, Olsder GJ (1999) Dynamic noncooperative game theory, vol 23, 2nd edn. Academic, New York
Bertsekas DP (1995) Dynamic programming and optimal control, vol 1, 3rd edn. Athena Scientific, Belmont
Brewer J (1978) Kronecker products and matrix calculus in system theory. IEEE Trans Circuits Syst 25(9):772–781
Engwerda JC, van den Broek WA, Schumacher JM (2000) Feedback Nash equilibria in uncertain infinite time horizon differential games. In: Proceedings of the 14th international symposium of mathematical theory of networks and systems, MTNS 2000, pp 1–6
Goodwin GC, Seron MM, De Doná JA (2006) Constrained control and estimation: an optimisation approach. Springer Science & Business Media, Berlin
Hotz VJ, Miller RA (1993) Conditional choice probabilities and the estimation of dynamic models. Rev Econ Stud 60(3):497–529
Hotz VJ, Miller RA, Sanders S, Smith J (1994) A simulation estimator for dynamic models of discrete choice. Rev Econ Stud 61(2):265–289
Jin W, Kulić D, Lin JF-S, Mou S, Hirche S (2019) Inverse optimal control for multiphase cost functions. IEEE Trans Robot 35(6):1387–1398
Jin W, Kulić D, Mou S, Hirche S (2021) Inverse optimal control from incomplete trajectory observations. Int J Robot Res 40(6–7):848–865
Wanxin J, Shaoshuai M (2021) Distributed inverse optimal control. Automatica 129:109658
Kalman RE (1964) When is a linear control system optimal? J Basic Eng 86(1):51–60
Keshavarz A, Wang Y, Boyd S (2011) Imputing a convex objective function. In: 2011 IEEE international symposium on intelligent control (ISIC). IEEE, pp 613–619
Lancaster P, Rodman L (1995) Algebraic riccati equations. Clarendon Press, Oxford
Lin JF-S, Bonnet V, Panchea AM, Ramdani N, Venture G, Kulić D (2016) Human motion segmentation using cost weights recovered from inverse optimal control. In: 2016 IEEE-RAS 16th international conference on humanoid robots (Humanoids). IEEE, pp 1107–1113
Ljung L (1999) System identification: theory for the user. Prentice Hall PTR, Upper Saddle River
Molloy TL, Ford JJ, Perez T (2018) Finite-horizon inverse optimal control for discrete-time nonlinear systems. Automatica 87:442–446
Molloy TL, Ford JJ, Perez T (2018) Online inverse optimal control on infinite horizons. In: 2018 IEEE conference on decision and control (CDC), pp 1663–1668
Molloy TL, Ford JJ, Perez T (2020) Online inverse optimal control for control-constrained discrete-time systems on finite and infinite horizons. Automatica 120:109109
Molloy TL, Tsai D, Ford JJ, Perez T (2016) Discrete-time inverse optimal control with partial-state information: a soft-optimality approach with constrained state estimation. In: 2016 IEEE 55th annual conference on decision and control (CDC), Las Vegas, NV
Mombaur K, Truong A, Laumond J-P (2010) From human to humanoid locomotion–an inverse optimal control approach. Auton Robots 28(3):369–383
Panchea AM, Ramdani N (2015) Towards solving inverse optimal control in a bounded-error framework. In: American control conference (ACC) 2015, pp 4910–4915
Parsapour M, Kulić D (2021) Recovery-matrix inverse optimal control for deterministic feedforward-feedback controllers. In: 2021 American control conference (ACC), pp 4765–4770
Priess MC, Conway R, Choi J, Popovich JM, Radcliffe C (2015) Solutions to the inverse LQR problem with application to biological systems analysis. IEEE Trans Control Syst Technol 23(2):770–777
Puydupin-Jamin A-S, Johnson M, Bretl T (2012) A convex approach to inverse optimal control and its application to modeling human locomotion. In: 2012 IEEE international conference on robotics and automation (ICRA), pp 531–536
Rust J (1987) Optimal replacement of GMC bus engines: an empirical model of Harold Zurcher. Econometrica 55(5):999–1033
Yokoyama N (2017) Inference of aircraft intent via inverse optimal control including second-order optimality condition. J Guid Control Dyn 41(2):349–359 Feb
Yu C, Yao L, Hao F, Jie C (2021) System identification approach for inverse optimal control of finite-horizon linear quadratic regulators. Automatica 129:109636
Zhang H, Li Y, Hu X (2019) Inverse optimal control for finite-horizon discrete-time linear quadratic regulator under noisy output. In: 2019 IEEE 58th conference on decision and control (CDC), pp 6663–6668
Han Z, Jack U, Hu X (2019) Inverse optimal control for discrete-time finite-horizon Linear quadratic regulators. Automatica 110:108593
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Molloy, T.L., Inga Charaja, J., Hohmann, S., Perez, T. (2022). Discrete-Time Inverse Optimal Control. In: Inverse Optimal Control and Inverse Noncooperative Dynamic Game Theory. Communications and Control Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-93317-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-93317-3_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-93316-6
Online ISBN: 978-3-030-93317-3
eBook Packages: EngineeringEngineering (R0)