Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Discrete-Time Inverse Optimal Control

  • Chapter
  • First Online:
Inverse Optimal Control and Inverse Noncooperative Dynamic Game Theory

Part of the book series: Communications and Control Engineering ((CCE))

  • 1005 Accesses

Abstract

In this chapter, we investigate inverse optimal control problems for discrete-time dynamical systems. We pose two discrete-time inverse optimal control problems involving the computation of the parameters of discrete-time optimal control cost functions from data. The problems differ in whether the available data consists of whole or truncated state and control sequences. We present and discuss methods for solving these problems based on bilevel optimization and discrete-time versions of the minimum principle. We specifically show that minimum-principle methods reduce to solving systems of linear equations or quadratic programs under a linear parameterization of the class of cost functions, and admit conditions under which they are guaranteed to provide unique cost-function parameters. Finally, we develop a bespoke technique for solving discrete-time inverse optimal control problems with linear dynamical systems and infinite-horizon quadratic cost functions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 129.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We shall consider variations of the truncated-sequence problem in which the horizon T is either known, unknown, or known only to be finite or infinite.

  2. 2.

    These approximate optimality approaches have been widely advocated for in the literature of inverse optimal control and inverse optimization (see, for example, the pioneering work of [15, 23]).

  3. 3.

    We use \(\Vert \cdot \Vert \) to denote the Euclidean norm.

  4. 4.

    At all other times \(k \in \{0, 1, \ldots , T-1\} \setminus \mathscr {K}\), the minimum of the Hamiltonian occurs with \(u_k\) on the boundary of the constraint \(\mathscr {U}\). Hence, \(\nabla _{u} H_k \left( x_k, u_k, \lambda _{k+1}, \theta \right) \) is potentially nonzero at times other than those in \(\mathscr {K}\).

  5. 5.

    Besides stabilizability, detectability also plays a fundamental role in LQ optimal control. We refer to textbooks (e.g., [2] or [4, Sect. 5.5]) for further information on stabilizability and detectability as well as the stronger notions of controllability and observability.

  6. 6.

    That is, P solves (3.85) and leads to a feedback law K of the form in (3.84) belonging to \(\mathscr {F}\).

  7. 7.

    We note that this corresponds to choosing basis functions according to Assumption 3.3, which are additionally nonredundant and nontrivial. The concepts of nonredundant and nontrivial basis functions were not discussed in the context of Assumption 3.3 since they were not necessary for the development and analysis of the minimum-principle methods in Sect. 3.4, though they may be of practical concern.

References

  1. Aghasadeghi N, Long A, Bretl T (2012) Inverse optimal control for a hybrid dynamical system with impacts. In: 2012 IEEE international conference on robotics and automation (ICRA), pp 4962–4967

    Google Scholar 

  2. Anderson BDO, Moore JB (1990) Optimal control: linear quadratic methods. Prentice Hall, Englewood Cliffs

    MATH  Google Scholar 

  3. Åström KJ, Wittenmark B (1995) Adaptive control, 2nd edn. Addison-Wesley, Reading

    Google Scholar 

  4. Basar T, Olsder GJ (1999) Dynamic noncooperative game theory, vol 23, 2nd edn. Academic, New York

    MATH  Google Scholar 

  5. Bertsekas DP (1995) Dynamic programming and optimal control, vol 1, 3rd edn. Athena Scientific, Belmont

    MATH  Google Scholar 

  6. Brewer J (1978) Kronecker products and matrix calculus in system theory. IEEE Trans Circuits Syst 25(9):772–781

    Google Scholar 

  7. Engwerda JC, van den Broek WA, Schumacher JM (2000) Feedback Nash equilibria in uncertain infinite time horizon differential games. In: Proceedings of the 14th international symposium of mathematical theory of networks and systems, MTNS 2000, pp 1–6

    Google Scholar 

  8. Goodwin GC, Seron MM, De Doná JA (2006) Constrained control and estimation: an optimisation approach. Springer Science & Business Media, Berlin

    MATH  Google Scholar 

  9. Hotz VJ, Miller RA (1993) Conditional choice probabilities and the estimation of dynamic models. Rev Econ Stud 60(3):497–529

    Article  MathSciNet  Google Scholar 

  10. Hotz VJ, Miller RA, Sanders S, Smith J (1994) A simulation estimator for dynamic models of discrete choice. Rev Econ Stud 61(2):265–289

    Article  Google Scholar 

  11. Jin W, Kulić D, Lin JF-S, Mou S, Hirche S (2019) Inverse optimal control for multiphase cost functions. IEEE Trans Robot 35(6):1387–1398

    Article  Google Scholar 

  12. Jin W, Kulić D, Mou S, Hirche S (2021) Inverse optimal control from incomplete trajectory observations. Int J Robot Res 40(6–7):848–865

    Article  Google Scholar 

  13. Wanxin J, Shaoshuai M (2021) Distributed inverse optimal control. Automatica 129:109658

    Google Scholar 

  14. Kalman RE (1964) When is a linear control system optimal? J Basic Eng 86(1):51–60

    Article  Google Scholar 

  15. Keshavarz A, Wang Y, Boyd S (2011) Imputing a convex objective function. In: 2011 IEEE international symposium on intelligent control (ISIC). IEEE, pp 613–619

    Google Scholar 

  16. Lancaster P, Rodman L (1995) Algebraic riccati equations. Clarendon Press, Oxford

    MATH  Google Scholar 

  17. Lin JF-S, Bonnet V, Panchea AM, Ramdani N, Venture G, Kulić D (2016) Human motion segmentation using cost weights recovered from inverse optimal control. In: 2016 IEEE-RAS 16th international conference on humanoid robots (Humanoids). IEEE, pp 1107–1113

    Google Scholar 

  18. Ljung L (1999) System identification: theory for the user. Prentice Hall PTR, Upper Saddle River

    MATH  Google Scholar 

  19. Molloy TL, Ford JJ, Perez T (2018) Finite-horizon inverse optimal control for discrete-time nonlinear systems. Automatica 87:442–446

    Article  MathSciNet  Google Scholar 

  20. Molloy TL, Ford JJ, Perez T (2018) Online inverse optimal control on infinite horizons. In: 2018 IEEE conference on decision and control (CDC), pp 1663–1668

    Google Scholar 

  21. Molloy TL, Ford JJ, Perez T (2020) Online inverse optimal control for control-constrained discrete-time systems on finite and infinite horizons. Automatica 120:109109

    Google Scholar 

  22. Molloy TL, Tsai D, Ford JJ, Perez T (2016) Discrete-time inverse optimal control with partial-state information: a soft-optimality approach with constrained state estimation. In: 2016 IEEE 55th annual conference on decision and control (CDC), Las Vegas, NV

    Google Scholar 

  23. Mombaur K, Truong A, Laumond J-P (2010) From human to humanoid locomotion–an inverse optimal control approach. Auton Robots 28(3):369–383

    Article  Google Scholar 

  24. Panchea AM, Ramdani N (2015) Towards solving inverse optimal control in a bounded-error framework. In: American control conference (ACC) 2015, pp 4910–4915

    Google Scholar 

  25. Parsapour M, Kulić D (2021) Recovery-matrix inverse optimal control for deterministic feedforward-feedback controllers. In: 2021 American control conference (ACC), pp 4765–4770

    Google Scholar 

  26. Priess MC, Conway R, Choi J, Popovich JM, Radcliffe C (2015) Solutions to the inverse LQR problem with application to biological systems analysis. IEEE Trans Control Syst Technol 23(2):770–777

    Google Scholar 

  27. Puydupin-Jamin A-S, Johnson M, Bretl T (2012) A convex approach to inverse optimal control and its application to modeling human locomotion. In: 2012 IEEE international conference on robotics and automation (ICRA), pp 531–536

    Google Scholar 

  28. Rust J (1987) Optimal replacement of GMC bus engines: an empirical model of Harold Zurcher. Econometrica 55(5):999–1033

    Article  Google Scholar 

  29. Yokoyama N (2017) Inference of aircraft intent via inverse optimal control including second-order optimality condition. J Guid Control Dyn 41(2):349–359 Feb

    Article  Google Scholar 

  30. Yu C, Yao L, Hao F, Jie C (2021) System identification approach for inverse optimal control of finite-horizon linear quadratic regulators. Automatica 129:109636

    Google Scholar 

  31. Zhang H, Li Y, Hu X (2019) Inverse optimal control for finite-horizon discrete-time linear quadratic regulator under noisy output. In: 2019 IEEE 58th conference on decision and control (CDC), pp 6663–6668

    Google Scholar 

  32. Han Z, Jack U, Hu X (2019) Inverse optimal control for discrete-time finite-horizon Linear quadratic regulators. Automatica 110:108593

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Electronic Engineering, University of Melbourne, Melbourne, VIC, Australia

    Timothy L. Molloy

  2. Department of Electrical Engineering and Information Technology, Karlsruhe Institute of Technology, Karlsruhe, Germany

    Jairo Inga Charaja & Sören Hohmann

  3. The Boeing Company, Boeing Defence Australia, Brisbane, QLD, Australia

    Tristan Perez

Authors
  1. Timothy L. Molloy
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jairo Inga Charaja
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sören Hohmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tristan Perez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Timothy L. Molloy .

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Molloy, T.L., Inga Charaja, J., Hohmann, S., Perez, T. (2022). Discrete-Time Inverse Optimal Control. In: Inverse Optimal Control and Inverse Noncooperative Dynamic Game Theory. Communications and Control Engineering. Springer, Cham. https://doi.org/10.1007/978-3-030-93317-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-93317-3_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-93316-6

  • Online ISBN: 978-3-030-93317-3

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation