Abstract
Discrete-time optimal control (DTOC) problems are large-scale optimization problems with a dynamic structure. In previous work this structure has been exploited to provide very fast and efficient local procedures. Two examples are the differential dynamic programming algorithm (DDP) and the stagewise Newton procedure—both require onlyO(N) operations per iteration, whereN is the number of timesteps. Both exhibit a quadratic convergence rate. However, most algorithms in this category do not have a satisfactory global convergence strategy. The most popular global strategy is shifting: this sometimes works poorly due to the lack of automatic adjustment to the shifting element.
In this paper we propose a method that incorporates the trust region idea with the local stagewise Newton's method. This method possesses advantages of both the trust region idea and the stagewise Newton's method, i.e., our proposed method has strong global and local convergence properties yet remains economical. Preliminary numerical results are presented to illustrate the behavior of the proposed algorithm. We also collect in the Appendix some DTOC problems that have appeared in the literature.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
D.P. Bertsekas, “Projected Newton methods for optimization problems with simple constraints”,SIAM J. Control and Optimization, 20:221–246, 1982.
B.D. Craven,Mathematical Programming and Control Theory, Chapman and Hall, London, 1978.
T.B. Culver and C.A. Shoemaker “Dynamic optimal control for groundwater remediation with flexible management periods”,Water Resources Research, 28:629–641, 1992.
P. Dyer and S.R. Mcreynolds,The computation and theory of optimal control, Academic Press, 1970.
R. Fletcher,Practical Methods of Optimization, John Wiley & Sons, Ltd., 1987.
D.M. Gay, “Computing optimal local constrained steps”,SIAM J. Sci. Stat. Comput., 2:186–197, 1981.
D. Jacobson and D. Mayne,Differential Dynamic Programming, Elsevier Sci. Publ., 1970.
L.-Z. Liao and C.A. Shoemaker, “Convergence in unconstrained discrete-time differential dynamic programming”,IEEE Trans. Automat. Contr., 36:692–706, 1991.
L.-Z. Liao and C.A. Shoemaker, “Advantages of differential dynamic programming over Newton's method for discrete-time optimal control problems”, Technical Report ctc92tr97, Advanced Computing Research Institute, Cornell University, 1992.
D.G. Luenberger,Linear and Nonlinear Programming, Addison-Wesley, Reading, Mass., 1984.
D. Mayne, “A second-order gradient method for determining optimal trajectories of non-linear discrete-time systems”,Intnl. J. Control, 3:85–95, 1966.
J.J. Moré, “Recent developments in algorithms and software for trust region methods”. In Bachem, A., Grötschel, M., and Korte, B., editors,Mathematical Programming, pages 258–287, New York, 1983. Springer-Verlag.
J.J. Moré and D.C. Sorensen, Computing a trust region step,SIAM J. Sci. Stat. Comput., 4:553–572, 1983.
D.M. Murray and S.J. Yakowitz, The application of optimal control methodology to nonlinear programming problems,Mathematical Programming, 21:331–347, 1981.
D.M. Murray and S.J. Yakowitz, “Differential dynamic programming and Newton's method for discrete optimal control problems”,J. of Optimization Theory and Applications, 43:395–414, 1984.
J. Nocedal and Y. Yuan, “Combining trust region and line search techniques”, Technical Report, Dept. of EE and Computer Science, Northwestern University, 1992.
J.F.A. De and O. Pantoja, “Differential dynamic programming and Newton's method”,Intnl. J. Control, 47:1539–1553, 1988.
G. Di, Pillo, L. Grippo and F. Lampariello, “A class of structured quasi-Newton algorithms for optimal control problems”, In Rauch, H.E., editor,IFAC Applications of nonlinear programming to optimization and control, pages 101–107. International Federation of Automatic Control, Pergamon Press, New York, 1983.
M.J.D. Powell, “A new algorithm for unconstrained optimization”, in Rosen, J.B., Mangasarian, O., and Ritter, K., editors,Nonlinear Programming, pages 31–65, New York, 1970, Academic Press.
D. Ralph, “A parallel method for discrete-time optimal control problems”, Technical Report ctc93tr118, Advanced Computing Research Institute, Cornell University, 1993.
S.J. Wright, “Partitioned dynamic programming for optimal control”,SIAM Journal on Optimization, 1:620–642, 1991.
S. Yakowitz and B. Rutherford, “Computational aspects of discrete-time optimal control”,Applied Mathematics and Computation, 15:29–45, 1984.
Author information
Authors and Affiliations
Additional information
Partially supported by the Cornell Theory Center, which receives major funding from the National Science Foundation and IBM Corporation, with additional support from the State of New York and its Corporate Research Institutes; and by NSF, AFOSR, and ONR through grant DMS-8920550.
Rights and permissions
About this article
Cite this article
Coleman, T.F., Liao, A. An efficient trust region method for unconstrained discrete-time optimal control problems. Comput Optim Applic 4, 47–66 (1995). https://doi.org/10.1007/BF01299158
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01299158