A limited memory quasi-Newton trust-region method for box constrained optimization
Pages 105 - 118
Abstract
By means of Wolfe conditions strategy, we propose a quasi-Newton trust-region method to solve box constrained optimization problems. This method is an adequate combination of the compact limited memory BFGS and the trust-region direction while the generated point satisfies the Wolfe conditions and therefore maintains a positive-definite approximation to the Hessian of the objective function. The global convergence and the quadratic convergence of this method are established under suitable conditions. Finally, we compare our algorithms (IWTRAL and IBWTRAL) with an active set trust-region algorithm (ASTRAL) Xu and Burke (2007) on the CUTEst box constrained test problems presented by Gould et al. (2015). Numerical results show that the presented method is competitive and totally interesting for solving box constrained optimization.
References
[1]
C.G. Broyden, J.E. Dennis, J.J. Moré, On the local and superlinear convergence of quasi-Newton methods, J. Inst. Math. Appl., 12 (1973) 233-246.
[2]
A.R. Conn, N.I.M. Gould, Ph.L. Toint, Convergence of quasi-Newton matrices generated by the symmetric rank one update, Math. Program. A, 50 (1991) 177-195.
[3]
J.E. Dennis, J.J. Moré, Quasi-Newton methods, motivation and theory, SIAM Rev., 19 (1977) 46-89.
[4]
D.H. Li, M. Fukushima, A modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math., 129 (2001) 15-35.
[5]
A.R. Conn, N.I.M. Gould, Ph.L. Toint, Trust Region Methods, Society for Industrial and Applied Mathematics, SIAM, Philadelphia, 2000.
[6]
J. Nocedal, S.J. Wright, Numerical Optimization, Springer, New York, 2006.
[7]
E.S. Levitin, B.T. Polyak, Constrained minimization methods, USSR Comput. Math. Math. Phys., 6 (1966) 1-50.
[8]
J.J. Moré, G. Toraldo, Algorithms for bound constrained quadratic programming problems, Numer. Math., 55 (1989) 377-400.
[9]
A.R. Conn, N.I.M. Gould, Ph.L. Toint, Global convergence of a class of trust region algorithms for optimization with simple bounds, SIAM J. Numer. Anal., 25 (1988) 433-460.
[10]
A.R. Conn, N.I.M. Gould, Ph.L. Toint, Testing a class of methods for solving minimization problems with simple bounds on the variables, Math. Comp., 50 (1988) 399-430.
[11]
J.V. Burke, J.J. Moré, On the identification of active constrained, SIAM J. Numer. Anal., 25 (1988) 1197-1211.
[12]
R.H. Byrd, P. Lu, J. Nocedal, C. Zhu, A limited memory algorithm for bound constrained optimization, SIAM J. Sci. Stat. Comput., 16 (1995) 1190-1208.
[13]
C.J. Lin, J.J. Moré, Newton's method for large bound-constrained optimization problems, SIAM J. Optim., 9 (1999) 1100-1127.
[14]
L. Xu, J.V. Burke, An Active Set ¿ ∞ -trust Region Algorithm for Box Constrained Optimization, Department of Mathematics, University of Washington, Seattle, WA 98195, U.S.A, 2007.
[15]
W. Cheng, A subspace modified PRP method for large-scale nonlinear box-constrained optimization, Numer. Funct. Anal. Optim., 33 (2012) 1372-1385.
[16]
L. Sun, G. He, Y. Wang, C. Zhou, An accurate active set Newton algorithm for large scale bound constrained optimization, Appl. Math., 56 (2011) 297-314.
[17]
X. Wang, Y.X. Yuan, A trust region method based on a new affine scaling technique for simple bounded optimization, Optim. Methods Softw., 28 (2013) 871-888.
[18]
G. Yuan, X. Lu, An active set limited memory BFGS algorithm for bound constrained optimization, Appl. Math. Model., 35 (2011) 3561-3573.
[19]
G. Yuan, Z. Wei, M. Zhang, An active-set projected trust region algorithm for box constrained optimization problems, J. Syst. Sci. Complex., 28 (2015) 1128-1147.
[20]
R.T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, 1970.
[21]
J.V. Burke, J.J. Moré, G. Toraldo, Convergence properties of trust region methods for linear and convex constraints, Math. Program., 47 (1990) 305-336.
[22]
L. Kaufman, Reduced storage, Quasi-Newton trust region approaches to function optimization, SIAM J. Optim., 10 (1999) 56-69.
[23]
J.B. Erway, R.F. Marcia, On efficiently computing the eigenvalues of limited-memory Quasi-Newton matrices, SIAM J. Matrix Anal. Appl., 36 (2015) 1338-1359.
[24]
D.C. Liu, J. Nocedal, On the limited memory BFGS method for large scale optimization, Math. Program., 45 (1989) 503-528.
[25]
J. Nocedal, Updating quasi-Newton matrices with limited storage, Math. Comp., 35 (1980) 773-782.
[26]
M. Ahookhosh, K. Amini, M. Kimiaei, A globally convergent trust-region method for large-scale symmetric nonlinear systems, Numer. Funct. Anal. Optim., 36 (2015) 830-855.
[27]
M. Ahookhosh, K. Amini, M. Kimiaei, M.R. Peyghami, A limited memory trust-region method with adaptive radius for large-scale unconstrained optimization, Bull. Iranian Math. Soc. (2016).
[28]
H. Esmaeili, M. Kimiaei, An efficient implementation of a trust region method for box constrained optimization, J. Appl. Math. Comput., 48 (2015) 495-517.
[29]
R.H. Byrd, J. Nocedal, R. Schnabel, Representation of quasi-Newton matrices and their use in limited memory methods, Math. Program., 63 (1994) 129-156.
[30]
Ph.L. Toint, Towards an efficient sparsity exploiting Newton method for minimization, in: Sparse Matrices and their Uses, Academic Press, New York, 1982, pp. 57-87.
[31]
J. Nocedal, Y.X. Yuan, Combining Trust-Region and Line-Search Techniques. Optimization Technology Center mar OTC 98/04 1998.
[32]
E.M. Gertz, Combination Trust-Region Line-Search Methods for Unconstrained Optimization, University of California, San Diego, 1999.
[33]
E.M. Gertz, A quasi-Newton trust-region method, Math. Program. Ser. A, 100 (2004) 447-470.
[34]
M.J.D. Powell, A new algorithm for unconstrained optimization, in: Nonlinear Programming, Academic Press, 1970.
[35]
S.W. Thomas, Sequential Estimation Techniques for Quasi-Newton Algorithms, Cornell University, 1975.
[36]
D.F. Shanno, K.H. Phua, Matrix conditioning and non-linear optimization, Math. Program., 14 (1978) 149-160.
[37]
E.M. Simantiraki, D.F. Shanno, An infeasible-interior-point method for linear complementarity problems, SIAM J. Optim., 7 (1997) 620-640.
[38]
N.I.M. Gould, D. Orban, Ph.L. Toint, CUTEst: a Constrained and Unconstrained Testing Environment with safe threads, Comput. Optim. Appl., 60 (2015) 545-557.
[39]
E. Dolan, J.J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002) 201-213.
- A limited memory quasi-Newton trust-region method for box constrained optimization
Recommendations
Projected quasi-Newton algorithm with trust region for constrained optimization
In Ref. 1, Nocedal and Overton proposed a two-sided projected Hessian updating technique for equality constrained optimization problems. Although local two-step Q-superlinear rate was proved, its global convergence is not assured. In this paper, we ...
A Feasible BFGS Interior Point Algorithm for Solving Convex Minimization Problems
We propose a BFGS primal-dual interior point method for minimizing a convex function on a convex set defined by equality and inequality constraints. The algorithm generates feasible iterates and consists in computing approximate solutions of the ...
A Global Convergence Theory for General Trust-Region-Based Algorithms for Equality Constrained Optimization
This work presents a global convergence theory for a broad class of trust-region algorithms for the smooth nonlinear programming problem with equality constraints. The main result generalizes Powell's 1975 result for unconstrained trust-region ...
Comments
Information & Contributors
Information
Published In
Copyright © Elsevier B.V.
Publisher
Elsevier Science Publishers B. V.
Netherlands
Publication History
Published: 01 September 2016
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 15 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in