research-article

A Limited Memory Algorithm for Bound Constrained Optimization

Authors:

Richard H. Byrd,

Jorge Nocedal, and

Ciyou ZhuAuthors Info & Claims

SIAM Journal on Scientific Computing, Volume 16, Issue 5

Pages 1190 - 1208

https://doi.org/10.1137/0916069

Published: 01 September 1995 Publication History

Abstract

An algorithm for solving large nonlinear optimization problems with simple bounds is described. It is based on the gradient projection method and uses a limited memory BFGS matrix to approximate the Hessian of the objective function. It is shown how to take advantage of the form of the limited memory approximation to implement the algorithm efficiently. The results of numerical tests on a set of large problems are reported.

References

[1]

A. V. Aho, J. E. Hopcroff, J. D. Ullman, The Design and Analysis of Computer Algorithms, Addison-Wesley, Reading, MA, 1974

Digital Library

[2]

B. M. Averick, J. J. Moré, User Guide for the MINPMCK-2 Test Problem Collection, Report, ANLIMCS-TM-157, Argonne National Laboratory, Mathematics and Computer Science Division, Argonne, IL, 1991

[3]

Dimitri P. Bertsekas, Projected Newton methods for optimization problems with simple constraints, SIAM J. Control Optim., 20 (1982), 221–246

Digital Library

[4]

I. Bongartz, A. R. Conn, N. I. M. Gould, PH. L. Toint, CUTE: Constrained and Unconstrained Testing Environment, Research Report, IBM T.J. Watson Research Center, Yorktown, NY, 1993

[5]

James V. Burke, Jorge J. Moré, On the identification of active constraints, SIAM J. Numer. Anal., 25 (1988), 1197–1211

Digital Library

[6]

Richard H. Byrd, Jorge Nocedal, Robert B. Schnabel, Representations of quasi-Newton matrices and their use in limited memory methods, Math. Programming, 63 (1994), 129–156

[7]

Paul H. Calamai, Jorge J. Moré, Projected gradient methods for linearly constrained problems, Math. Programming, 39 (1987), 93–116

Digital Library

[8]

Andrew R. Conn, Nicholas I. M. Gould, Ph. L. Toint, Testing a class of methods for solving minimization problems with simple bounds on the variables, Math. Comp., 50 (1988), 399–430

[9]

A. R. Conn, N. I. M. Gould, Ph. L. Toint, Global convergence of a class of trust region algorithms for optimization with simple bounds, SIAM J. Numer. Anal., 25 (1988), 433–460

Digital Library

[10]

A. R. Conn, N. I. M. Gould, Ph. L. Toint, LANCELOT: A FORTRAN package for Large-Scale Nonlinear Optimization, Release A, Springer Series in Computational Mathematics, Vol. 17, Springer-Verlag, Berlin, 1992xx+330

[11]

A. R. Conn, J. Moré, 1993, Private communication

[12]

John E. Dennis, Jr., Robert B. Schnabel, Numerical methods for unconstrained optimization and nonlinear equations, Prentice Hall Series in Computational Mathematics, Prentice Hall Inc., Englewood Cliffs, NJ, 1983xiii+378

[13]

Harwell Subroutine Library, Release 10, Oxfordshire, UK, 1990

[14]

Jean Charles Gilbert, Claude Lemaréchal, Some numerical experiments with variable-storage quasi-Newton algorithms, Math. Programming, 45 (1989), 407–435

Digital Library

[15]

Philip E. Gill, Walter Murray, Margaret H. Wright, Practical optimization, Academic Press Inc. [Harcourt Brace Jovanovich Publishers], London, 1981xvi+401

[16]

A. A. Goldstein, Convex programming in Hilbert space, Bull. Amer. Math. Soc., 70 (1964), 709–710

[17]

E. S. Levitin, B. T. Polyak, Constrained minimization problems, USSR Comput. Math. Math. Phys., 6 (1966), 1–50

[18]

Dong C. Liu, Jorge Nocedal, On the limited memory BFGS method for large scale optimization, Math. Programming, 45 (1989), 503–528

[19]

J. J. Moré, D. J. Thuente, On Line Search Algorithms with Guaranteed Sufficient Decrease, 1990, Mathematics and Computer Science Division Preprint MCS-P153-0590, Argonne National Laboratory, Argonne, IL

[20]

Jorge J. Moré, Gerardo Toraldo, Algorithms for bound constrained quadratic programming problems, Numer. Math., 55 (1989), 377–400

Digital Library

[21]

Jorge Nocedal, Updating quasi-Newton matrices with limited storage, Math. Comp., 35 (1980), 773–782

[22]

J. M. Ortega, W. C. Rheinboldt, Iterative solution of nonlinear equations in several variables, Academic Press, New York, 1970xx+572

Digital Library

[23]

C. Zhu, R. H. Byrd, P. Lu, J. Nocedal, LBFGS-B : Fortran Subroutines for Large-Scale Bound Constrained Optimization, Report, NAM-11, EELS Department, Northwestern University, 1994

Cited By

Xiao NLiu XToh K(2024)Dissolving Constraints for Riemannian OptimizationMathematics of Operations Research10.1287/moor.2023.136049:1(366-397)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1287/moor.2023.1360
Zhao AGu TBi ZYang FYan CZeng XLin ZHu WZhou D(2024)D3PBO: Dynamic Domain Decomposition-based Parallel Bayesian Optimization for Large-scale Analog Circuit SizingACM Transactions on Design Automation of Electronic Systems10.1145/364381129:3(1-25)Online publication date: 31-Jan-2024
https://dl.acm.org/doi/10.1145/3643811
Zhu SWang JSun JYang JLin XWang TZhang LCheng P(2024)Better Pay Attention Whilst FuzzingIEEE Transactions on Software Engineering10.1109/TSE.2023.333812950:2(190-208)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1109/TSE.2023.3338129
Show More Cited By

Index Terms

A Limited Memory Algorithm for Bound Constrained Optimization
1. Mathematics of computing
  1. Mathematical analysis
    1. Mathematical optimization
2. Theory of computation
  1. Design and analysis of algorithms
    1. Mathematical optimization

Index terms have been assigned to the content through auto-classification.

Recommendations

A multi-start opposition-based particle swarm optimization algorithm with adaptive velocity for bound constrained global optimization

In this paper we present a multi-start particle swarm optimization algorithm for the global optimization of a function subject to bound constraints. The procedure consists of three main steps. In the initialization phase, an opposition learning strategy ...
Read More
Dynamic Neighborhood Hybrid Particle Swarm Optimization for Constrained Optimization
ICCIS '10: Proceedings of the 2010 International Conference on Computational and Information Sciences

Particle swarm optimization (PSO) is simple and efficient, but there is serious premature convergence for solving constrained optimization problem. In order to control premature convergence, this paper proposed dynamic neighborhood hybrid particle swarm ...
Read More
A social learning particle swarm optimization algorithm for scalable optimization

Social learning plays an important role in behavior learning among social animals. In contrast to individual (asocial) learning, social learning has the advantage of allowing individuals to learn behaviors from others without incurring the costs of ...
Read More

Comments

Information & Contributors

Information

Published In

cover image SIAM Journal on Scientific Computing

SIAM Journal on Scientific Computing Volume 16, Issue 5

Sep 1995

221 pages

ISSN:1064-8275

Issue’s Table of Contents

Copyright © 1995 Society for Industrial and Applied Mathematics.

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 01 September 1995

Author Tags

65
49

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

451
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

Xiao NLiu XToh K(2024)Dissolving Constraints for Riemannian OptimizationMathematics of Operations Research10.1287/moor.2023.136049:1(366-397)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1287/moor.2023.1360
Zhao AGu TBi ZYang FYan CZeng XLin ZHu WZhou D(2024)D3PBO: Dynamic Domain Decomposition-based Parallel Bayesian Optimization for Large-scale Analog Circuit SizingACM Transactions on Design Automation of Electronic Systems10.1145/364381129:3(1-25)Online publication date: 31-Jan-2024
https://dl.acm.org/doi/10.1145/3643811
Zhu SWang JSun JYang JLin XWang TZhang LCheng P(2024)Better Pay Attention Whilst FuzzingIEEE Transactions on Software Engineering10.1109/TSE.2023.333812950:2(190-208)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1109/TSE.2023.3338129
Bhaumik BDe SChangdar S(2024)Deep learning based solution of nonlinear partial differential equations arising in the process of arterial blood flowMathematics and Computers in Simulation10.1016/j.matcom.2023.10.011217:C(21-36)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1016/j.matcom.2023.10.011
Siino MTinnirello ILa Cascia M(2024)Is text preprocessing still worth the time? A comparative survey on the influence of popular preprocessing methods on Transformers and traditional classifiersInformation Systems10.1016/j.is.2023.102342121:COnline publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1016/j.is.2023.102342
Yoshida WHirose K(2024)Fast same-step forecast in SUTSE model and its theoretical propertiesComputational Statistics & Data Analysis10.1016/j.csda.2023.107861190:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.csda.2023.107861
Günel K(2024)Weighted Centroids in Adaptive Nelder–Mead SimplexApplied Soft Computing10.1016/j.asoc.2023.111178151:COnline publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1016/j.asoc.2023.111178
Cubadda GGiancaterini FHecq AJasiak J(2024)Optimization of the generalized covariance estimator in noncausal processesStatistics and Computing10.1007/s11222-024-10437-134:4Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s11222-024-10437-1
Rhein FHibbe LNirschl H(2024)Hybrid modeling of hetero-agglomeration processes: a framework for model selection and arrangementEngineering with Computers10.1007/s00366-023-01809-840:1(583-604)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s00366-023-01809-8
Ng IZheng YDong XZhang KOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)On the identifiability of sparse ICA without assuming non-GaussianityProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3668202(47960-47990)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3668202
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents