article

Free access

A modular system of algorithms for unconstrained minimization

Editor: John R. Rice Authors:

Robert B. Schnabel,

John E. Koonatz,

Barry E. WeissAuthors Info & Claims

ACM Transactions on Mathematical Software (TOMS), Volume 11, Issue 4

Pages 419 - 440

https://doi.org/10.1145/6187.6192

Published: 01 December 1985 Publication History

PDF eReader

Abstract

We describe a new package, UNCMIN, for finding a local minimizer of a real valued function of more than one variable. The novel feature of UNCMIN is that it is a modular system of algorithms, containing three different step selection strategies (line search, dogleg, and optimal step) that may be combined with either analytic or finite difference gradient evaluation and with either analytic, finite difference, or BFGS Hessian approximation. We present the results of a comparison of the three step selection strategies on the problems in More, Garbow, and Hillstrom in two separate cases: using finite difference gradients and Hessians, and using finite difference gradients with BFGS Hessian approximations. We also describe a second package, REVMIN, that uses optimization algorithms identical to UNCMIN but obtains values of user-supplied functions by reverse communication.

References

[1]

DENNIS, J. E., JR., GAY, D. M., AND WELSCH, R. E. An adaptive nonlinear least-square algorithm. ACM Trans. Math. Softw. 7 (1981), 348-368.

Crossref

Google Scholar

[2]

DENNIS, J. E., JR., AND MEI, H. H.W. Two new unconstrained optimization algorithms which use function and gradient values. J. Optimization Theory and Its Applications 28 (1979), 453-482.

Google Scholar

[3]

DENNIS, J. E., JR., AND SCHNABEL, R. B. Numerical Methods for Nonlinear Equations and Unconstrained Optimization. Prentice-Hall, Englewood Cliffs, N. J., 1983.

Google Scholar

[4]

FLETCHER, R. Practical Methods of Optimization, Vol. 1. Unconstrained Optimization. Wiley, New York, 1980.

Crossref

Google Scholar

[5]

GAY, D.M. Some tips for writing portable software. Tech. Rept. TR-14, Center for Computational Research in Economics and Management Science, Massachusetts Institute of Technology, Cambridge, Mass., 1980.

Google Scholar

[6]

GAY, D.M. Computing optimal locally constrained steps. SIAM J. Sci. Stat. Comput. 2, (1981), 186-197.

Google Scholar

[7]

GAY, D.M. Subroutines for unconstrained minimization using a model/trust-region approach. ACM Trans. Math. Softw. 9 (1983), 503-524.

Crossref

Google Scholar

[8]

GILL, P. E., AND MURRAY, W. Newton-type methods for unconstrained and linearly constrained optimization. Math. Program. 28 (1974), 311-350.

Google Scholar

[9]

GILL, P. E., MURRAY, W., PICKEN, S. M., AND WRIGHT, M.H. The design and structure of a Fortran program library for optimization. ACM Trans. Math. Softw. 5 (1979), 259-283.

Crossref

Google Scholar

[10]

GILL, P. E., MURRAY, W., AND WRIGHT, M.H. Praztical Optimization. Academic Press, New York, 1981.

Google Scholar

[11]

GOLDFARB, D. Factorized variable metric methods for unconstrained optimization. Math. Comput. 30 (1976), 796-811.

Google Scholar

[12]

HAMMING, R.W. Introduction to Applied Numerical Analysis. McGraw-Hill, New York, 1971.

Crossref

Google Scholar

[13]

Harwell Subroutine Library, A Catalogue of Subroutines (M. J. Hopper, Ed.), Computer Science and Systems Division, A.E.R.E. Harwell, Oxon., England.

Google Scholar

[14]

IMSL Library Reference Manual, International Mathematical and Statistical Libraries, Houston, Tex.

Google Scholar

[15]

KROGH, F. T. VODQ/SVDQ/DVDQwVariable order integrators for numerical solution of ordinary differential equations. Subroutine Write-Up, Section 314, Jet Propulsion Laboratory, Pasadena, Calif.

Google Scholar

[16]

LAwsoN, C., BLOCK, N., AND GARRETT, R. Fortran IV subroutines for contour plotting. Technical Memo 106, Section 314, Jet Propulsion Laboratory, Pasadena, Calif.

Google Scholar

[17]

MINPACK Documentation, Applied Mathematics Division, Argonne National Laboratory, Argonne, Ill.

Google Scholar

[18]

MOR~, J.J. The Levenberg-Marquardt algorithm: Implementation and theory. In G. A. Watson, Numerical Analysis, Dundee 1977, Lecture Notes in Mathematics 630. Ed., Springer-Verlag, Berlin, pp. 105-116.

Google Scholar

[19]

MOR~, J. J. On the design of optimization software. In S. Incerti and G. Treccani, Eds., Otimazzazione Nonlineare e Applicazioni. Pitagora Editrice, Bologna, Italy, 1980.

Google Scholar

[20]

MOR~:, J.J. Notes on optimization software. In M. J. D. Powell, Ed., Nonlinear Optimization 1981, Academic Press, New York, 1982, pp. 339-352.

Google Scholar

[21]

MOR~, J. J., GARBOW, B. S., AND HILLSTROM, Ko E. Testing unconstrained optimization software. ACM Trans. Math. Softw. 7 (1981), 17-41.

Crossref

Google Scholar

[22]

MOR~:, J. J., AND SORENSEN, D.C. Computing a trust region step. SIAM J. Sci. Stat. Comput. 4, (1983), 553-572.

Google Scholar

[23]

NAG Fortran Library Manual, The Numerical Algorithms Group (USA), Downers Grove, Ill.

Google Scholar

[24]

OSTERWEIL, L. J., AND FOSDICK, L.D. DAVE--A validation error detection and documentation system for Fortran programs. Softw. Pract. Exp. 6, (1976), 473-486.

Google Scholar

[25]

RYDER, B.G. The PFORT verifier. Softw. Pract. Exp. 4, (1974), 359-377.

Google Scholar

[26]

SHANNO, D. F., AND PHUA, K.H. Minimization of unconstrained multivariate functions. ACM Trans. Math. Softw. 2, {1976), 87-94.

Crossref

Google Scholar

[27]

SHULTZ, G. A., SCHNABEL, R. B., AND BYRD, R.H. A family of trust region based algorithms for unconstrained minimization with strong global convergence properties. SIAM J. Numer. Anal. 22, (1985), 47-67.

Google Scholar

[28]

SORENSEN, D.C. Newton's method with a model trust region modification. SIAM Jo Numer. Anal. 19, {1982), 409-426.

Google Scholar

[29]

WAGENER, J.L. Status of work towards revision of programming language Fortran. SIGNUM Newsletter 19, 3 (July 1984).

Crossref

Google Scholar

[30]

WEISS, B.E. A modular software package for solving unconstrained nonlinear optimization problems. M. S. thesis, Department of Computer Science, University of Colorado at Boulder, 1980.

Google Scholar

Cited By

View all

Zhang MCai BDai WKong DZhao HZhang J(2024)Learning brain connectivity in social cognition with dynamic network regressionThe Annals of Applied Statistics10.1214/24-AOAS194218:4Online publication date: 1-Dec-2024
https://doi.org/10.1214/24-AOAS1942
Ding YSomers VSi B(2024)Multi-modal mixed-type structural equation modeling with structured sparsity for subgroup discovery from heterogeneous health dataIISE Transactions10.1080/24725854.2024.2445776(1-22)Online publication date: 23-Dec-2024
https://doi.org/10.1080/24725854.2024.2445776
Vetr NGay NAdkins JAlbertson BAmar DAmper MArmenteros JAshley EAvila-Pacheco JBae DBalci ABamman MBararpour NBarton EJean Beltran PBergman BBessesen DBodine SBooth FBouverat BBuford TBurant CCaputo TCarr SChambers TChavez CChikina MChiu RCicha MClish CCoen PCooper DCornell ECutter GDalton KDasari SDennis CEsser KEvans CFarrar RFernádez FGadde KGagne NGaul DGe YGerszten RGoodpaster BGoodyear LGritsenko MGuevara KHaddad FHansen JHarris MHastie THennig KHershman SHevener AHirshman MHou ZHsu FHuffman KHung CHutchinson-Bunch CIvanova AJackson BJankowski CJimenez-Morales DJin CJohannsen NNewton RKachman MKe BKeshishian HKohrt WKramer KKraus WLanza ILeeuwenburgh CLessard SLester BLi JLindholm MLira ALiu XLu CMakarewicz NManer-Smith KMani DMany GMarjanovic NMarshall AMarwaha SMay SMelanson EMiller MMonroe MMoore SMoore RMoreau KMundorff CMusi NNachun DNair VNair KNestor MNicklas BNigro PNudelman GOrtlund EPahor MPearce CPetyuk VPiehowski PPincas HPowers SPresby DQian WRadom-Aizik SRaja ARamachandran KRamaker MRamos IRankinen TRaskind ARasmussen BRavussin ERector RRejeski WRichards CRirak SRobbins JRooney JRubenstein ARuf-Zamojski FRushing SSagendorf TSamdarshi MSanford JSavage ESchauer ISchenk SSchwartz RSealfon SSeenarine NSmith KSmith GSnyder MSoni TOliveira De Sousa LSparks LSteep AStowe CSun YTeng CThalacker-Mercer AThyfault JTibshirani RTracy RTrappe STrappe TUppal KVangeti SVasoya MVolpi EVornholt AWalkup MWalsh MWheeler MWilliams JWu SXia AYan ZYu XZang CZaslavsky EZebarjadi NZhang TZhao BZhen JMontgomery S(2024)The impact of exercise on gene regulation in association with complex trait geneticsNature Communications10.1038/s41467-024-45966-w15:1Online publication date: 1-May-2024
https://doi.org/10.1038/s41467-024-45966-w
Show More Cited By

Index Terms

A modular system of algorithms for unconstrained minimization

Recommendations

Nonlinear complementarity as unconstrained and constrained minimization

The nonlinear complementarity problem is cast as an unconstrained minimization problem that is obtained from an augmented Lagrangian formulation. The dimensionality of the unconstrained problem is the same as that of the original problem, and the ...
An unconstrained smooth minimization reformulation of the second-order cone complementarity problem

A popular approach to solving the nonlinear complementarity problem (NCP) is to reformulate it as the global minimization of a certain merit function over źn. A popular choice of the merit function is the squared norm of the Fischer-Burmeister function, ...
A class of variable-metric-secant algorithms for unconstrained minimization

Reviews

Reviewer: Henry W. Mosteller

This paper describes UNCMIN, a modular system of FORTRAN subroutines for solving :7S x f( x), :9F:Y where x is an n-dimensional vector and f is a scalar function of x. The first and second derivatives of f should be continuous. This is an important restriction. Many real problems have discontinuous function values as well as first and second derivatives. A nice feature of the package is its modular approach. It offers the following options: (1)Three different step selection methods—line search, dogleg, and hookstep. (2)Analytic or finite difference gradient evaluation. (3)Analytic, finite difference, or BFGS Hessian calculation. The Newton minimization algorithm is used to find the minimum. A large number of standard test functions were minimized using various choices from the above options. Some conclusions are: the choice of step selection method does not change the results much; and the BFGS Hessian approximation gives a smaller number of function evaluations compared to the finite difference Hessian. The ideas in this paper are explained in more detail in [1]. The reader who is interested in this approach to unconstrained minimization should read this book.

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Mathematical Software

ACM Transactions on Mathematical Software Volume 11, Issue 4

Dec. 1985

131 pages

ISSN:0098-3500

EISSN:1557-7295

DOI:10.1145/6187

Editor:
John R. Rice
Purdue Univ. West Lafayette, IN

Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 1985

Published in TOMS Volume 11, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

141
Total Citations
View Citations
1,420
Total Downloads

Downloads (Last 12 months)139
Downloads (Last 6 weeks)16

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Zhang MCai BDai WKong DZhao HZhang J(2024)Learning brain connectivity in social cognition with dynamic network regressionThe Annals of Applied Statistics10.1214/24-AOAS194218:4Online publication date: 1-Dec-2024
https://doi.org/10.1214/24-AOAS1942
Ding YSomers VSi B(2024)Multi-modal mixed-type structural equation modeling with structured sparsity for subgroup discovery from heterogeneous health dataIISE Transactions10.1080/24725854.2024.2445776(1-22)Online publication date: 23-Dec-2024
https://doi.org/10.1080/24725854.2024.2445776
Vetr NGay NAdkins JAlbertson BAmar DAmper MArmenteros JAshley EAvila-Pacheco JBae DBalci ABamman MBararpour NBarton EJean Beltran PBergman BBessesen DBodine SBooth FBouverat BBuford TBurant CCaputo TCarr SChambers TChavez CChikina MChiu RCicha MClish CCoen PCooper DCornell ECutter GDalton KDasari SDennis CEsser KEvans CFarrar RFernádez FGadde KGagne NGaul DGe YGerszten RGoodpaster BGoodyear LGritsenko MGuevara KHaddad FHansen JHarris MHastie THennig KHershman SHevener AHirshman MHou ZHsu FHuffman KHung CHutchinson-Bunch CIvanova AJackson BJankowski CJimenez-Morales DJin CJohannsen NNewton RKachman MKe BKeshishian HKohrt WKramer KKraus WLanza ILeeuwenburgh CLessard SLester BLi JLindholm MLira ALiu XLu CMakarewicz NManer-Smith KMani DMany GMarjanovic NMarshall AMarwaha SMay SMelanson EMiller MMonroe MMoore SMoore RMoreau KMundorff CMusi NNachun DNair VNair KNestor MNicklas BNigro PNudelman GOrtlund EPahor MPearce CPetyuk VPiehowski PPincas HPowers SPresby DQian WRadom-Aizik SRaja ARamachandran KRamaker MRamos IRankinen TRaskind ARasmussen BRavussin ERector RRejeski WRichards CRirak SRobbins JRooney JRubenstein ARuf-Zamojski FRushing SSagendorf TSamdarshi MSanford JSavage ESchauer ISchenk SSchwartz RSealfon SSeenarine NSmith KSmith GSnyder MSoni TOliveira De Sousa LSparks LSteep AStowe CSun YTeng CThalacker-Mercer AThyfault JTibshirani RTracy RTrappe STrappe TUppal KVangeti SVasoya MVolpi EVornholt AWalkup MWalsh MWheeler MWilliams JWu SXia AYan ZYu XZang CZaslavsky EZebarjadi NZhang TZhao BZhen JMontgomery S(2024)The impact of exercise on gene regulation in association with complex trait geneticsNature Communications10.1038/s41467-024-45966-w15:1Online publication date: 1-May-2024
https://doi.org/10.1038/s41467-024-45966-w
HU TZHANG YWANG HJIN HLIU BLIN ZMA JWANG XLIU QLIU HCHEN ZZHOU RJIN PZHU JLIU GBEI QLIN XXIE Z(2024)Biological nitrogen fixation in paddy soils is driven by multiple edaphic factors and available phosphorus is the greatest contributorPedosphere10.1016/j.pedsph.2023.09.00234:6(993-1001)Online publication date: Dec-2024
https://doi.org/10.1016/j.pedsph.2023.09.002
Khmeleva GSemenychev VKorobetskaya AKurnikova MFedorenko RTóth B(2023)Comparative Research of Internal and Border Regions: Analyzing the Differences in the Cyclical Dynamics of Industries for Industrial Policy and Territorial DevelopmentEconomies10.3390/economies1103008911:3(89)Online publication date: 13-Mar-2023
https://doi.org/10.3390/economies11030089
Bacri TBerentsen GBulla JStøve B(2023)Computational issues in parameter estimation for hidden Markov models with template model builderJournal of Statistical Computation and Simulation10.1080/00949655.2023.222678893:18(3421-3457)Online publication date: 2-Jul-2023
https://doi.org/10.1080/00949655.2023.2226788
Naveau MKon Kam King GRincent RSansonnet LDelattre M(2023)Bayesian high-dimensional covariate selection in non-linear mixed-effects models using the SAEM algorithmStatistics and Computing10.1007/s11222-023-10367-434:1Online publication date: 18-Dec-2023
https://dl.acm.org/doi/10.1007/s11222-023-10367-4
Fu YKennedy G(2023)Quasi-Newton corrections for compliance and natural frequency topology optimization problemsStructural and Multidisciplinary Optimization10.1007/s00158-023-03630-966:8Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1007/s00158-023-03630-9
Charvat H(2022)Using the Lambert Function to Estimate Shared Frailty Models with a Normally Distributed Random InterceptThe American Statistician10.1080/00031305.2022.211093977:1(41-50)Online publication date: 26-Sep-2022
https://doi.org/10.1080/00031305.2022.2110939
O’Neill B(2022)Computing highest density regions for continuous univariate distributions with known probability functionsComputational Statistics10.1007/s00180-021-01133-z37:2(613-649)Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1007/s00180-021-01133-z
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Nonlinear complementarity as unconstrained and constrained minimization

An unconstrained smooth minimization reformulation of the second-order cone complementarity problem

A class of variable-metric-secant algorithms for unconstrained minimization

Reviews

Access critical reviews of Computing literature here

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations