Abstract
We propose a new choice for the parameter in the Broyden class and derive and discuss properties of the resulting self-complementary quasi-Newton update. Our derivation uses a variational principle that minimizes the extent to which the quasi-Newton relation is violated on a prior step. We discuss the merits of the variational principle used here vis-a-vis the other principle in common use, which minimizes deviation from the current Hessian or Hessian inverse approximation in an appropriate Frobenius matrix norm. One notable advantage of our principle is an inherent symmetry that results in the same update being obtained regardless of whether the Hessian matrix or the inverse Hessian matrix is updated.
We describe the relationship of our update to the BFGS, SR1 and DFP updates under particular assumptions on line search accuracy, type of function being minimized (quadratic or nonquadratic) and norm used in the variational principle.
Some considerations concerning implementation are discussed and we also give a numerical illustration based on an experimental implementation using MATLAB.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
M. Al-Baali, “An efficient class of quasi-Newton algorithms in the Broyden family,” Report No. 9, Department of Electronics, Informatics and Systems, University of Calabria (Calabria, Italy, 1992).
C.G. Broyden, “The convergence of a class of double-rank minimization algorithms,” Parts I and II.Journal of the Institute of Mathematical Applications 6 (1970) 76–90, 222–236.
R.H. Byrd, D.C. Liu and J. Nocedal, “On the behaviour of Broyden's class of quasi-Newton methods,”SIAM Journal on Optimization 2 (1992) 533–557.
A.R. Conn, N.I.M. Gould and Ph.L. Toint, “Convergence of quasi-Newton matrices generated by the symmetric rank one update,”Mathematical Programming 50 (1991) 177–195.
W.C. Davidon, “Variable metric methods for minimization,” Research and Development Report ANL-5990 (Rev.), Argonne National Laboratory (Argonne, IL, 1959). (Reprinted inSIAM Journal on Optimization 1 (1991).)
W.C. Davidon, “Optimally conditioned optimization algorithms without line searches,”Mathematical Programming 9 (1975) 1–30.
W.C. Davidon, R.B. Mifflin and J.L. Nazareth, “Some comments on notation for quasi-Newton methods,”Optima 32 (1991) 3–4.
L.C.W. Dixon, “Quasi-Newton algorithms generate identical points,”Mathematical Programming 2 (1972) 383–387.
J.E. Dennis and J.J. Moré, “Quasi-Newton methods, motivation and theory,”SIAM Review 19 (1977) 46–89.
J.E. Dennis and R.B. Schnabel,Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Prentice-Hall, Englewood Cliffs, NJ, 1983).
R. Fletcher,Practical Methods of Optimization, Vol. 1, Unconstrained Optimization (John Wiley & Sons, New York, 1980).
J. Guo, “On the local convergence of the symmetric rank one method,” Department of Pure and Applied Mathematics, Washington State University (preprint, 1992).
S. Hoshino, “A formulation of variable metric methods,”Journal of the Institute of Mathematics and its Applications 10 (1972) 394–403.
H. Khalfan, R.H. Byrd and R.B. Schnabel, “A theoretical and experimental study of the symmetric rank one update,”SIAM Journal on Optimization 3 (1993) 1–24.
J.J. Moré, B.S. Garbow and K.E. Hillstrom, “Testing unconstrained optimization software,”ACM Transactions on Mathematical Software 7 (1981) 17–41.
J.L. Nazareth, “An alternative variational principle for variable metric updating,”Mathematical Programming 30 (1984) 99–104.
M.J.D. Powell, “How bad are the BFGS and DFP methods when the objective function is quadratic?”Mathematical Programming 34 (1986) 34–47.
Y. Zhang and R.P. Tewarson, “Quasi-Newton algorithms with updates from the preconvex part of Broyden's family,”IMA Journal of Numerical Analysis 8 (1988) 487–509.
Author information
Authors and Affiliations
Additional information
Corresponding author.
Rights and permissions
About this article
Cite this article
Mifflin, R.B., Nazareth, J.L. The least prior deviation quasi-Newton update. Mathematical Programming 65, 247–261 (1994). https://doi.org/10.1007/BF01581698
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01581698