Preconditioning of truncated-Newton methods

SG Nash - SIAM Journal on Scientific and Statistical Computing, 1985 - SIAM
SG Nash
SIAM Journal on Scientific and Statistical Computing, 1985SIAM
In this paper we discuss the use of truncated-Newton methods, a flexible class of iterative
methods, in the solution of large-scale unconstrained minimization problems. At each major
iteration, the Newton equations are approximately solved by an inner iterative algorithm. The
performance of the inner algorithm, and in addition the total method, can be greatly
improved by the addition of preconditioning and scaling strategies. Preconditionings can be
developed using either the outer nonlinear algorithm or using information computed during …
In this paper we discuss the use of truncated-Newton methods, a flexible class of iterative methods, in the solution of large-scale unconstrained minimization problems. At each major iteration, the Newton equations are approximately solved by an inner iterative algorithm. The performance of the inner algorithm, and in addition the total method, can be greatly improved by the addition of preconditioning and scaling strategies. Preconditionings can be developed using either the outer nonlinear algorithm or using information computed during the inner iteration. Several preconditioning schemes are derived and tested.
Numerical tests show that a carefully chosen truncated-Newton method can perform well in comparison with nonlinear conjugate-gradient-type algorithms. This is significant, since the two classes of methods have comparable storage and operation counts, and they are the only practical methods for solving many large-scale problems. In addition, with the Hessian matrix available, the truncated-Newton algorithm performs like Newton’s method, usually considered the best general method for this problem.
Society for Industrial and Applied Mathematics