research-article

Inexact Preconditioned Conjugate Gradient Method with Inner-Outer Iteration

Authors:

Gene H. Golub,

Qiang YeAuthors Info & Claims

SIAM Journal on Scientific Computing, Volume 21, Issue 4

Pages 1305 - 1320

https://doi.org/10.1137/S1064827597323415

Published: 01 January 1999 Publication History

Abstract

An important variation of preconditioned conjugate gradient algorithms is inexact preconditioner implemented with inner-outer iterations [G. H. Golub and M. L. Overton, Numerical Analysis, Lecture Notes in Math. 912, Springer, Berlin, New York, 1982], where the preconditioner is solved by an inner iteration to a prescribed precision. In this paper, we formulate an inexact preconditioned conjugate gradient algorithm for a symmetric positive definite system and analyze its convergence property. We establish a linear convergence result using a local relation of residual norms. We also analyze the algorithm using a global equation and show that the algorithm may have the superlinear convergence property when the inner iteration is solved to high accuracy. The analysis is in agreement with observed numerical behavior of the algorithm. In particular, it suggests a heuristic choice of the stopping threshold for the inner iteration. Numerical examples are given to show the effectiveness of this choice and to compare the convergence bound.

References

[1]

F. Bauer, A. Householder, Some inequalities involving the euclidean condition of a matrix, Numer. Math., 2 (1960), 308–311

Digital Library

Google Scholar

[2]

P. Concus, G. H. Golub, and D. P. O’Leary, A Generalized Conjugate Gradient Method for the Numerical Solution of Elliptic Partial Differential Equations, in Sparse Matrix Computations, J. R. Bunch and D. J. Rose, eds., Academic Press, New York, NY, 1976.

Google Scholar

[3]

Howard Elman, Gene Golub, Inexact and preconditioned Uzawa algorithms for saddle point problems, SIAM J. Numer. Anal., 31 (1994), 1645–1661

Digital Library

Google Scholar

[4]

Eldar Giladi, Gene Golub, Joseph Keller, Inner and outer iterations for the Chebyshev algorithm, SIAM J. Numer. Anal., 35 (1998), 300–319

Digital Library

Google Scholar

[5]

G. H. Golub and M. L. Overton, Convergence of a two‐stage Richardson iterative procedure for solving systems of linear equations, in Numerical Analysis, G. A. Watson, ed., Lecture Notes in Math. 912, Springer, New York, Heidelberg, Berlin, 1982, pp. 128–139.

Google Scholar

[6]

Gene Golub, Michael Overton, The convergence of inexact Chebyshev and Richardson iterative methods for solving linear systems, Numer. Math., 53 (1988), 571–593

Digital Library

Google Scholar

[7]

G. H. Golub and C. F. Van Loan, Matrix Computations, The Johns Hopkins University Press, Baltimore, 1983.

Google Scholar

[8]

G. H. Golub and Q. Ye, Inexact Preconditioned Conjugate Gradient Method with Inner‐Outer Iteration Technical report 97‐04, SCCM, Stanford University, Stanford, CA, 1997.

Google Scholar

[9]

H. Munthe‐Kaas, The Convergence Rate of Inexact Preconditioned Steepest Descent Algorithm for Solving Linear Systems, Technical report, Department of Computer Science, Stanford University, Stanford, CA, 1987.

Google Scholar

[10]

Charles Tong, Qiang Ye, Analysis of the finite precision bi‐conjugate gradient algorithm for nonsymmetric linear systems, Math. Comp., 69 (2000), 1559–1575

Digital Library

Google Scholar

[11]

A. van der Sluis, H. van der Vorst, The rate of convergence of conjugate gradients, Numer. Math., 48 (1986), 543–560

Digital Library

Google Scholar

[12]

D. M. Young, Iterative Solution of Large Linear Systems, Academic Press, New York, 1971.

Google Scholar

Cited By

View all

Hu MLou YWang BYan MYang XYe Q(2023)Accelerated Sparse Recovery via Gradient Descent with Nonlinear Conjugate Gradient MomentumJournal of Scientific Computing10.1007/s10915-023-02148-y95:1Online publication date: 8-Mar-2023
https://dl.acm.org/doi/10.1007/s10915-023-02148-y
Diab MFrommer AKahl K(2023)A flexible short recurrence Krylov subspace method for matrices arising in the time integration of port-Hamiltonian systems and ODEs/DAEs with a dissipative HamiltonianBIT10.1007/s10543-023-00999-363:4Online publication date: 10-Nov-2023
https://dl.acm.org/doi/10.1007/s10543-023-00999-3
Ichimura TFujita KKusakabe RKoyama KMurakami SKikuchi YHori THori MInoue HNose TKawashima TLalith MWolf FShende SCulhane CAlam SJagode H(2022)Extreme scale earthquake simulation with uncertainty quantificationProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.5555/3571885.3571890(1-11)Online publication date: 13-Nov-2022
https://dl.acm.org/doi/10.5555/3571885.3571890
Show More Cited By

Index Terms

Inexact Preconditioned Conjugate Gradient Method with Inner-Outer Iteration

Index terms have been assigned to the content through auto-classification.

Recommendations

Scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization

A scaled memoryless BFGS preconditioned conjugate gradient algorithm for solving unconstrained optimization problems is presented. The basic idea is to combine the scaled memoryless BFGS method and the preconditioning technique in the frame of the ...
Preconditioned AHSS iteration method for singular saddle point problems

In this paper, for solving the singular saddle point problems, we present a new preconditioned accelerated Hermitian and skew-Hermitian splitting (AHSS) iteration method. The semi-convergence of this method and the eigenvalue distribution of the ...
Restrictive preconditioners for conjugate gradient methods for symmetric positive definite linear systems

The restrictively preconditioned conjugate gradient (RPCG) method for solving large sparse system of linear equations of a symmetric positive definite and block two-by-two coefficient matrix is further studied. In fact, this RPCG method is essentially ...

Comments

Information & Contributors

Information

Published In

cover image SIAM Journal on Scientific Computing

SIAM Journal on Scientific Computing Volume 21, Issue 4

1999

559 pages

ISSN:1064-8275

Issue’s Table of Contents

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 01 January 1999

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

49
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Hu MLou YWang BYan MYang XYe Q(2023)Accelerated Sparse Recovery via Gradient Descent with Nonlinear Conjugate Gradient MomentumJournal of Scientific Computing10.1007/s10915-023-02148-y95:1Online publication date: 8-Mar-2023
https://dl.acm.org/doi/10.1007/s10915-023-02148-y
Diab MFrommer AKahl K(2023)A flexible short recurrence Krylov subspace method for matrices arising in the time integration of port-Hamiltonian systems and ODEs/DAEs with a dissipative HamiltonianBIT10.1007/s10543-023-00999-363:4Online publication date: 10-Nov-2023
https://dl.acm.org/doi/10.1007/s10543-023-00999-3
Ichimura TFujita KKusakabe RKoyama KMurakami SKikuchi YHori THori MInoue HNose TKawashima TLalith MWolf FShende SCulhane CAlam SJagode H(2022)Extreme scale earthquake simulation with uncertainty quantificationProceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis10.5555/3571885.3571890(1-11)Online publication date: 13-Nov-2022
https://dl.acm.org/doi/10.5555/3571885.3571890
Ichimura TFujita KKoyama KKusakabe RKikuchi YHori THori MMaddegedara LOhi NNishiki TInoue HMinami KNishizawa STsuji MUeda N(2022)152K-computer-node parallel scalable implicit solver for dynamic nonlinear earthquake simulationInternational Conference on High Performance Computing in Asia-Pacific Region10.1145/3492805.3492814(18-29)Online publication date: 7-Jan-2022
https://dl.acm.org/doi/10.1145/3492805.3492814
Mousavirad SGandomi AHomayoun H(2022)A Clustering-based Differential Evolution Boosted by a Regularisation-based Objective Function and a Local Refinement for Neural Network Training2022 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC55065.2022.9870211(1-8)Online publication date: 18-Jul-2022
https://dl.acm.org/doi/10.1109/CEC55065.2022.9870211
Liu CLee C(2022)On the structured backward error of inexact Arnoldi methods for (skew)-Hermitian and (skew)-symmetric eigenvalue problemsBIT10.1007/s10543-017-0660-257:4(1083-1108)Online publication date: 11-Mar-2022
https://dl.acm.org/doi/10.1007/s10543-017-0660-2
Ichimura TFujita KKoyama KKikuchi YKusakabe RMinami KInoue HNishizawa STsuji MNishiki THori MMaddegedara LUeda NRobinson T(2021)Fast scalable implicit solver with convergence of equation-based modeling and data-driven learningProceedings of the Platform for Advanced Scientific Computing Conference10.1145/3468267.3470616(1-12)Online publication date: 5-Jul-2021
https://dl.acm.org/doi/10.1145/3468267.3470616
Tu JClark MJung CMawhinney RRobinson T(2021)Solving DWF dirac equation using multi-splitting preconditioned conjugate gradient with tensor cores on NVIDIA GPUsProceedings of the Platform for Advanced Scientific Computing Conference10.1145/3468267.3470613(1-11)Online publication date: 5-Jul-2021
https://dl.acm.org/doi/10.1145/3468267.3470613
Flegar GAnzt HCojean TQuintana-Ortí E(2021)Adaptive Precision Block-Jacobi for High Performance Preconditioning in the Ginkgo Linear Algebra SoftwareACM Transactions on Mathematical Software10.1145/344185047:2(1-28)Online publication date: 26-Apr-2021
https://dl.acm.org/doi/10.1145/3441850
Mousavirad SOliva DHinojosa SSchaefer G(2021)Differential Evolution-based Neural Network Training Incorporating a Centroid-based Strategy and Dynamic Opposition-based Learning2021 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC45853.2021.9504801(1233-1240)Online publication date: 28-Jun-2021
https://dl.acm.org/doi/10.1109/CEC45853.2021.9504801
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

References

Cited By

Index Terms

Recommendations

Scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization

Preconditioned AHSS iteration method for singular saddle point problems

Restrictive preconditioners for conjugate gradient methods for symmetric positive definite linear systems

Comments

Information

Published In

Publisher

Publication History

Author Tags

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations