article

Flexible Variants of Block Restarted GMRES Methods with Application to Geophysics

Authors:

Henri Calandra,

Xavier VasseurAuthors Info & Claims

SIAM Journal on Scientific Computing, Volume 34, Issue 2

Pages 714 - 736

https://doi.org/10.1137/10082364X

Published: 01 March 2012 Publication History

Abstract

In a wide number of applications in computational science and engineering the solution of large linear systems of equations with several right-hand sides given at once is required. Direct methods based on Gaussian elimination are known to be especially appealing in that setting. Nevertheless, when the dimension of the problem is very large, preconditioned block Krylov space solvers are often considered as the method of choice. The purpose of this paper is thus to present iterative methods based on block restarted GMRES that allow variable preconditioning for the solution of linear systems with multiple right-hand sides. The use of flexible methods is especially of interest when approximate possibly iterative solvers are considered in the preconditioning phase. First we introduce a new variant of block flexible restarted GMRES that includes a strategy for detecting when a linear combination of the systems has approximately converged. This explicit block size reduction is often called deflation. We analyze the main properties of this flexible method based on deflation and notably prove that the Frobenius norm of the block residual is always nonincreasing. We also present a flexible variant based on both deflation and truncation to especially be used in case of limited memory. Finally we illustrate the numerical behavior of these flexible block methods for large industrial simulations arising in geophysics, where indefinite linear systems of size up to 1 billion unknowns with multiple right-hand sides have been successfully solved in a parallel distributed memory environment.

References

[1]

J. I. Aliaga, D. L. Boley, R. W. Freund, and V. Hernández, A Lanczos-type method for multiple starting vectors, Math. Comp., 69 (2000), pp. 1577-1601.

[2]

F. Aminzadeh, J. Brac, and T. Kunz, $3$D Salt and Overthrust models, in SEG/EAGE Modeling Series I, Society of Exploration Geophysicists, 1997.

[3]

O. Axelsson and P. S. Vassilevski, A black box generalized conjugate gradient solver with inner iterations and variable-step preconditioning, SIAM J. Matrix Anal. Appl., 12 (1991), pp. 625-644.

[4]

J. Baglama, D. Calvetti, G. H. Golub, and L. Reichel, Adaptively preconditioned GMRES algorithms, SIAM J. Sci. Comput., 20 (1998), pp. 243-269.

[5]

Z. Bai, D. Day, and Q. Ye, ABLE: An adaptive block Lanczos for non Hermitian eigenvalue problems, SIAM J. Matrix Anal. Appl., 20 (1999), pp. 1060-1082.

[6]

A. H. Baker, J. M. Dennis, and E. R. Jessup, An efficient block variant of GMRES, SIAM J. Sci. Comput., 27 (2006), pp. 1608-1626.

[7]

A. H. Baker, R. D. Falgout, T. V. Kolev, and U. M. Yang, Multigrid smoothers for ultraparallel computing, SIAM J. Sci. Comput., 33 (2011), pp. 2864-2887.

[8]

J.-P. Berenger, A perfectly matched layer for absorption of electromagnetic waves, J. Comput. Phys., 114 (1994), pp. 185-200.

[9]

J.-P. Berenger, Three-dimensional perfectly matched layer for absorption of electromagnetic waves, J. Comput. Phys., 127 (1996), pp. 363-379.

[10]

P. A. Businger and G. Golub, Linear least squares solutions by Householder transformations, Numer. Math., 7 (1965), pp. 269-276.

[11]

G. Cohen, Higher-Order Numerical Methods for Transient Wave Equations, Springer, New York, 2002.

[12]

J. Cullum and T. Zhang, Two-sided Arnoldi and non-symmetric Lanczos algorithms, SIAM J. Matrix Anal. Appl., 24 (2002), pp. 303-319.

[13]

L. Elbouyahyaoui, A. Messaoudi, and H. Sadok, Algebraic properties of the block GMRES and block Arnoldi methods, Electron. Trans. Numer. Anal., 33 (2009), pp. 207-220.

[14]

H. Elman, O. Ernst, D. O'Leary, and M. Stewart, Efficient iterative algorithms for the stochastic finite element method with application to acoustic scattering, Comput. Methods Appl. Mech. Engrg., 194 (2005), pp. 1037-1055.

[15]

J. Erhel, K. Burrage, and B. Pohl, Restarted GMRES preconditioned by deflation, J. Comput. Appl. Math., 69 (1996), pp. 303-318.

[16]

R. W. Freund and M. Malhotra, A block QMR algorithm for non-Hermitian linear systems with multiple right-hand sides, Linear Algebra Appl., 254 (1997), pp. 119-157.

[17]

G. H. Golub and C. F. V. Loan, Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, 1996.

[18]

A. E. Guennouni, K. Jbilou, and H. Sadok, A block version of BICGSTAB for linear systems with multiple right-hand sides, Electron. Trans. Numer. Anal., 16 (2003), pp. 129-142.

[19]

M. H. Gutknecht and T. Schmelzer, The block grade of a block Krylov space, Linear Algebra Appl., 430 (2009), pp. 174-185.

[20]

M. H. Gutknecht, Block Krylov space methods for linear systems with multiple right-hand sides: An introduction, in Modern Mathematical Models, Methods and Algorithms for Real World Systems, A. Siddiqi, I. Duff, and O. Christensen, eds., Anamaya Publishers, New Delhi, India, 2006, pp. 420-447.

[21]

G.-D. Gu and Z. Cao, A block GMRES method augmented with eigenvectors, Appl. Math. Comput., 121 (2001), pp. 271-289.

[22]

N. J. Higham, Functions of Matrices: Theory and Computation, SIAM, Philadelphia, 2008.

[23]

Y. P. Hong and C. T. Pan, Rank revealing QR factorizations and the singular value decomposition, Math. Comp., 58 (1992), pp. 213-232.

[24]

R. Horn and C. R. Johnson, Topics in Matrix Analysis, Cambridge University Press, Cambridge, UK, 1991.

[25]

I. M. Jaimoukha and E. M. Kasenally, Krylov subspace methods for solving large Lyapounov equations, SIAM J. Numer. Anal., 31 (1994), pp. 227-251.

[26]

K. Jbilou, A. Messaoudi, and H. Sadok, Global FOM and GMRES algorithms for matrix equations, Appl. Numer. Math., 31 (1999), pp. 49-63.

[27]

J. Langou, Iterative Methods for Solving Linear Systems with Multiple Right-Hand Sides, Ph.D. thesis, CERFACS, 2003.

[28]

J. Langou, For a few iterations less, Eighth Copper Mountain Conference on Iterative Methods, Copper Mountain, CO, 2004.

[29]

G. Li, A block variant of the GMRES method on massively parallel processors, Parallel Comput., 23 (1997), pp. 1005-1019.

[30]

D. Loher, Reliable Nonsymmetric Block Lanczos Algorithms, Ph.D. thesis, Swiss Federal Institute of Technology Zurich (ETHZ), Switzerland, 2006.

[31]

R. B. Morgan, Restarted block GMRES with deflated restarting, Appl. Numer. Math., 54 (2005), pp. 222-236.

[32]

A. A. Nikishin and A. Y. Yeremin, Variable block CG algorithms for solving large sparse symmetric positive definite linear systems on parallel computers, I: General iterative scheme, SIAM J. Matrix Anal. Appl., 16 (1995), pp. 1135-1153.

[33]

Y. Notay and P. S. Vassilevski, Recursive Krylov-based multigrid cycles, Numer. Linear Algebra Appl., 15 (2008), pp. 473-487.

[34]

Y. Notay, Flexible conjugate gradients, SIAM J. Sci. Comput., 22 (2000), pp. 1444-1460.

[35]

D. P. O'Leary, The block conjugate gradient algorithm and related methods, Linear Algebra Appl., 29 (1980), pp. 293-322.

[36]

X. Pinel, A Perturbed Two-Level Preconditioner for the Solution of Three-Dimensional Heterogeneous Helmholtz Problems with Applications to Geophysics, Ph.D. thesis, CERFACS, 2010.

[37]

M. Robbé and M. Sadkane, Exact and inexact breakdowns in the block GMRES method, Linear Algebra Appl., 419 (2006), pp. 265-285.

[38]

A. Ruhe, Implementation aspects of band Lanczos algorithms for computation of eigenvalues of large sparse symmetric matrices, Math. Comp., 33 (1979), pp. 680-687.

[39]

Y. Saad, A flexible inner-outer preconditioned GMRES algorithm, SIAM J. Sci. Statist. Comput., 14 (1993), pp. 461-469.

[40]

V. Simoncini and E. Gallopoulos, A hybrid block GMRES method for nonsymmetric systems with multiple right-hand sides, J. Comput. Appl. Math., 66 (1995), pp. 457-469.

[41]

V. Simoncini and E. Gallopoulos, An iterative method for nonsymmetric systems with multiple right-hand sides, SIAM J. Sci. Comput., 16 (1995), pp. 917-933.

[42]

V. Simoncini and E. Gallopoulos, Convergence properties of block GMRES and matrix polynomials, Linear Algebra Appl., 247 (1996), pp. 97-119.

[43]

V. Simoncini and D. B. Szyld, Flexible inner-outer Krylov subspace methods, SIAM J. Numer. Anal., 40 (2003), pp. 2219-2239.

[44]

P. Soudais, Iterative solution methods of a $3$-D scattering problem from arbitrary shaped multidielectric and multiconducting bodies, IEEE Trans. Antennas and Propagation, 42 (1994), pp. 954-959.

[45]

F. Sourbier, S. Operto, J. Virieux, P. Amestoy, and J. Y. L. Excellent, FWT$2$D: A massively parallel program for frequency-domain full-waveform tomography of wide-aperture seismic data, part 1: Algorithm, Comput. Geosci., 35 (2009), pp. 487-495.

[46]

F. Sourbier, S. Operto, J. Virieux, P. Amestoy, and J. Y. L. Excellent, FW$2$D : A massively parallel program for frequency-domain full-waveform tomography of wide-aperture seismic data, part 2: Numerical examples and scalability analysis, Comput. Geosci., 35 (2009), pp. 496-514.

[47]

D. B. Szyld and J. A. Vogel, FQMR: A flexible quasi-minimal residual method with inexact preconditioning, SIAM J. Sci. Comput., 23 (2001), pp. 363-380.

[48]

A. Toselli and O. Widlund, Domain Decomposition methods: Algorithms and Theory, Springer Ser. Comput. Math. 34, Springer, New York, 2004.

[49]

U. Trottenberg, C. W. Oosterlee, and A. Schüller, Multigrid, Academic Press, New York, 2001.

[50]

H. A. van der Vorst and C. Vuik, GMRESR: A family of nested GMRES methods, Numer. Linear Algebra Appl., 1 (1994), pp. 369-386.

[51]

J. Virieux and S. Operto, An overview of full waveform inversion in exploration geophysics, Geophys., 74 (2009), pp. WCC127-WCC152.

[52]

B. Vital, Etude de Quelques Méthodes de Résolution de Problème Linéaire de Grande Taille sur Multiprocesseur, Ph.D. thesis, Université de Rennes, 1990.

Cited By

Jiang NYang H(2023)Artificial compressibility SAV ensemble algorithms for the incompressible Navier-Stokes equationsNumerical Algorithms10.1007/s11075-022-01382-z92:4(2161-2188)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s11075-022-01382-z
Jiang NYang H(2023)Fast and Accurate Artificial Compressibility Ensemble Algorithms for Computing Parameterized Stokes–Darcy Flow EnsemblesJournal of Scientific Computing10.1007/s10915-022-02069-294:1Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s10915-022-02069-2
Jiang NYang H(2022)Numerical investigation of two second-order, stabilized SAV ensemble methods for the Navier–Stokes equationsAdvances in Computational Mathematics10.1007/s10444-022-09977-948:5Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1007/s10444-022-09977-9
Show More Cited By

Recommendations

A Modified Block Flexible GMRES Method with Deflation at Each Iteration for the Solution of Non-Hermitian Linear Systems with Multiple Right-Hand Sides
^† Special Section: 2012 Copper Mountain Conference

We propose a variant of the block GMRES method for the solution of linear systems of equations with multiple right-hand sides. We investigate a deflation strategy to detect when a linear combination of approximate solutions is already known that avoids ...
Deflated and Restarted Symmetric Lanczos Methods for Eigenvalues and Linear Equations with Multiple Right-Hand Sides

A deflated restarted Lanczos algorithm is given for both solving symmetric linear equations and computing eigenvalues and eigenvectors. The restarting limits the storage so that finding eigenvectors is practical. Meanwhile, the deflating from the ...
Restarted block-GMRES with deflation of eigenvalues
6th IMACS International symposium on iterative methods in scientific computing

Block-GMRES is an iterative method for solving nonsymmetric systems of linear equations with multiple right-hand sides. Restarting may be needed, due to orthogonalization expense or limited storage. We discuss how restarting affects convergence and the ...

Comments

Information & Contributors

Information

Published In

cover image SIAM Journal on Scientific Computing

SIAM Journal on Scientific Computing Volume 34, Issue 2

April 2012

829 pages

ISSN:1064-8275

Issue’s Table of Contents

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 01 March 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jiang NYang H(2023)Artificial compressibility SAV ensemble algorithms for the incompressible Navier-Stokes equationsNumerical Algorithms10.1007/s11075-022-01382-z92:4(2161-2188)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s11075-022-01382-z
Jiang NYang H(2023)Fast and Accurate Artificial Compressibility Ensemble Algorithms for Computing Parameterized Stokes–Darcy Flow EnsemblesJournal of Scientific Computing10.1007/s10915-022-02069-294:1Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s10915-022-02069-2
Jiang NYang H(2022)Numerical investigation of two second-order, stabilized SAV ensemble methods for the Navier–Stokes equationsAdvances in Computational Mathematics10.1007/s10444-022-09977-948:5Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1007/s10444-022-09977-9
Tajaddini AWu GSaberi-Movahed FAzizizadeh N(2021)Two New Variants of the Simpler Block GMRES Method with Vector Deflation and Eigenvalue Deflation for Multiple Linear SystemsJournal of Scientific Computing10.1007/s10915-020-01376-w86:1Online publication date: 2-Jan-2021
https://dl.acm.org/doi/10.1007/s10915-020-01376-w
Zhong HWu GChen G(2019)A flexible and adaptive simpler block GMRES with deflated restarting for linear systems with multiple right-hand sidesJournal of Computational and Applied Mathematics10.1016/j.cam.2014.12.040282:C(139-156)Online publication date: 3-Jan-2019
https://dl.acm.org/doi/10.1016/j.cam.2014.12.040
Sun DHuang TCarpentieri BJing Y(2019)A New Shifted Block GMRES Method with Inexact Breakdowns for Solving Multi-Shifted and Multiple Right-Hand Sides Linear SystemsJournal of Scientific Computing10.1007/s10915-018-0787-678:2(746-769)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1007/s10915-018-0787-6
Jolivet PTournier PWest J(2016)Block iterative methods and recycling for improved scalability of linear solversProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis10.5555/3014904.3014927(1-14)Online publication date: 13-Nov-2016
https://dl.acm.org/doi/10.5555/3014904.3014927
Meng JLi HJing Y(2016)A new deflated block GCROT( m , k ) method for the solution of linear systems with multiple right-hand sidesJournal of Computational and Applied Mathematics10.1016/j.cam.2015.12.029300:C(155-171)Online publication date: 1-Jul-2016
https://dl.acm.org/doi/10.1016/j.cam.2015.12.029
Meng JZhu PLi H(2014)A block GCROT ( m , k ) method for linear systems with multiple right-hand sidesJournal of Computational and Applied Mathematics10.5555/2753878.2754054255:C(544-554)Online publication date: 1-Jan-2014
https://dl.acm.org/doi/10.5555/2753878.2754054
Meng JZhu PLi H(2014)A block GCROT(m,k) method for linear systems with multiple right-hand sidesJournal of Computational and Applied Mathematics10.5555/2532869.2532968255(544-554)Online publication date: 1-Jan-2014
https://dl.acm.org/doi/10.5555/2532869.2532968

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents